Generic Connector¶

The generic Connector of Beemaster is responsible for connecting a honeypot with the Bro cluster. The Connecter generally accepts JSON-formatted input via HTTP, maps it to Broker-compatible data types and sends a Broker-message to a configurable endpoint.

You are highly encouraged to always run the Connector on the same machine as the corresponding honeypot, use only one Connector per honeypot and hide the Connector from the outside.

The following topics are be discussed:

Configuration of the Connector
Usage with and without Docker
Setup development environment

Configuration¶

Connection¶

Connection-related settings of the Connecter can be configured by passing arguments at start or specifying a configuration file. The following arguments are supported:

positional arguments:
  file                  Configuration file to use.

optional arguments:
  -h, --help              Show this help message and exit
  --laddr address         Address to listen on.
  --lport port            Port to listen on.
  --saddr address         Address to send to.
  --sport port            Port to send to.
  --mappings directory    Directory to look for mappings.
  --topic topic           Topic for sent messages.
  --endpoint_prefix name  Prefix name for the Broker endpoint.
  --id connector_id       This connector's unique id.
  --log-file file         The file to log to. 'stderr' and 'stdout' work as special names for standard-error and -out respectively.
  --log-level level       Set the log-level. {'INFO', 'DEBUG', 'WARNING', 'ERROR', 'CRITICAL'}
  --log-datefmt format    Set the date/time format to use for logging ('asctime' placeholder in log-format). Python's strftime format is used.
  --log-format format     Set the logging format. Use the 'asctime' placeholder to include the date. See the python docs for more information on this.

The positional argument accepts configuration files written in YAML with the following format:

listen:
    address: 0.0.0.0                        # Address to listen on.
    port: 8080                              # Port to listen on.
send:
    address: 127.0.0.1                      # Address to send to.
    port: 5000                              # Port to send to.
mappings: mappings                          # Directory to look for mappings.
broker:
    topic: honeypot/dionaea/                # Topic for sent messages.
    endpoint_prefix: beemaster-connector-   # Prefix name for the broker endpoint.

The values shown in the example above are the default values the Connecter falls back to, in case no arguments are passed.

By default, the Connecter uses the hostname to identify itself. You can change it to whatever name you like, but it must be a unique name in your network:

connector_id: my_unique_connector_name       # Remove this to use the hostname by default

Furthermore, the Connecter is able to write logs. Just let him know in what information you are interested in:

logging:
    file: stderr
    level: ERROR
    datefmt: None
    format: "[ %(asctime)s | %(name)10s | %(levelname)8s ] %(message)s"

Tip: Writing the INFO level to stdout or a file, mounted by the host to the Docker container, makes it easier to see the traffic throughput of the Connecter.

Mapping¶

The mappings used by the Connecter are configurable via YAML files. Below is an example of a mapping for a Dionaea access event:

# one file per access type
#
# name: name of the event (for bro)
# mapping: the structure of the arriving json
#          to map to the desired broker-type
#          (implemented in mapper)
# message: the structure of the message, as it
#          will be put into the broker message
name: dionaea_access
mapping:
    timestamp: time_point
    src_hostname: string
    src_ip: address
    src_port: port_count
    dst_ip: address
    dst_port: port_count
    connection:
        type: string
        protocol: string
        transport: string
message:
    - timestamp
    - dst_ip
    - dst_port
    - src_hostname
    - src_ip
    - src_port
    - transport
    - protocol

Once you create a mapping, be sure to create the corresponding event handler on the Bro side of the connection.

Usage¶

The Connector can be used within a Docker container or locally for testing. We advise you to run the Connector always on the same host as the Dionaea honeypot.

Important: Make sure that the configured Broker endpoint (send to) is available as the Connector will not accept data on its listening port (listen from) otherwise.

Note: We provide a setup for a Raspberry Pi. The official project documentation (in German) describes the setup.

Docker Setup¶

If you want to use the Connector in conjunction with Dionaea, you can use the following compose file (and make sure all directories are present, or change them accordingly).

By default (inside the container), the contents of the conf directory are copied into the src directory. Thus the connector.py can be started by passing the configuration filename directly (see the docker-compose file example below):

version: '2'
services:
  connector:
    build: ./mp-ids-hp/connector
    command: ["config-docker.yaml"]
  dionaea:
    build: ./mp-ids-hp/dionaea
    ports:
      - "21:21"
      - "23:23"
      - "53:53/udp"
      - "53:53"
      - "80:80"
      - "123:123/udp"
      - "443:443"
      - "445:445"
      - "3306:3306"

Run docker-compose up --build.

Be sure to expose only those ports of Dionaea you want to be accessible for attackers.

Instead of passing config-docker.yaml as an argument (which is a configuration adjusted for this compose file), you could also pass your own values, e.g.:

  connector:
    command: ["--sport","1337","--topic","leetevent/"]

You can also run the Connecter as a standalone container by appending the correct arguments to the docker run command:

docker build -t connector .
docker run connector --sport 1337 --topic leetevent/

For testing purposes, you might want to run Dionaea and the Connector together with Bro. Have a look at the compose file of the Beemaster Bro repository. There you can find a simple Bro cluster setup.

Without Docker¶

Ensure that you sourced the virtual environment as described here. The Connector needs to run on Python 2 and requires modules which are not located in the src folder but included in the environment.

Start the Connector via python connector.py and use the correct arguments for your environment. This repository holds a configuration file that can be used for local testing, which is identical to the default configuration, apart from sending to port 9999.

python connector.py ../conf/config-local.yaml

It will bind to port 8080 and listen for JSON post input. See the Dionaea readme for information about how to communicate with it.

Setup Development Environment¶

The setup.sh script can be used to set up a development and testing environment.

./setup.sh -h
# Usage: ./setup.sh -h
#        ./setup.sh [-i|u] [-d] [-s]
#        ./setup.sh -c
#
# Options:
#     -h      Print this help message and exit.
#     -i      Install virtualenv, if not installed.
#     -u      Update environment(s) with new requirements.
#     -d      Setup development environment (for linting etc.)
#     -s      Adds symlinks for easier environment sourcing.
#     -c      Removes everything, created by this setup and exits.

Development¶

Execute ./setup.sh -d to setup all required environments. flake8 will be accessible in the main directory so you do not necessarily need to source the environment to use the linter (if sourced, flake8 will be in your PATH).

Execution Only¶

If you only need to run the code, use ./setup.sh to install the minimal environment.

Source the environment with . env/bin/activate (or use the symlink, provided by ./setup.sh -s). Be aware, that the activation only applies for the current shell.