search

BlackZork / mqmgateway

MQTT gateway for modbus networks
GNU Affero General Public License v3.0
42 stars 18 forks source link
arm gateway linux modbus mqtt

readme

MQMGateway - MQTT gateway for modbus networks

A multithreaded C++ service that exposes data from multiple Modbus networks as MQTT topics.

docker build

Main features:

MQMGateway depends on libmodbus and Mosquitto MQTT library. See main CMakeLists.txt for full list of dependencies. It is developed under Linux, but it should be easy to port it to other platforms.

License

This software is dual-licensed:

For a commercial-friendly license and support please see http://mqmgateway.zork.pl.

Third-party licenses

This software includes "A single-producer, single-consumer lock-free queue for C++" written by Cameron Desrochers. See license terms in LICENSE.md

Installation

From sources

  1. git clone https://github.com/BlackZork/mqmgateway.git#branch=master

    You can aslo use branch=<tagname> to clone specific release or download sources from Releases page

  2. Install dependencies:

    1. boost
    2. libmodbus
    3. mosquitto
    4. yaml-cpp
    5. rapidJSON
    6. exprtk (optional, for exprtk expressions language support in yaml declarations)
    7. Catch2 (optional, for unit tests)

  3. Configure and build project:

    cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr -S (project dir) -B (build dir)
make
make install

    You can add -DWITHOUT_TESTS=1 to skip build of unit test executable.

  4. Copy config.template.yaml to /etc/modmqttd/config.yaml and adjust it.

  5. Copy modmqttd.service to /etc/systemd/system and start service:

  systemctl start modmqttd

Docker image

Docker images for various archicetures (i386, arm6, arm7, amd64) are available in packages section.

  1. Pull docker image using instructions provided in packages section.

  2. Copy config.template.yaml and example docker-compose file to working directory

  3. Edit and rename config.template.yaml to config.yaml. In docker-compose.yml adjust devices section to provide serial modbus devices from host to docker container.

  4. Run docker-compose up -d in working directory to start service.

Logging

modqmttd has six log levels: CRITICAL, ERROR, WARNING, INFO, DEBUG, TRACE, numbered from 1 to 6. When debugging, you can increase the default INFO log level by passing --loglevel to modmqttd:

modmqttd --config=<path> --loglevel=5

DEBUG is more useful for general troubleshotting, TRACE generates a lot of output and is not recommended for production use.

Configuration

modmqttd configuration file is in YAML format. It is divided into three main sections:

For quick example see config.template.yaml in source directory.

Configuration values

modmqttd section

modbus section

Modbus section contains a list of modbus networks modmqttd should connect to. Modbus network configuration parameters are listed below:

MQTT section

The mqtt section contains broker definition and modbus register mappings. Mappings describe how modbus data should be published as mqtt topics.

A list of topics where modbus values are published to MQTT broker and subscribed for writing data received from MQTT broker to modbus registers.

Topic default values:

A commands section.

A single command is defined using following settings.

The state section

The state sections defines how to publish modbus data to MQTT broker. State can be mapped to a single register, an unnamed and a named list of registers. Following table shows what kind of output is generated for each type:

Value type Default output
single register uint16_t register data as string
unnamed list JSON array with uint16_t register data as string
named list JSON map with values as uint16_t register data as string

It is also possible to combine and output an unnamed list of registers as a single value using converter. See converters section for details.

Register list can be defined in two ways:

  1. As starting register and count:

    state:
name: mqtt_combined_val
converter: std.int32
register: net.1.12
count: 2

    This declaration creates a poll group. Poll group is read from modbus slave using a single modbus_read_registers(3) call. Overlapping poll groups are merged with each other and with poll groups defined in modbus section.

  2. as list of registers:

    state:
- name: humidity
  register: net.1.12
  register_type: input
  # optional
  converter: std.divide(100,2)
- name:  temp1
  register: net.1.300
  register_type: input

    This declaration do not create a poll group, but allows to construct MQTT topic data from different slaves, even on different modbus networks. On exception is if there are poll groups defined in modbus section, that overlaps state register definitions. In this case data is polled using poll group.

    • refresh

    Overrides mqtt.refresh for this state topic

    When state is a single modbus register value:

    • name

    The last part of topic name where value should be published. Full topic name is created as topic_name/state_name

    • register (required)

    Modbus register address in the form of .. If register_number is a decimal, then first register address is 1. If register_number is a hexadecimal, then first register address is 0.

    network_name and slave_id are optional if default values are set for a topic

    • register_type (optional, default: holding)

    Modbus register type: coil, bit, input, holding

    • count (optional, default: 1)

      If defined, then this describes register range to poll. Register range is always polled with a single modbus_read_registers(3) call

    • converter (optional)

    The name of function that should be called to convert register uint16_t value to MQTT UTF-8 value. Format of function name is plugin_name.function_name. See converters for details.

    The following examples show how to combine name, register, register_type, and converter to output different state values:

  3. single value

    state:
name: mqtt_val
register: net.1.12
register_type: coil

  4. unnamed list, each register is polled with a separate modbus_read_registers call

    state:
name: mqtt_list
registers:
  - register: net.1.12
    register_type: input
  - register: net.1.100
    register_type: input

  5. multiple registers converted to single MQTT value, polled with single modbus_read_registers call

    state:
name: mqtt_combined_val
converter: std.int32
register: net.1.12
count: 2

  6. named list (map)

    state:
- name: humidity
  register: net.1.12
  register_type: input
  # optional
  converter: std.divide(100,2)
- name:  temp1
  register: net.1.13
  register_type: input

In all of above examples refresh can be added at any level to set different values to whole list or a single register.

Lists and maps can be nested if needed:

state:
  - name: humidity
    register: net.1.12
    count: 2
    converter: std.float()
  - name: other_params
    registers:
      - name: "temp1"
        register: net.1.14
        count: 2
        converter: std.int32()
      - name: "temp2"
        register: net.1.16
        count: 2
        converter: std.int32()

MQTT output: {"humidity": 32.45, "other_params": { "temp1": 23, "temp2": 66 }}

The availability section

For each state topic there is another availability topic defined by default. If all data required for a state is read from the Modbus registers without errors, the value "1" is published by default. If there is a network or device error when polling register data value "0" is published. This is the default behavior if the availability section is not defined.

Availablity flag is always published after the state value. If the availability flag is 0, then the current state value may contain outdated or invalid data or may not be published at all.

Availability section extends this default behaviour by defining a single or list of modbus registers that should be read to check if state data is valid. This could be i.e. some fault indicator or hardware switch state.

Configuration values:

register, register_type can form a registers: list when multiple registers should be read. In this case converter is mandatory and no nesting is allowed. See examples in state section.

Data conversion

Data read from modbus registers is by default converted to string and published to MQTT broker.

MQMGateway uses conversion plugins to convert state data read from modbus registers to mqtt value and command mqtt payload to register value, for example to combine multiple modbus registers into single value, use mask to extract one bit, or perform some simple divide operations.

Converter can also be used to convert mqtt command payload to register value.

String converter arguments can be passed in single or double quotes.

Standard converters

Converter functions are defined in libraries dynamically loaded at startup. MQMGateway contains std library with basic converters ready to use:

Converter usage examples

Converter can be added to modbus register in state and command section.

When a state is a single modbus register:

  state:
    register: device1.slave2.12
    register_type: input
    converter: std.divide(10,2)

When a state is combined from multiple modbus registers:

  state:
    register: device1.slave2.12
    register_type: input
    count: 2
    converter: std.int32()

When a mqtt command payload should be converted to register value:

  commands:
    - name: set_val
      register: device1.slave2.12
      register_type: input
      converter: std.divide(10)

When an availability value should be computed from multiple registers:

  availability:
    register: device1.slave2.12
    register_type: input
    count: 2
    converter: std.int32()
    available_value: 65537

Exprtk converter.

Exprtk converter allows to use exprtk expression language to convert register data to mqtt value. Register values are defined as R0..Rn variables.

Examples

Division of two registers with precision 3:

  objects:
    - topic: test_state
      state:
        converter: expr.evaluate("R0 / R1", 3)
        registers:
          - register: tcptest.1.2
            register_type: input
          - register: tcptest.1.300
            register_type: input

Reading the state of a 32-bit float value (byte order ABCD) spanning two registers (R0 = BA, R1 = DC) with precision 3:

  objects:
    - topic: test_state
      state:
        converter: expr.evaluate("flt32be(R0, R1)", 3)
        register: tcptest.1.2
        register_type: input
        count: 2

Adding custom converters

Custom converters can be added by creating a C++ dynamically loaded library with conversion classes. There is a header only libmodmqttconv that provide base classes for plugin and converter implementations.

Here is a minimal example of custom conversion plugin with help of boost dll library loader:


#include <boost/config.hpp> // for BOOST_SYMBOL_EXPORT
#include "libmodmqttconv/converterplugin.hpp"

class MyConverter : public DataConverter {
    public:
        //called by modmqttd to set coverter arguments
        virtual void setArgs(const std::vector<std::string>& args) {
            mShift = getIntArg(0, args);
        }

        // Conversion from modbus registers to mqtt value
        // Used when converter is defined for state topic
        // ModbusRegisters contains one register or as many as
        // configured in unnamed register list.
        virtual MqttValue toMqtt(const ModbusRegisters& data) const {
            int val = data.getValue(0);
            return MqttValue::fromInt(val << mShift);
        }

        // Conversion from mqtt value to modbus register data
        // Used when converter is defined for command topic
        virtual ModbusRegisters toModbus(const MqttValue& value, int registerCount) const {
            int val = value.getInt();
            ModbusRegisters ret;
            for (int i = 0; i < registerCount; i++) {
              val = val >> mShift
              ret.prependValue(val);
            }
            return ret;
        }

        virtual ~MyConverter() {}
    private:
      int mShift = 0;
};

class MyPlugin : ConverterPlugin {
    public:
        // name used in configuration as plugin prefix.
        virtual std::string getName() const { return "myplugin"; }
        virtual IStateConverter* getStateConverter(const std::string& name) {
            if (name == "myconverter")
                return new MyConverter();
            return nullptr;
        }
        virtual ~MyPlugin() {}
};

// modmqttd search for "converter_plugin" C symbol in loaded dll
extern "C" BOOST_SYMBOL_EXPORT MyPlugin converter_plugin;
MyPlugin converter_plugin;

Compilation on linux:

g++ -I<path to mqmgateway source dir> -fPIC -shared myplugin.cpp -o myplugin.so

myconverter from this example can be used like this:

modmqttd:
  converter_search_path:
    - <myplugin.so dir>
  converter_plugins:
    - myplugin.so
modbus:
  networks:
    - name: tcptest
      address: localhost
      port: 501
mqtt:
  objects:
    - topic: test_topic
    command:
      name: set_val
      register: tcptest.2.12
      register_type: input
      converter: myplugin.myconverter(1)
    state:
      name: test_val
      register: tcptest.2.12
      register_type: input
      converter: myplugin.myconverter(1)

For more examples see libstdconv source code.

Multi-device definitions

Multi-device defintions allows to set slave properties or create a single topic for multiple modbus devices of the same type. This greatly reduces the number of configuration sections that differ only by slave address or modbus network name.

Multi-device MQTT topics

If there are many devices of the same type then a MQTT topic for group of devices can be defined by setting slave value to list of modbus slave addresses. Then you have to add either slave_name or slave_address placeholder in the topic string like this:

modbus:
  networks:
    - name: basement
      slaves:
        - address: 1
          name: meter1
        - address: 2
          name: meter2
        [...]
mqtt:
  objects:
    - topic: ${network}/${slave_name}/node_${slave_address}
      slave: 1,2,3,8-9
      network: basement, roof
      state:
        register: 1

In the above example 10 registers will be polled and their values will be published as /basement/meter1/node_1/state, /basement/meter2/node_2/state and so on.

Slave names are required only if ${slave_name} placeholder is used.

Multi device modbus slave definitions.

Address list can be used in modbus.networks.slaves.address to define properties for multiple slaves at once:

modbus:
  networks:
    - name: basement
      slaves:
        - address: 1,2,3,5-18
          poll_groups:
            - register: 3
              count: 10
            - register: 30
              count: 5

A single slave address can be listed in multiple entries like this:

modbus:
  networks:
    - name: basement
      slaves:
        - address: 1
          name: meter1
          response_timeout: 50ms
        - address: 2
          name: meter2
        - address: 1,2
          response_timeout: 100ms
          poll_groups:
            - register: 3
              count: 10

This example will set response timeout 100ms for both slaves - overriding value for slave1. Two poll groups are defined for reading registers 3-13 for both slaves.