A multithreaded C++ service that exposes data from multiple Modbus networks as MQTT topics.
Main features:
MQMGateway depends on libmodbus and Mosquitto MQTT library. See main CMakeLists.txt for full list of dependencies. It is developed under Linux, but it should be easy to port it to other platforms.
This software is dual-licensed:
For a commercial-friendly license and support please see http://mqmgateway.zork.pl.
This software includes "A single-producer, single-consumer lock-free queue for C++" written by Cameron Desrochers. See license terms in LICENSE.md
git clone https://github.com/BlackZork/mqmgateway.git#branch=master
You can aslo use branch=<tagname>
to clone specific release or download sources from Releases page
Install dependencies:
Configure and build project:
cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr -S (project dir) -B (build dir)
make
make install
You can add -DWITHOUT_TESTS=1 to skip build of unit test executable.
Copy config.template.yaml to /etc/modmqttd/config.yaml and adjust it.
Copy modmqttd.service to /etc/systemd/system and start service:
systemctl start modmqttd
Docker images for various archicetures (i386, arm6, arm7, amd64) are available in packages section.
Pull docker image using instructions provided in packages section.
Copy config.template.yaml and example docker-compose file to working directory
Edit and rename config.template.yaml to config.yaml. In docker-compose.yml adjust devices section to provide serial modbus devices from host to docker container.
Run docker-compose up -d
in working directory to start service.
modqmttd has six log levels: CRITICAL, ERROR, WARNING, INFO, DEBUG, TRACE, numbered from 1 to 6. When debugging, you can increase the default INFO log level by passing --loglevel
modmqttd --config=<path> --loglevel=5
DEBUG is more useful for general troubleshotting, TRACE generates a lot of output and is not recommended for production use.
modmqttd configuration file is in YAML format. It is divided into three main sections:
For quick example see config.template.yaml in source directory.
timespan format: "([0-9]+)(ms|s|min)"
Used for various timeout interval configuration entries
converter_search_path (optional)
List of paths where to search for converter plugins. If path does not start with '/' then it is treated as relative to current working directory
converter_plugins (optional)
List of converter plugins to load. Modmqttd search for plugins in all directories specified in converter_search_path list
Modbus section contains a list of modbus networks modmqttd should connect to. Modbus network configuration parameters are listed below:
name (required)
Unique name for network - referenced in mqtt mappings
response_timeout (optional, default 500ms)
A default timeout interval used to wait for modbus response. See modbus_set_response_timeout(3) for details.
response_data_timeout (optional, default 0)
A default timeout interval used to wait for data when reading response from modbus device. See modbus_set_byte_timeout(3) for details.
delay_before_first_command (timespan, optional, default 0ms)
Required silence period before issuing first modbus command to a slave. This delay is applied only when gateway switches to a diffrent slave.
This option is useful for RTU networks with a mix of very slow and fast responding slaves. Adding a delay before slow slave can significantly reduce amount of read erorrs if mqmgateway is configured to poll very fast.
delay_before_command (timespan, optional, default 0ms)
Same as delay_before_first_command, but a delay is applied to every modbus command sent on this network.
read_retries (optional, default 1)
A number of retries after a modbus read command fails. A failed command will trigger a publish of "0" value
to all availability topics for objects which needs command registers for state
or command
section.
write_retries (optional, default 2)
A number of retries after a modbus write command fails.
RTU device settings
For details, see modbus_new_rtu(3)
A path to modbus RTU device
The baud rate of the communication
N for none, E for even, O for odd
the number of data bits (5,6,7,8)
serial port stop bit value (0 or 1)
serial port mode: rs232 or rs485. See modbus_rtu_set_serial_mode(3)
modbus Request To Send mode (up or down). See modbus_rtu_set_rts(3)
modbus Request To Send delay period in microseconds. See modbus_rtu_set_rts_delay(3)
TCP/IP device settings
IP address of a device
TCP port of a device
watchdog (optional)
An optional configuration section for modbus connection watchdog. Watchdog monitors modbus command errors. If there is no successful command execution in watch_period, then it restarts the modbus connection.
Additionally for RTU network the device path is checked on the first error and then in small (300ms) time periods. If modbus RTU device is unplugged, then connection is restarted.
The amount of time after which the connection should be reestablished if there has been no successful execution of a modbus command.
slaves (optional) An optional slave list with modbus specific configuration like register groups to poll (see poll groups below) and timing constraints
address (required)
Modbus slave address. Multiple comma-separated values or ranges are supported like this:
1,2,3-10,12,30
name (optional)
Name for use in topic ${slave_name}
placeholder.
delay_before_first_command (timespan, optional)
Same as global delay_before_first_command but applies to this slave only.
delay_before_command (timespan, optional)
Same as global delay_before_command but applies to this slave only.
response_timeout (optional)
Overrides modbus.response_timeout for this slave
Overrides modbus.response_data_timeout for this slave
A number of retries after a modbus read command to this slave fails. Uses the global read_retries if not defined.
A number of retries after a modbus write command to this slave fails. Uses the global write_retries if not defined.
poll_groups (optional)
An optional list of modbus register address ranges that will be polled with a single modbus_read_registers(3) call.
An example poll group definition to poll 20 INPUT registers at once:
poll_groups:
- register: 1
register_type: input
count: 20
This definition allows to use single modbus read call to read all data that is needed for multiple topics declared in MQTT section. If there are no topics that use modbus data from a poll group then that poll group is ignored.
If MQTT topic uses its own register range and this range overlaps a poll group like this:
slaves:
- address: 1
poll_groups:
- register: 1
register_type: input
count: 20
[…]
state:
- name: humidity
register: 1.18
register_type: input
count 5
then poll group will be extended to count=23 to issue a single call for reading all data needed for humidity
topic in single modus read call.
The mqtt section contains broker definition and modbus register mappings. Mappings describe how modbus data should be published as mqtt topics.
client_id (required)
name used to connect to mqtt broker.
refresh (timespan, optional, default 5s)
A timespan used to poll modbus registers. This setting is propagated down to an object and register definitions. If this value is less than the target network can handle then newly scheduled read commands will be merged with those already in modbus command queue.
publish_mode (string, optional, default on_change)
A default mode for publishing mqtt values for all topics, that do not have their own publish_mode
declared.
broker (required)
This section contains configuration settings used to connect to MQTT broker.
MQTT boker IP address
MQTT broker TCP port
The number of seconds after which the bridge should send a ping if no other traffic has occurred.
The username to be used to connect to MQTT broker
The password to be used to connect to MQTT broker
This option enables TLS for connecting to MQTT broker
cafile (optional)
Path to a file containing the PEM encoded trusted CA certificate files. If this option is not set, OS provided CA certificates are used.
objects (required)
A list of topics where modbus values are published to MQTT broker and subscribed for writing data received from MQTT broker to modbus registers.
topic (required)
The base name for modbus register mapping. A mapping must contain at least one of following sections:
Topic can contain placeholders for modbus network and slave properies in form ${placeholder_name}
. Following placeholders are supported:
refresh (optional)
Overrides mqtt.refresh for all state and availability sections in this topic
network (optional)
Sets default network name for all state, commands and availability sections in this topic.
Multiple comma-separated values are supported. If more than one value is set, then you have to include ${network}
placeholder in topic
value.
slave (optional)
Sets default modbus slave address for all state, commands and availability sections in this topic.
Multiple values are supported as comma-separated list of numbers or number ranges like this:
1,2,3,5-18,21
If more than one value is set, then you have to include ${slave_address}
or ${slave_name}
in topic
value.
For examples see Multi-device definitions section.
publish_mode (optional)
Overrides mqtt.publish_mode
for this topic. See mqtt.publish_mode
for available modes.
A single command is defined using following settings.
name (required)
A command topic name to subscribe. Full name is created as topic_name/command_name
register (required)
Modbus register address in the form of <network_name>.<slave_id>.<register_number>
If register_number
is a decimal, then first register address is 1.
If register_number
is a hexadecimal, then first register address is 0.
network_name
and slave_id
are optional if default values are set for a topic
register_type (required)
Modbus register type: coil, holding
count (optional)
Number of registers to write. If set to > 1, then modbus_write_registers(3)/modbus_write_bits(3) is called.
converter (optional)
The name of function that should be called to convert mqtt value to uint16_t value. Format of function name is plugin name.function name
. See converters for details.
Example of MQTT command topic declaration:
objects:
topic: test_switch commands:
Publishing value 100 to topic test_switch/set will write value 1 to register 2 on slave 1.
Unless you provide a custom converter MQMGateway expects register value as UTF-8 string value or json array with register values. You must provide exactly the same number of values as registers to write.
The state sections defines how to publish modbus data to MQTT broker. State can be mapped to a single register, an unnamed and a named list of registers. Following table shows what kind of output is generated for each type:
Value type | Default output |
---|---|
single register | uint16_t register data as string |
unnamed list | JSON array with uint16_t register data as string |
named list | JSON map with values as uint16_t register data as string |
It is also possible to combine and output an unnamed list of registers as a single value using converter. See converters section for details.
Register list can be defined in two ways:
As starting register and count:
state:
name: mqtt_combined_val
converter: std.int32
register: net.1.12
count: 2
This declaration creates a poll group. Poll group is read from modbus slave using a single modbus_read_registers(3) call. Overlapping poll groups are merged with each other and with poll groups defined in modbus section.
as list of registers:
state:
- name: humidity
register: net.1.12
register_type: input
# optional
converter: std.divide(100,2)
- name: temp1
register: net.1.300
register_type: input
This declaration do not create a poll group, but allows to construct MQTT topic data from different slaves, even on different modbus networks. On exception is if there are poll groups defined in modbus section, that overlaps state register definitions. In this case data is polled using poll group.
Overrides mqtt.refresh for this state topic
When state is a single modbus register value:
The last part of topic name where value should be published. Full topic name is created as topic_name/state_name
Modbus register address in the form of register_number
is a decimal, then first register address is 1.
If register_number
is a hexadecimal, then first register address is 0.
network_name
and slave_id
are optional if default values are set for a topic
holding
)Modbus register type: coil, bit, input, holding
count (optional, default: 1)
If defined, then this describes register range to poll. Register range is always polled with a single modbus_read_registers(3) call
converter (optional)
The name of function that should be called to convert register uint16_t value to MQTT UTF-8 value. Format of function name is plugin_name.function_name
. See converters for details.
The following examples show how to combine name, register, register_type, and converter to output different state values:
single value
state:
name: mqtt_val
register: net.1.12
register_type: coil
unnamed list, each register is polled with a separate modbus_read_registers call
state:
name: mqtt_list
registers:
- register: net.1.12
register_type: input
- register: net.1.100
register_type: input
multiple registers converted to single MQTT value, polled with single modbus_read_registers call
state:
name: mqtt_combined_val
converter: std.int32
register: net.1.12
count: 2
named list (map)
state:
- name: humidity
register: net.1.12
register_type: input
# optional
converter: std.divide(100,2)
- name: temp1
register: net.1.13
register_type: input
In all of above examples refresh can be added at any level to set different values to whole list or a single register.
Lists and maps can be nested if needed:
state:
- name: humidity
register: net.1.12
count: 2
converter: std.float()
- name: other_params
registers:
- name: "temp1"
register: net.1.14
count: 2
converter: std.int32()
- name: "temp2"
register: net.1.16
count: 2
converter: std.int32()
MQTT output: {"humidity": 32.45, "other_params": { "temp1": 23, "temp2": 66 }}
For each state topic there is another availability topic defined by default. If all data required for a state is read from the Modbus registers without errors, the value "1" is published by default. If there is a network or device error when polling register data value "0" is published. This is the default behavior if the availability section is not defined.
Availablity flag is always published after the state value. If the availability flag is 0, then the current state value may contain outdated or invalid data or may not be published at all.
Availability section extends this default behaviour by defining a single or list of modbus registers that should be read to check if state data is valid. This could be i.e. some fault indicator or hardware switch state.
Configuration values:
name (required)
The last part of topic name where availability flag should be published. Full topic name is created as topic.name/availability.name
register (required)
Modbus register address in the form of register_number
is a decimal, then first register address is 1.
If register_number
is a hexadecimal, then first register address is 0.
network_name
and slave_id
are optional if default values are set for a topic
register_type (required)
Modbus register type: coil, input, holding
count
If defined, then this describes register range to poll. Register range is always polled with a single modbus_read_registers(3) call
converter (optional)
The name of function that should be called to convert register uint16_t values to MQTT UTF-8 value. Format of function name is plugin_name.function_name
. See converters for details.
After conversion, MQTT value is compared to available_value. "1" is published if values are equal,
otherwise "0".
available_value (optional, default 1)
Expected MQTT value read from availability register (or list of registers passed to converter) when availability flag should be set to "1". If other value is read then availability flag is set to "0".
register, register_type can form a registers: list when multiple registers should be read. In this case converter is mandatory and no nesting is allowed. See examples in state section.
Data read from modbus registers is by default converted to string and published to MQTT broker.
MQMGateway uses conversion plugins to convert state data read from modbus registers to mqtt value and command mqtt payload to register value, for example to combine multiple modbus registers into single value, use mask to extract one bit, or perform some simple divide operations.
Converter can also be used to convert mqtt command payload to register value.
String converter arguments can be passed in single or double quotes.
Converter functions are defined in libraries dynamically loaded at startup. MQMGateway contains std library with basic converters ready to use:
divide
Usage: state, command
Arguments:
Divides value by divisor and rounds to (precision) digits after the decimal. For modbus data supports uint16 in single register and uint32 value in two registers. For int32 mode the first modbus register holds higher byte, the second holds lower byte if 'low first' is not passed. With 'low_first' argument the first modbus register holds lower byte, the second holds higher byte.
multiply
Usage: state, command
Arguments:
Multiples value. See 'divide' for description of 'precision' and 'low_first' arguments.
int8
Usage: state
Arguments:
Parses and writes modbus register data as signed int8. Second byte is parsed by default, pass first
as argument to read first byte.
uint8
Usage: state
Arguments:
Parses and writes modbus register data as unsigned int8. Second byte is parsed by default, pass first
as argument to read first byte.
int16
Usage: state, command
Parses and writes modbus register data as signed int16.
int32
Usage: state, command
Arguments:
Combines two modbus registers into one 32bit value or writes 32bit mqtt value to two modbus registers. Without arguments the first modbus register holds higher byte, the second holds lower byte. With 'low_first' argument the first modbus register holds lower byte, the second holds higher byte.
uint32
Usage: state, command
Arguments:
Same as int32, but modbus registers are interpreted as unsingned int32.
float32
Usage: state, command Arguments:
Combines two modbus registers into one 32bit float or writes mqtt value to two modbus registers as float. Without arguments the first modbus register holds higher byte, the second holds lower byte. With 'low_first' argument the first modbus register holds lower byte, the second holds higher byte.
If 'swap_bytes' is defined, then bytes in both registers are swapped before reading and writing. Float value stored on four bytes ABCD will be written to modbus registers R0, R1 as:
bitmask
Usage: state
Arguments:
Applies a mask to value read from modbus register.
bit
Usage: state (single holding or input register)
Arguments:
Returns 1 if given bit is set, 0 otherwise
string
Usage: state, command
Parses and writes modbus register data as string.
Register data is expected as C-Style string in UTF-8 (or ASCII) encoding, e.g. 0x4142
for the string AB.
If there not Nul 0x0 byte at the end, then string size is determined by the number of registers, configured using the count
setting.
When writing, converters puts all bytes from Mqtt payload into register bytes. If payload is shorter, then remaining bytes are zeroed.
map Usage: state, command (single register only)
Arguments:
"{register_value1: mqtt_value1, register_value2: mqtt_value2}"
Returns mqtt value that is mapped to a single register value. If read register value is not mapped then its value is published as is.
Map key must be a single 16-bit value. All keys must be unique.
Map value can be a 32-bit int value or a string. All values must be unique.
Special characters {}:,"\
must be escaped with \
.
When used in command section reverse mapping is done.
Examples:
converter: std.map('{1:11, 0x2:"two", 3:"escaped: \""}')
Due to yaml limitation, no space between key and value is allowed, unless you use |
format:
converter: |
std.map('{1: 11, 0x2: 2}')
Curly braces are optional:
converter: std.map('1:-1,6:9,8:"42"')
Converter can be added to modbus register in state and command section.
When a state is a single modbus register:
state:
register: device1.slave2.12
register_type: input
converter: std.divide(10,2)
When a state is combined from multiple modbus registers:
state:
register: device1.slave2.12
register_type: input
count: 2
converter: std.int32()
When a mqtt command payload should be converted to register value:
commands:
- name: set_val
register: device1.slave2.12
register_type: input
converter: std.divide(10)
When an availability value should be computed from multiple registers:
availability:
register: device1.slave2.12
register_type: input
count: 2
converter: std.int32()
available_value: 65537
Exprtk converter allows to use exprtk expression language to convert register data to mqtt value. Register values are defined as R0..Rn variables.
evaluate
Usage: state
Arguments:
The following custom functions for 32-bit numbers are supported in the expression.
ABCD means a number composed of the byte array [A, B, C, D]
,
where A is the most significant byte (MSB) and D is the least-significant byte (LSB).
int32(R0, R1)
: Cast to signed integer ABCD from R0
== AB and R1
== CD.int32(R1, R0)
: Cast to signed integer ABCD from R0
== CD and R1
== AB.uint32(R0, R1)
: Cast to unsigned integer ABCD from R0
== AB and R1
== CD.uint32(R1, R0)
: Cast to unsigned integer ABCD from R0
== CD and R1
== AB.flt32(R0, R1)
: Cast to float ABCD from R0
== AB and R1
== CD.flt32(R1, R0)
: Cast to float ABCD from R0
== CD and R1
== AB.flt32be(R0, R1)
: Cast to float ABCD from R0
== BA and R1
== DC.flt32be(R1, R0)
: Cast to float ABCD from R0
== DC and R1
== BA.
If modbus register contains signed integer data, you can use this cast in the expression:
int16(R0)
: Cast uint16 value from `R0' to int16Division of two registers with precision 3:
objects:
- topic: test_state
state:
converter: expr.evaluate("R0 / R1", 3)
registers:
- register: tcptest.1.2
register_type: input
- register: tcptest.1.300
register_type: input
Reading the state of a 32-bit float value (byte order ABCD
) spanning two registers (R0 = BA
, R1 = DC
) with precision 3:
objects:
- topic: test_state
state:
converter: expr.evaluate("flt32be(R0, R1)", 3)
register: tcptest.1.2
register_type: input
count: 2
Custom converters can be added by creating a C++ dynamically loaded library with conversion classes. There is a header only libmodmqttconv that provide base classes for plugin and converter implementations.
Here is a minimal example of custom conversion plugin with help of boost dll library loader:
#include <boost/config.hpp> // for BOOST_SYMBOL_EXPORT
#include "libmodmqttconv/converterplugin.hpp"
class MyConverter : public DataConverter {
public:
//called by modmqttd to set coverter arguments
virtual void setArgs(const std::vector<std::string>& args) {
mShift = getIntArg(0, args);
}
// Conversion from modbus registers to mqtt value
// Used when converter is defined for state topic
// ModbusRegisters contains one register or as many as
// configured in unnamed register list.
virtual MqttValue toMqtt(const ModbusRegisters& data) const {
int val = data.getValue(0);
return MqttValue::fromInt(val << mShift);
}
// Conversion from mqtt value to modbus register data
// Used when converter is defined for command topic
virtual ModbusRegisters toModbus(const MqttValue& value, int registerCount) const {
int val = value.getInt();
ModbusRegisters ret;
for (int i = 0; i < registerCount; i++) {
val = val >> mShift
ret.prependValue(val);
}
return ret;
}
virtual ~MyConverter() {}
private:
int mShift = 0;
};
class MyPlugin : ConverterPlugin {
public:
// name used in configuration as plugin prefix.
virtual std::string getName() const { return "myplugin"; }
virtual IStateConverter* getStateConverter(const std::string& name) {
if (name == "myconverter")
return new MyConverter();
return nullptr;
}
virtual ~MyPlugin() {}
};
// modmqttd search for "converter_plugin" C symbol in loaded dll
extern "C" BOOST_SYMBOL_EXPORT MyPlugin converter_plugin;
MyPlugin converter_plugin;
Compilation on linux:
g++ -I<path to mqmgateway source dir> -fPIC -shared myplugin.cpp -o myplugin.so
myconverter from this example can be used like this:
modmqttd:
converter_search_path:
- <myplugin.so dir>
converter_plugins:
- myplugin.so
modbus:
networks:
- name: tcptest
address: localhost
port: 501
mqtt:
objects:
- topic: test_topic
command:
name: set_val
register: tcptest.2.12
register_type: input
converter: myplugin.myconverter(1)
state:
name: test_val
register: tcptest.2.12
register_type: input
converter: myplugin.myconverter(1)
For more examples see libstdconv source code.
Multi-device defintions allows to set slave properties or create a single topic for multiple modbus devices of the same type. This greatly reduces the number of configuration sections that differ only by slave address or modbus network name.
If there are many devices of the same type then a MQTT topic for group of devices can be defined by setting slave
value to list of modbus slave addresses. Then you have to add either slave_name
or slave_address
placeholder in the topic string like this:
modbus:
networks:
- name: basement
slaves:
- address: 1
name: meter1
- address: 2
name: meter2
[...]
mqtt:
objects:
- topic: ${network}/${slave_name}/node_${slave_address}
slave: 1,2,3,8-9
network: basement, roof
state:
register: 1
In the above example 10 registers will be polled and their values will be published as /basement/meter1/node_1/state
, /basement/meter2/node_2/state
and so on.
Slave names are required only if ${slave_name}
placeholder is used.
Address list can be used in modbus.networks.slaves.address
to define properties for multiple slaves at once:
modbus:
networks:
- name: basement
slaves:
- address: 1,2,3,5-18
poll_groups:
- register: 3
count: 10
- register: 30
count: 5
A single slave address can be listed in multiple entries like this:
modbus:
networks:
- name: basement
slaves:
- address: 1
name: meter1
response_timeout: 50ms
- address: 2
name: meter2
- address: 1,2
response_timeout: 100ms
poll_groups:
- register: 3
count: 10
This example will set response timeout 100ms for both slaves - overriding value for slave1. Two poll groups are defined for reading registers 3-13 for both slaves.