Feature/add callbacks to register get and set functions

brainelectronics commented 1 year ago

Added

Custom callback functions can be registered on client (ModbusRTU or ModbusTCP) side with new parameters on_set_cb and on_get_cb available from modbus.py functions add_coil and add_hreg. Functions add_ist and add_ireg support only on_get_cb, see #31
Example callback usage shown in TCP client example
Documentation for callback functions in USAGE

Changed

Typing hint Callable is now subscriptable

brainelectronics commented 1 year ago

@beyonlo and @jd-boyd you can test the changes in 2.3.0-rc22.dev51 available from Test PyPi MicroPython Modbus

The updated documentation can be found at the RTD MicroPython Modbus optional callbacks section

beyonlo commented 1 year ago

@brainelectronics this is an amazing feature!! :partying_face:

Example callback usage shown in TCP client example

This url is broken.

The updated documentation can be found at the RTD MicroPython Modbus optional callbacks section

That doc is very good! I used that as reference!

Results: works, with just one problem: using a list of COILS, after set it to new values, read it cause OSError: invalid response CRC, but the Slave COIL get callback function works. That problem do not happen on the HREGS. Details below:

Callback functions and register_definitions:

def my_coil_set_cb(reg_type, address, val):
    print('Custom callback, called on setting {} at {} to: {}'.
          format(reg_type, address, val))

def my_coil_get_cb(reg_type, address, val):
    print('Custom callback, called on getting {} at {}, currently: {}'.
          format(reg_type, address, val))

def my_hregs_set_cb(reg_type, address, val):
    print('Custom callback, called on setting {} at {} to: {}'.
          format(reg_type, address, val))

def my_hregs_get_cb(reg_type, address, val):
    print('Custom callback, called on getting {} at {}, currently: {}'.
          format(reg_type, address, val))

# common slave register setup, to be used with the Master example above
register_definitions = {
    "COILS": {
        "EXAMPLE_COIL": {
            "register": 123,
            "len": 1,
            "val": 1
        },
        "COIL_SIGNALS": {
            "register": 125,
            "len": 16,
            "val": [1,1,1,1,1,0,0,0,0,1,1,1,0,0,0,1],
            "on_get_cb": my_coil_get_cb,
            "on_set_cb": my_coil_set_cb
        }
    },
    "HREGS": {
        "EXAMPLE_HREG": {
            "register": 93,
            "len": 1,
            "val": 19
        },
        "HREG_VALUES": {
            "register": 130,
            "len": 33,
            "val": [34, 12, 14, 0, 10, 4, 2, 1345, 34, 8, 1100, 350, 456, 754, 324, 423, 530, 90, 320, 34, 244, 355, 606, 656, 640, 620, 677, 623, 234, 567, 34, 56, 68],
            "on_get_cb": my_hregs_get_cb,
            "on_set_cb": my_hregs_set_cb
        }
    },
    "ISTS": {
        "EXAMPLE_ISTS": {
            "register": 67,
            "len": 1,
            "val": 0
        }
    },
    "IREGS": {
        "EXAMPLE_IREG": {
            "register": 10,
            "len": 2,
            "val": 60001
        }
    }
}

Slave RTU:

$ mpremote run rtu_client_example_with_callback.py 
Running ModBus version: 2.3.0-rc22.dev51

Custom callback, called on getting COILS at 125, currently: [1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1]
Custom callback, called on setting COILS at 125 to: [False, True, True]
Custom callback, called on getting COILS at 125, currently: [False, True, True]
Custom callback, called on getting HREGS at 130, currently: [34, 12, 14, 0, 10, 4, 2, 1345, 34, 8, 1100, 350, 456, 754, 324, 423, 530, 90, 320, 34, 244, 355, 606, 656, 640, 620, 677, 623, 234, 567, 34, 56, 68]
Custom callback, called on setting HREGS at 130 to: [2, 65532, 6, 65280, 1024]
Custom callback, called on getting HREGS at 130, currently: [2, 65532, 6, 65280, 1024]

Master RTU:

>>> from umodbus import version
>>> print('Running ModBus version: {}'.format(version.__version__))
Running ModBus version: 2.3.0-rc22.dev51
>>> 
>>> from umodbus.serial import Serial as ModbusRTUMaster
>>> rtu_pins = (17, 18)
>>> 
>>> host = ModbusRTUMaster(baudrate=115200, data_bits=8, stop_bits=1, parity=None, pins=rtu_pins, ctrl_pin=15)
>>> 
>>> host.read_coils(slave_addr=10, starting_addr=125, coil_qty=16)
[True, True, True, True, True, False, False, False, False, True, True, True, False, False, False, True]
>>> host.write_multiple_coils(slave_addr=10, starting_address=125, output_values=[1,1,0])
True
>>> host.read_coils(slave_addr=10, starting_addr=125, coil_qty=16) <--- Here happen the error, after set to new values
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/lib/umodbus/common.py", line 136, in read_coils
  File "/lib/umodbus/serial.py", line 289, in _send_receive
  File "/lib/umodbus/serial.py", line 322, in _validate_resp_hdr
OSError: invalid response CRC
>>> host.read_holding_registers(slave_addr=10, starting_addr=130, register_qty=33, signed=False)
(34, 12, 14, 0, 10, 4, 2, 1345, 34, 8, 1100, 350, 456, 754, 324, 423, 530, 90, 320, 34, 244, 355, 606, 656, 640, 620, 677, 623, 234, 567, 34, 56, 68)
>>> host.write_multiple_registers(slave_addr=10, starting_address=130, register_values=[2, -4, 6, -256, 1024], signed=True)
True
>>> host.read_holding_registers(slave_addr=10, starting_addr=130, register_qty=33, signed=False)
(2, 65532, 6, 65280, 1024)
>>>

beyonlo commented 1 year ago

@brainelectronics one more problem: the print callback shows always signed as False.

Slave:

$ mpremote run rtu_client_example_with_callback.py 
Running ModBus version: 2.3.0-rc22.dev51
Custom callback, called on setting HREGS at 130 to: [2, 65532, 6, 65280, 1024]
Custom callback, called on getting HREGS at 130, currently: [2, 65532, 6, 65280, 1024]

Master:

>>> host.write_multiple_registers(slave_addr=10, starting_address=130, register_values=[2, -4, 6, -256, 1024], signed=True)
True
>>> host.read_holding_registers(slave_addr=10, starting_addr=130, register_qty=33, signed=True)
(2, -4, 6, -256, 1024)
>>>

Look that number -4 is print as 65532 and number -256 as 65280.

Thank you!

beyonlo commented 1 year ago

@brainelectronics thinking in callback features, is possible to call a function/method inside the register_definitions to update the register_definitions values everytime that a request is received. Just a example:

vvv = 345
def values_hregs():
    global vvv
    vvv += 1
    print('-----------')
    print(vvv)
    return [34, vvv, 14, 0, 10, 4, 2, 1345]

register_definitions = {
        "HREG_VALUES": {
            "register": 130,
            "len": 33,
            "val": values_hregs(),
            "on_get_cb": my_hregs_get_cb,
            "on_set_cb": my_hregs_set_cb
        }
    }

Slave:

$ mpremote run rtu_client_example_with_callback.py 
Running ModBus version: 2.3.0-rc22.dev51
-----------
346
Custom callback, called on getting HREGS at 130, currently: [34, 346, 14, 0, 10, 4, 2, 1345]
Custom callback, called on getting HREGS at 130, currently: [34, 346, 14, 0, 10, 4, 2, 1345]
Custom callback, called on getting HREGS at 130, currently: [34, 346, 14, 0, 10, 4, 2, 1345]

Master:

>>> host.read_holding_registers(slave_addr=10, starting_addr=130, register_qty=33, signed=False)
(34, 346, 14, 0, 10, 4, 2, 1345)
>>> host.read_holding_registers(slave_addr=10, starting_addr=130, register_qty=33, signed=False)
(34, 346, 14, 0, 10, 4, 2, 1345)
>>> host.read_holding_registers(slave_addr=10, starting_addr=130, register_qty=33, signed=False)
(34, 346, 14, 0, 10, 4, 2, 1345)
>>>

Of course that function (values_hregs()) is loaded just the first time and that's why the vvv value is not change in the Master requests - that's correct. But is possible in the Slave to create something (like as a callback) to read a function every time that a request happens? Like as a dynamic register_definitions. Maybe this PR is not the best place, so I can to create a new issue as feature if you agree.

Thank you in advance!

Edit:

That is very useful feature, like as to read the GPIOs for the INPUT STATUS (ISTS), read the ADCs for the INPUT REGISTERS (IREGS) and for HOLDING REGISTERS (HREGS) and COILS as well

brainelectronics commented 1 year ago

@brainelectronics thinking in callback features, is possible to call a function/method inside the register_definitions to update the register_definitions values everytime that a request is received. Just a example:

I would suggest to move the on_get_cb from after to before sending the response.

brainelectronics commented 1 year ago

@brainelectronics one more problem: the print callback shows always signed as False.

@beyonlo I know this issue, but I can't solve it 😞 . The interpretation of whether it is a signed value or not it done on the requester side, that's why you have to set the signed parameter on any read operation. The client just stores the value but has no further informations about it.

brainelectronics commented 1 year ago

@brainelectronics this is an amazing feature!! 🥳

Example callback usage shown in TCP client example

This url is broken. It is currently broken in the PR as the URL is resolved as https://github.com/brainelectronics/micropython-modbus/pull/examples/tcp_client_example.py which will become a valid URL after merge

The updated documentation can be found at the RTD MicroPython Modbus optional callbacks section

That doc is very good! I used that as reference!

Results: works, with just one problem: using a list of COILS, after set it to new values, read it cause OSError: invalid response CRC, but the Slave COIL get callback function works. That problem do not happen on the HREGS. Details below:

@beyonlo yes, this is expected. As you set the coils of 125 to [1,1,0] but then request 16 coils (which are not available, only 3) the CRC is invalid. This again relates to #35 and will be solved after this PR

beyonlo commented 1 year ago

@brainelectronics thinking in callback features, is possible to call a function/method inside the register_definitions to update the register_definitions values everytime that a request is received. Just a example:

I would suggest to move the on_get_cb from after to before sending the response.

I'm sorry, but I do not understand how to do that. Could you please paste a simple example?

Just to be sure what I'm talking about, is about that function values_hregs() that I call in the "val": values_hregs(), I mean, do not has relation with the on_get_cb, that is there just as old example test. If i missing some understand, please point me.

beyonlo commented 1 year ago

@brainelectronics one more problem: the print callback shows always signed as False.

@beyonlo I know this issue, but I can't solve it disappointed . The interpretation of whether it is a signed value or not it done on the requester side, that's why you have to set the signed parameter on any read operation. The client just stores the value but has no further informations about it.

@brainelectronics but when Slave receive a Master request, the Slave always receive the signed parameter because Master always send that signed parameter, right? So, Slave could just, before to call the callbacks, to use that signed parameter to execute the callback on_get_cb/on_set_cb showing the correct values. Does that make sense for you?

beyonlo commented 1 year ago

@beyonlo yes, this is expected. As you set the coils of 125 to [1,1,0] but then request 16 coils (which are not available, only 3) the CRC is invalid. This again relates to #35 and will be solved after this PR

@brainelectronics unfortunately that error happen with coil_qty=3 as well. Details below:

Slave:

$ mpremote run rtu_client_example_with_callback.py 
Running ModBus version: 2.3.0-rc22.dev51
Custom callback, called on getting COILS at 125, currently: [1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1]
Custom callback, called on setting COILS at 125 to: [False, True, True]
Custom callback, called on getting COILS at 125, currently: [False, True, True]
Custom callback, called on getting COILS at 125, currently: [False, True]
Custom callback, called on getting COILS at 125, currently: [False]
Custom callback, called on getting COILS at 125, currently: [False, True, True]
Custom callback, called on getting COILS at 125, currently: [False, True, True]

Master:

>>> from umodbus.serial import Serial as ModbusRTUMaster
>>> rtu_pins = (17, 18)
>>> host = ModbusRTUMaster(baudrate=115200, data_bits=8, stop_bits=1, parity=None, pins=rtu_pins, ctrl_pin=15)
>>> host.read_coils(slave_addr=10, starting_addr=125, coil_qty=16)
[True, True, True, True, True, False, False, False, False, True, True, True, False, False, False, True]
>>> host.write_multiple_coils(slave_addr=10, starting_address=125, output_values=[1,1,0])
True
>>> host.read_coils(slave_addr=10, starting_addr=125, coil_qty=3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/lib/umodbus/common.py", line 136, in read_coils
  File "/lib/umodbus/serial.py", line 289, in _send_receive
  File "/lib/umodbus/serial.py", line 322, in _validate_resp_hdr
OSError: invalid response CRC
>>> host.read_coils(slave_addr=10, starting_addr=125, coil_qty=2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/lib/umodbus/common.py", line 136, in read_coils
  File "/lib/umodbus/serial.py", line 289, in _send_receive
  File "/lib/umodbus/serial.py", line 322, in _validate_resp_hdr
OSError: invalid response CRC
>>> host.read_coils(slave_addr=10, starting_addr=125, coil_qty=1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/lib/umodbus/common.py", line 136, in read_coils
  File "/lib/umodbus/serial.py", line 289, in _send_receive
  File "/lib/umodbus/serial.py", line 322, in _validate_resp_hdr
OSError: invalid response CRC
>>> host.read_coils(slave_addr=10, starting_addr=125, coil_qty=8)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/lib/umodbus/common.py", line 136, in read_coils
  File "/lib/umodbus/serial.py", line 289, in _send_receive
  File "/lib/umodbus/serial.py", line 322, in _validate_resp_hdr
OSError: invalid response CRC
>>> 
>>> host.read_coils(slave_addr=10, starting_addr=125, coil_qty=4)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/lib/umodbus/common.py", line 136, in read_coils
  File "/lib/umodbus/serial.py", line 289, in _send_receive
  File "/lib/umodbus/serial.py", line 322, in _validate_resp_hdr
OSError: invalid response CRC
>>>

brainelectronics commented 1 year ago

@brainelectronics but when Slave receive a Master request, the Slave always receive the signed parameter because Master always send that signed parameter, right? So, Slave could just, before to call the callbacks, to use that signed parameter to execute the callback on_get_cb/on_set_cb showing the correct values. Does that make sense for you?

@beyonlo your idea makes really sense, but looking at the code will explain you further things. I'll take your example with -4 vs 65532 and use a single register to make it as simple as possible.

https://github.com/brainelectronics/micropython-modbus/blob/59e1c6c8bc41c9b893f0c17d54229c924b7c3817/umodbus/common.py#L295-L298

https://github.com/brainelectronics/micropython-modbus/blob/59e1c6c8bc41c9b893f0c17d54229c924b7c3817/umodbus/functions.py#L146-L151

>>> import struct
>>>
>>> struct.pack('>BHH', 0x06, 123, 65532)
b'\x06\x00{\xff\xfc'
>>>
>>> struct.pack('>BHh', 0x06, 123, -4)
b'\x06\x00{\xff\xfc'
>>>
>>> struct.pack('>BHh', 0x06, 123, -4) == struct.pack('>BHH', 0x06, 123, 65532)
True

The only reason why you need to specify the signed parameter is to properly pack the data in the end. But the packed data is the same. So the client device just gets this packed data and stores it. On a request the data is returned as stored and only on requester side (host) unpacked with respect to the signed flag.

https://github.com/brainelectronics/micropython-modbus/blob/59e1c6c8bc41c9b893f0c17d54229c924b7c3817/umodbus/common.py#L203

brainelectronics commented 1 year ago

I'm sorry, but I do not understand how to do that. Could you please paste a simple example?

Just to be sure what I'm talking about, is about that function values_hregs() that I call in the "val": values_hregs(), I mean, do not has relation with the on_get_cb, that is there just as old example test. If i missing some understand, please point me.

@beyonlo I'm sorry as well. This was nothing you should solve, it was just a suggestion, which I now implemented in https://github.com/brainelectronics/micropython-modbus/pull/51/commits/5f0e25dc04f411572ba6d7b498fbca0036e34ff3

jd-boyd commented 1 year ago

I know I'm late, but it looks good and thank you.

beyonlo commented 1 year ago

I'm sorry, but I do not understand how to do that. Could you please paste a simple example? Just to be sure what I'm talking about, is about that function values_hregs() that I call in the "val": values_hregs(), I mean, do not has relation with the on_get_cb, that is there just as old example test. If i missing some understand, please point me.

@beyonlo I'm sorry as well. This was nothing you should solve, it was just a suggestion, which I now implemented in 5f0e25d

@brainelectronics Now I see in the tcp_client_example.py an example with this working - update register_definitions with new values before sending response - that's very nice, thank you.

def my_inputs_register_get_cb(reg_type, address, val):
    # usage of global isn't great, but okay for an example
    global client

    print('Custom callback, called on getting {} at {}, currently: {}'.
          format(reg_type, address, val))

    # any operation should be as short as possible to avoid response timeouts
    new_val = val[0] + 1

    # It would be also possible to read the latest ADC value at this time
    # adc = machine.ADC(12)     # check MicroPython port specific syntax
    # new_val = adc.read()

    client.set_ireg(address=address, value=new_val)
    print('Incremented current value by +1 before sending response')

I see as well a example to reset all values - very nice!!

def reset_data_registers_cb(reg_type, address, val):
    # usage of global isn't great, but okay for an example
    global client
    global register_definitions

    print('Resetting register data to default values ...')
    client.setup_registers(registers=register_definitions)
    print('Default values restored')

I liked so much that new mode to register the callbacks:

# coils and holding register support callbacks for set and get
register_definitions['COILS']['EXAMPLE_COIL']['on_set_cb'] = my_coil_set_cb
register_definitions['COILS']['EXAMPLE_COIL']['on_get_cb'] = my_coil_get_cb
register_definitions['HREGS']['EXAMPLE_HREG']['on_set_cb'] = \
    my_holding_register_set_cb
register_definitions['HREGS']['EXAMPLE_HREG']['on_get_cb'] = \
    my_holding_register_get_cb

# discrete inputs and input registers support only get callbacks as they can't
# be set externally
register_definitions['ISTS']['EXAMPLE_ISTS']['on_get_cb'] = \
    my_discrete_inputs_register_get_cb
register_definitions['IREGS']['EXAMPLE_IREG']['on_get_cb'] = \
    my_inputs_register_get_cb

# reset all registers back to their default value with a callback
register_definitions['COILS']['RESET_REGISTER_DATA_COIL']['on_set_cb'] = \
    reset_data_registers_cb

As this is very useful, maybe is a good idea to put those examples together with that RTD MicroPython Modbus optional callbacks section

beyonlo commented 1 year ago

The only reason why you need to specify the signed parameter is to properly pack the data in the end. But the packed data is the same. So the client device just gets this packed data and stores it. On a request the data is returned as stored and only on requester side (host) unpacked with respect to the signed flag.

@brainelectronics All right, it's clear now! I'm sorry for taking so long to understand. In the first time that you told I already should understood - thank you for the patience.

brainelectronics / micropython-modbus

Feature/add callbacks to register get and set functions #51

Added

Changed