Modbus_tk driver does not support two drivers pointing at the same IP address

bonicim commented 2 years ago

Description of Issue

When creating two modbus_tk drivers that have different registry and map configs but use the same IP address, only one of the drivers can successfully scrape points. Note that if one removes the working driver and restarts the Platform Driver, then the failing driver will work and start scraping its device. Related to #2403.

Affected Version

'main'

Expected

Actual

Steps to Reproduce

High level

Create two modbus_tk driver config files that use the same IP address
Create two different modbus_tk registers and maps
Add both modbus_tk driver configs to the Platform Driver's configuration store
Observe the logs for the error messages

Examples of the modbus_tk driver configs:

driver 1

{
    "driver_config": {
        "device_address": "12.34.567.890",
        "register_map": "config://driver1_map.csv"
    },
    "driver_type": "modbus_tk",
    "registry_config":"config://driver1_tk.csv",
}

driver 2

{
    "driver_config": {
        "device_address": "12.34.567.890",
        "register_map": "config://driver2_map.csv"
    },
    "driver_type": "modbus_tk",
    "registry_config":"config://driver2_tk.csv",
}

Exact Steps

Install the two drivers

Commands to install drivers:

# add the driver registries and maps to Platform Driver's config store
vctl config store platform.driver driver1_tk.csv driver1_tk.csv --csv
vctl config store platform.driver driver1_map.csv driver1_map.csv --csv
vctl config store platform.driver driver2_tk.csv driver2_tk.csv --csv
vctl config store platform.driver driver2_map.csv driver2_map.csv --csv

# add drivers to the platform.driver agent
vctl config store platform.driver devices/campus/building driver1.config
vctl config store platform.driver devices/campus/building driver2.config

Wait a short while (3 seconds) and observe the logs for the expected error message:

platform_driver.driver ERROR: Failed to scrape campus/building/bess:
Traceback (most recent call last):
File "/home/volttron/.volttron/agents/1a78c500-d193-42fa-9656-90e461476534/platform_driveragent-4.0/platform_driver/driver.py", line 244, in periodic_read
results = self.interface.scrape_all()
File "/home/volttron/.volttron/agents/1a78c500-d193-42fa-9656-90e461476534/platform_driveragent-4.0/platform_driver/interfaces/__init__.py", line 607, in scrape_all
result = self._scrape_all()
File "/home/volttron/.volttron/agents/1a78c500-d193-42fa-9656-90e461476534/platform_driveragent-4.0/platform_driver/interfaces/modbus_tk/__init__.py", line 374, in _scrape_all
field, value, timestamp in self.modbus_client.dump_all())
File "/home/volttron/.volttron/agents/1a78c500-d193-42fa-9656-90e461476534/platform_driveragent-4.0/platform_driver/interfaces/modbus_tk/client.py", line 698, in dump_all
self.read_all()
File "/home/volttron/.volttron/agents/1a78c500-d193-42fa-9656-90e461476534/platform_driveragent-4.0/platform_driver/interfaces/modbus_tk/client.py", line 695, in read_all
self.read_request(r)
File "/home/volttron/.volttron/agents/1a78c500-d193-42fa-9656-90e461476534/platform_driveragent-4.0/platform_driver/interfaces/modbus_tk/client.py", line 676, in read_request
results = self.client.execute(
File "/home/volttron/volttron/env/lib/python3.8/site-packages/modbus_tk/utils.py", line 39, in new
raise excpt
File "/home/volttron/volttron/env/lib/python3.8/site-packages/modbus_tk/utils.py", line 37, in new
ret = fcn(*args, **kwargs)
File "/home/volttron/volttron/env/lib/python3.8/site-packages/modbus_tk/modbus.py", line 306, in execute
response_pdu = query.parse_response(response)
File "/home/volttron/volttron/env/lib/python3.8/site-packages/modbus_tk/modbus_tcp.py", line 134, in parse_response
raise ModbusInvalidResponseError("Response length is only {0} bytes. ".format(len(response)))
modbus_tk.exceptions.ModbusInvalidResponseError: Response length is only 0 bytes.

Additional Details

Below is a diagram of the entire setup:

modbus_tk_bess drawio

acedrew commented 2 years ago

Here's a very, very rough hotfix that we have working in production as of today, it will simply force the modbus requests to be serial for each (host, port) pair: https://github.com/acedrew/volttron/tree/hotfix-modbus-tk/services/core/MasterDriverAgent/master_driver (this is based on 7.x) By rough I mean it's only implemented for reading bulk scrapes, no other use cases.

Most of the changes are in modbus_tk/client.py, and driver/driver_locks.py, though the initial lock config is handled in modbustk/__init_\.py

bonicim commented 2 years ago

Here's a very, very rough hotfix that we have working in production as of today, it will simply force the modbus requests to be serial for each (host, port) pair: https://github.com/acedrew/volttron/tree/hotfix-modbus-tk/services/core/MasterDriverAgent/master_driver (this is based on 7.x) By rough I mean it's only implemented for reading bulk scrapes, no other use cases.

Most of the changes are in modbus_tk/client.py, and driver/driver_locks.py, though the initial lock config is handled in modbus_tk/init.py

Thanks. I will take a look at this. I need to dive deep on modbus_tk so this will be a good segue into that package.

VOLTTRON / volttron