Intermittent Incomplete Modbus Messages When Polling a Slave Device

BhanuKiranChaluvadi commented 1 month ago

I am attempting to log data from a Modbus slave device at a high rate, targeting one read per second. However, I am frequently encountering issues with incomplete Modbus messages. Occasionally, I receive complete messages, but the inconsistency is problematic.

Environment

Python Version: 3.12.0
Operating System: Windows
Pymodbus Version: 3.6.6
Modbus Hardware: Bronkhorst MFC

Configuration

Client Type: RTU - Sync

Problem Description

The goal is to consistently log information from the Modbus slave every second. However, the reception of incomplete messages is frequent, complicating the logging process. I need advice on optimizing the parameters or the loop rate to improve the reliability of the data acquisition.

Steps to Reproduce

Connect to the Modbus slave using the specified configurations.
Attempt to read from the slave device once every second.

Expected Behavior

Receive complete Modbus messages for every polling cycle without errors or incomplete data.

Actual Behavior

Frequent incomplete messages are received, with only occasional complete messages. This is illustrated in the debug logs provided below.

Code and Logs

from pymodbus.client.sync import ModbusSerialClient as ModbusClient
from pymodbus.transaction import ModbusRtuFramer as Framer

client = ModbusClient(
    framer=Framer.RTU,
    port="COM9",
    baudrate=19200,
    parity="E",
    stopbits=1,
    timeout=1,
)
client.connect()
response = client.read_holding_registers(address=41216, count=2, slave=1)
print(response)

Debug Logs

2024-04-29 09:28:40,078 DEBUG Reading Coils
2024-04-29 09:28:40,080 DEBUG Current transaction state - IDLE
2024-04-29 09:28:40,080 DEBUG Running transaction 1
2024-04-29 09:28:40,080 DEBUG SEND: 0x2 0x3 0xa1 0x0 0x0 0x2 0xe7 0xc4
2024-04-29 09:28:40,080 DEBUG Resetting frame - Current Frame in buffer
2024-04-29 09:28:40,080 DEBUG New Transaction state "SENDING"
2024-04-29 09:28:40,080 DEBUG Changing transaction state from "SENDING" to "WAITING FOR REPLY"
2024-04-29 09:28:40,103 DEBUG Incomplete message received, Expected 9 bytes Received 5 bytes !!!!
2024-04-29 09:28:40,103 DEBUG Changing transaction state from "WAITING FOR REPLY" to "PROCESSING REPLY"
2024-04-29 09:28:40,103 DEBUG RECV: 0x2 0x3 0x4 0x3b 0xc8
2024-04-29 09:28:40,103 DEBUG Processing: 0x2 0x3 0x4 0x3b 0xc8
2024-04-29 09:28:40,103 DEBUG Frame - not ready
2024-04-29 09:28:40,103 DEBUG Getting transaction 2
2024-04-29 09:28:40,210 DEBUG Changing transaction state from "PROCESSING REPLY" to "TRANSACTION_COMPLETE"

Questions

How can I adjust the parameters to ensure more consistent receipt of complete messages?
Is there a recommended loop rate or method for polling this type of Modbus device to minimize data loss?

janiversen commented 1 month ago

Look at ou4 examples !

use async that i# a couple of factors faster

I ran a productions system with 3.6.1 in sync mode, and with a poll rate of 100ms, in essence 30 requests/second pr second without problem.

However I used tcp use seem to use serial, that have a lot more constraints:

Low baudrate will limit the throughput severely
your devices response time might be a limiting factor

With a request pr second the library is basically sleeping 80-90% of realtime, so you4 problem is somewhere else either your device is responding with partial packets (VERY unusual) or your usb converter is behaving badly (very normal, a lot of rs-485 usb converters handle the packets badly).

janiversen commented 1 month ago

we have no "polling" you issue a request and wait for the reply.

github-actions[bot] commented 2 weeks ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

janiversen commented 2 weeks ago

Closing due to inactivity.

crash1013 commented 2 weeks ago

I have not been able to use pymodbus serial in Python version 11 or version 12. I see the same errors that are shown above, incomplete message received. I use the same code in python 9 and 10 and it works perfectly. It seems like the speed improvements that were made in python version 11 are not compatible with pymodbus serial.

janiversen commented 2 weeks ago

Please use the spply debug log call, and let's see a debug log.

janiversen commented 2 weeks ago

our test harness uses pyserial heavily but with its socket option, I will try to setup a physical serial line and test.

janiversen commented 2 weeks ago

This is interesting! I just made a test, connected my mac and solar inverter with rs485 (via a usb hub) and it worked without errors with 9.600

crash1013 commented 2 weeks ago

I will get this done for you today. I would have done it earlier but I have been very busy. Thanks for your rapid response.

I have installed virtual machines for python 3.9, 3.10, 3.11 and 3.12. I see lost information in 3.11 and 3.12 but not 3.9 and 3.10.

From: jan iversen @.> Sent: Wednesday, June 12, 2024 2:55 PM To: pymodbus-dev/pymodbus @.> Cc: crash1013 @.>; Comment @.> Subject: Re: [pymodbus-dev/pymodbus] Intermittent Incomplete Modbus Messages When Polling a Slave Device (Issue #2193)

Please use the spply debug log call, and let's see a debug log.

— Reply to this email directly, view it on GitHub https://github.com/pymodbus-dev/pymodbus/issues/2193#issuecomment-2163703244 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ADMDNRQWATSURKFGMV2LZLLZHCKQZAVCNFSM6AAAAABHSZITL2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRTG4YDGMRUGQ . You are receiving this because you commented. https://github.com/notifications/beacon/ADMDNRRERPDTXG7B3JNYWRLZHCKQZA5CNFSM6AAAAABHSZITL2WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTUA6564Y.gif Message ID: @. @.> >

janiversen commented 2 weeks ago

OK, we have for other reasons decided to make a v3.6.9 that will be done this weekend, so I need your log fast to include a fix in that release.

crash1013 commented 2 weeks ago

Here are the logs, one for each python version 3.9 through 3.12. 3.9 and 3.10 are perfect. 3.11 and 3.12 show the same error. All are running the same script. Only the interpreter is changed. They are all using 3.6.8 except python 3.12 is using pymodbus==3.6.7 test_logfile-V3.9.txt test_logfile-V3.10.txt test_logfile-V3.11.txt test_logfile-V3.12.txt

janiversen commented 2 weeks ago

First of all let me say again, change to async will help you a lot ! pyserial remains the same, but the pymodbus transport layer is "hardened".

From what I see pyserial or your usb stick "eats" the last byte:

Incomplete message received, Expected 21 bytes Received 20 bytes !!!!
Changing transaction state from "WAITING FOR REPLY" to "PROCESSING REPLY"
RECV: 0xf8 0x3 0x10 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x48
Processing: 0xf8 0x3 0x10 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x48
Frame - not ready

Normally the failing byte would be part of the next receive, but its not:

RECV: 0xf8 0x3 0x2 0x0 0xb 0x65 0x97

So my experience tells me it's likely a cable problem or a missing pull-up resistor. I have today a serial connection to my solar inverter and read: 1 value every 50ms 10 values every 1s 90 values every 15minutes that runs with python 3.12 on a RPI4, without problems (and the library isn't even sweating, so 1 read/second is slow).

You might try to lower the baud rate, since windows is known to have problem with usb-serial devices at higher rates.

I cannot tell you why its works with lower versions of pymodbus, since I do not have them installed.

crash1013 commented 1 week ago

It could be the case, but on Python 3.9 and 3.10 the stick does not eat the last byte. Python 3.12 on Raspberry PI OS is somewhat different than running on windows, perhaps not apples and oranges but certainly not equal.

From: jan iversen @.> Sent: Friday, June 14, 2024 12:52 PM To: pymodbus-dev/pymodbus @.> Cc: crash1013 @.>; Comment @.> Subject: Re: [pymodbus-dev/pymodbus] Intermittent Incomplete Modbus Messages When Polling a Slave Device (Issue #2193)

First of all let me say again, change to async will help you a lot ! pyserial remains the same, but the pymodbus transport layer is "hardened".

From what I see pyserial or your usb stick "eats" the last byte:

Incomplete message received, Expected 21 bytes Received 20 bytes !!!! Changing transaction state from "WAITING FOR REPLY" to "PROCESSING REPLY" RECV: 0xf8 0x3 0x10 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x48 Processing: 0xf8 0x3 0x10 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x48 Frame - not ready

Normally the failing byte would be part of the next receive, but its not:

RECV: 0xf8 0x3 0x2 0x0 0xb 0x65 0x97

So my experience tells me it's likely a cable problem or a missing pull-up resistor. I have today a serial connection to my solar inverter and read: 1 value every 50ms 10 values every 1s 90 values every 15minutes that runs with python 3.12 on a RPI4, without problems (and the library isn't even sweating, so 1 read/second is slow).

You might try to lower the baud rate, since windows is known to have problem with usb-serial devices at higher rates.

I cannot tell you why its works with lower versions of pymodbus, since I do not have them installed.

— Reply to this email directly, view it on GitHub https://github.com/pymodbus-dev/pymodbus/issues/2193#issuecomment-2168418554 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ADMDNRVUXEV3A5CXUYINT7LZHMNUXAVCNFSM6AAAAABHSZITL2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRYGQYTQNJVGQ . You are receiving this because you commented. https://github.com/notifications/beacon/ADMDNRVHMKBZ4UL7TRXO6CTZHMNUXA5CNFSM6AAAAABHSZITL2WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTUBH5YPU.gif Message ID: @. @.> >

janiversen commented 1 week ago

Agreed but I have no idea on how to help, I do not have windows installed, and anyhow it seems to be a problem outside pymodbus.

There is a PR that to set rs485 parameters that might help you.

pymodbus-dev / pymodbus