TLS implementation seems broken?

[dries007]

Versions

• Python: 3.11, 3.8
• OS: Linux, Windows10
• Pymodbus: release (3.6.8), dev (a0e3044103d881b59feb93ba1d6191486371154d)
• Modbus Hardware (if used): n/a

Pymodbus Specific

• Server: device, not pymodbus
• Client: TLS

Description

Calling read_holding_registers results in Connection unexpectedly closed 0.000 seconds into read of 2002 bytes without response from slave before it closed connection

When testing with umodbus or modbus-tk, our device replied with the expected values.

After debugging we notice that pymodbus sends quite different bytes over the socket than the other two libraries.

Code samples for the other libraries included below, including byte arrays send & received.

Code and Logs

import asyncio
import ssl
import traceback
import logging
logging.basicConfig(format="%(asctime)s [%(levelname)s] %(name)s: %(message)s", level=logging.DEBUG)
# logging.basicConfig(level=logging.DEBUG)

import pymodbus

pymodbus.pymodbus_apply_logging_config("DEBUG")

from pymodbus.client import ModbusTlsClient
# from pymodbus.client import AsyncModbusTlsClient
ip = 'xxxxxxxxxxxxxxxxxxxxx'
port = 802
address = 1000
slave = 1
count = 1

def _provide_ssl_context():
    sslctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
    sslctx.check_hostname = False
    sslctx.verify_mode = ssl.CERT_NONE

    sslctx.options |= ssl.OP_NO_TLSv1_1
    sslctx.options |= ssl.OP_NO_TLSv1
    sslctx.options |= ssl.OP_NO_SSLv3
    sslctx.options |= ssl.OP_NO_SSLv2

    return sslctx

client = ModbusTlsClient(host=ip, port=port, server_hostname=ip, sslctx=_provide_ssl_context())
client.connect()
client.read_holding_registers(slave, address, count)

Note: Same thing happens without _provide_ssl_context in older versions. We added it to make sure SSL does not accept the wrong SSL versions (requirement on our end) and does not fail because selfsigned cert. This is the exact same context as used by manual testing with SSL sockets, umodbus and modbus-tk.

...: DeprecationWarning: ssl.OP_NO_SSL*/ssl.OP_NO_TLS* options are deprecated
  sslctx.options |= ssl.OP_NO_TLSv1_1
...: DeprecationWarning: ssl.OP_NO_SSL*/ssl.OP_NO_TLS* options are deprecated
  sslctx.options |= ssl.OP_NO_TLSv1
2024-05-22 16:20:54,296 DEBUG logging:103 Current transaction state - IDLE
2024-05-22 16:20:54,296 [DEBUG] pymodbus.logging: Current transaction state - IDLE
2024-05-22 16:20:54,296 DEBUG logging:103 Running transaction 1
2024-05-22 16:20:54,296 [DEBUG] pymodbus.logging: Running transaction 1
2024-05-22 16:20:54,297 DEBUG logging:103 SEND: 0x3 0x0 0x1 0x3 0xe8
2024-05-22 16:20:54,297 [DEBUG] pymodbus.logging: SEND: 0x3 0x0 0x1 0x3 0xe8
2024-05-22 16:20:54,297 DEBUG logging:103 New Transaction state "SENDING"
2024-05-22 16:20:54,297 [DEBUG] pymodbus.logging: New Transaction state "SENDING"
2024-05-22 16:20:54,297 DEBUG logging:103 Changing transaction state from "SENDING" to "WAITING FOR REPLY"
2024-05-22 16:20:54,297 [DEBUG] pymodbus.logging: Changing transaction state from "SENDING" to "WAITING FOR REPLY"
2024-05-22 16:20:54,797 DEBUG logging:103 Transaction failed. (Modbus Error: [Connection] ModbusTlsClient xxxxxxxxxx:802: Connection unexpectedly closed 0.000 seconds into read of 2002 bytes without response from slave before it closed connection) 
2024-05-22 16:20:54,797 [DEBUG] pymodbus.logging: Transaction failed. (Modbus Error: [Connection] ModbusTlsClient xxxxxxxxxx:802: Connection unexpectedly closed 0.000 seconds into read of 2002 bytes without response from slave before it closed connection) 
2024-05-22 16:20:54,797 DEBUG logging:103 Processing: 
2024-05-22 16:20:54,797 [DEBUG] pymodbus.logging: Processing: 
2024-05-22 16:20:54,797 DEBUG logging:103 Getting transaction 1
2024-05-22 16:20:54,797 [DEBUG] pymodbus.logging: Getting transaction 1
2024-05-22 16:20:54,797 DEBUG logging:103 Changing transaction state from "PROCESSING REPLY" to "TRANSACTION_COMPLETE"
2024-05-22 16:20:54,797 [DEBUG] pymodbus.logging: Changing transaction state from "PROCESSING REPLY" to "TRANSACTION_COMPLETE"

Raw socket code

import socket
import ssl
import time

host = 'xxxxxxxxxxxxxxx'
port = 802

def _provide_ssl_context():
    sslctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
    sslctx.check_hostname = False
    sslctx.verify_mode = ssl.CERT_NONE

    sslctx.options |= ssl.OP_NO_TLSv1_1
    sslctx.options |= ssl.OP_NO_TLSv1
    sslctx.options |= ssl.OP_NO_SSLv3
    sslctx.options |= ssl.OP_NO_SSLv2

    return sslctx

context = _provide_ssl_context()

print("getdefaulttimeout", socket.getdefaulttimeout())

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ssock = context.wrap_socket(sock, server_hostname=host)
ssock.settimeout(3)
ssock.connect((host, port))
time.sleep(0.5)
print("TLS version:", ssock.version())
# print(time.time())
# print(ssock.getpeername())
print(ssock.send(b'\n{\x00\x00\x00\x06\x01\x03\x03\xe8\x00\x01'))
# print(ssock.getpeername())
# print(time.time())
print("recv:", ssock.recv(2002))
# <<< b'\n{\x00\x00\x00\x05\x01\x03\x02\x00\x02'
# print(ssock.getpeername())
# print(time.time())

umodbus

Works, We use a proprietary library layer I cannot share here, but the raw socket code above is the equivalent of what it ends up calling.

This is the relevant snippet I can share:

message = tcp.read_holding_registers(
    slave_id=slave, starting_address=address, quantity=count
)
retval = tcp.send_message(message, self.sock)

modbus-tk

#!/usr/bin/env python
# -*- coding: utf_8 -*-
"""
 Modbus TestKit: Implementation of Modbus protocol in python

 (C)2009 - Luc Jean - luc.jean@gmail.com
 (C)2009 - Apidev - http://www.apidev.fr

 This is distributed under GNU LGPL license, see license.txt
"""

from __future__ import print_function

import modbus_tk
import modbus_tk.defines as cst
from modbus_tk import modbus_tcp, hooks
import logging

host = 'xxxxxxxxxxxxxxxxxxxxx'
port = 802

def main():
    """main"""
    logger = modbus_tk.utils.create_logger("console", level=logging.DEBUG)

    def on_after_recv(data):
        master, bytes_data = data
        logger.info(bytes_data)

    hooks.install_hook('modbus.Master.after_recv', on_after_recv)

    try:

        def on_before_connect(args):
            master = args[0]
            logger.debug("on_before_connect {0} {1}".format(master._host, master._port))

        hooks.install_hook("modbus_tcp.TcpMaster.before_connect", on_before_connect)

        def on_after_recv(args):
            response = args[1]
            logger.debug("on_after_recv {0} bytes received".format(len(response)))

        hooks.install_hook("modbus_tcp.TcpMaster.after_recv", on_after_recv)

        # Connect to the slave
        master = modbus_tcp.TcpMaster(host=host, port=port)
        master.set_timeout(5.0)
        logger.info("connected")

        logger.info(master.execute(1, cst.READ_HOLDING_REGISTERS, 1000, 1))

    except modbus_tk.modbus.ModbusError as exc:
        logger.error("%s- Code=%d", exc, exc.get_exception_code())

if __name__ == "__main__":
    main()

[janiversen]

Seem you have it all covered, so please feel free to submit a pull request.

I am not personally using tls, so if you submit changes that work for you, then they are very likely to be accepted.

[dries007]

I'll have a talk with our PO, I can't guarantee we'll get the required time.

[janiversen]

Ok, one good argument is that you are using gratis software, which others spend a lot of unpaid time maintaining. The success of the library depends on many each making a small part.

[dries007]

Oh I know, and so does he, but I still have to ask.

[janiversen]

Closing as being not able to reproduce. We managed to test against a real device (thanks to my old colleagues at Siemens), and it worked both with/without providing sslctx.

There might still be a problem, but without help we cannot pinpoint it.

[dries007]

OK, I'll forward this to the responcible team. In the meantime I've figured out that internally that device is using pymodbus, which makes this double strange, but I suspect they are doing something non-standard. But it's unlikely to be your problem. Thanks for looking into it.

[janiversen]

OK.

Be aware if the device uses an old version of pymodbus, especially if it uses the 2.x versions, there are likely errors in the tis implementation. We changed quite tls a lot in the 3.5.x versions (any maybe also in 3.6.x, please see the release notes).

[jla]

@dries007 Maybe is not the same problem but in my case it was related to #2014

from pymodbus.framer import FramerType

client = ModbusTlsClient(host=ip, port=port, server_hostname=ip, framer=FramerType.SOCKET, sslctx=_provide_ssl_context())

[dries007]

Thanks! I'll check it out