Unblock python threads in SOEM blocking functions

[GullinBustin]

In Python, when 2 or more threads are created they can't work in parallel because of GIL. Some functions in Python (such as sleep) allow running another thread during its execution. In PySOEM, sdo_read and sdo_write (for example) await the slave response, but this wait does not unblock Python threads.

This pull request solves this problem for windows by modifying the osal.c and nicdrv.c files and unblock Python threads in the blocking functions, as osal_usleep. Fix the problem on osal_usleep is easy, but for ecx_waitinframe_red the only way is to add an osal_usleep call to reduce the number of iterations and allow running other threads meanwhile.

Tests

I made a manual test to verify SDO reads do not block the threads. The test has a main thread and a secondary thread:

• The main thread has a loop that makes SDO reads (40 iterations).
• Meanwhile, the secondary thread has a loop that saves the current timestamp and does a little sleep (5 ms) (100 iterations).
• Finally, the list of timestamps saved is plotted.

The result of the new implementation: The secondary thread is hardly affected by the SDO reads.

The result of the old implementation: The secondary thread is very affected by the SDO reads. When the SDO reads loop ends, the secondary thread works like in the new implementation case.

Test code:

import time
from threading import Thread

import pysoem
import matplotlib.pyplot as plt

THREAD_ITERATIONS = 40
READ_ITERATIONS = 100
interface_name = "\\Device\\NPF_{18CCD8AA-98F3-46F2-BE78-DAE9CC53D7BB}"
times = []

def show_plot(y_data):
    x_data = list(range(len(y_data)))
    last_point = len(y_data)-1
    middle_point = len(y_data)//2
    m = (y_data[last_point] - y_data[middle_point]) / \
        (x_data[last_point] - x_data[middle_point])
    b = (y_data[last_point] - m * x_data[last_point])
    # Calculates the lineal equation with the last point and the middle point

    fig, ax = plt.subplots()
    ax.plot(x_data, y_data, 'o')
    ax.plot([0, x_data[last_point]], [b, y_data[last_point]],
            label=f"y={m:.4f}x+{b:.4f}")
    ax.set(xlabel='N', ylabel='time (s)')
    ax.legend()
    ax.grid()
    plt.show()

def thread_func():
    for _ in range(THREAD_ITERATIONS):
        times.append(time.time())
        time.sleep(0.005)

master = pysoem.Master()
master.open(interface_name)
master.config_init()
slave = master.slaves[0]
thread_instance = Thread(target=thread_func)
thread_instance.start()
for _ in range(READ_ITERATIONS):
    status_word = slave.sdo_read(0x2011, 0)
thread_instance.join()
times_with_0 = [(x - times[0]) for x in times]
show_plot(times_with_0)

[GullinBustin]

@bnjmnp I added comments in the main changes of this pull request. A big percentage of the changes are only a copy of SOEM files. The real changes are to add Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS to the blocking functions and replace the files in setup.py.

[bnjmnp]

@GullinBustin: Thank you for sharing your ideas.

What is the issue with SDO communication being slow? Cyclic PDO communication is actually the more time critical part of EtherCAT.

Did you check if your changes might have a negative effect on PDO cycles running in parallel?

Also, did you consider patching my fork of the SOEM repo (submodule) in the same way I already patched it? This would be a more clean way of introducing changes to the C code? Consider the case when pople introduced fixes or improvements to the main SOEM repo. It is much nicer to apply the patches with the forked SOEM submodule method, than manually updating the local copies of osal.c and nicdrv.c.

[GullinBustin]

@bnjmnp Thanks for your reply.

I didn't notice that the submodule points to a fork of your own instead of the original SOEM repo. Definitely, that changes must be done in the SOEM repository.

About effects on the PDO communications, it should not affect, but I will do better testing.

What is the issue with SDO communication being slow? Cyclic PDO communication is actually the more time critical part of EtherCAT.

About this question, the problem is not SDO communications are slow, the problem is that all the communications by EtherCAT (PDO too) block Python threads. In my case, it is a problem because I have a program with a GUI and the SDOs block the GUI thread. I want to do something similar to what the Python socket module does, where threads are unblocked when it waits for socket response: https://github.com/python/cpython/blob/main/Modules/socketmodule.c#L967. But in this case, doing it is not trivial.

In conclusion, I will close this PR because these changes should be done in SOEM repository, but I will study it more before creating the new PR because I'm not completely happy with the current solution. Add sleep works for me, but it could be problematic depending on the devices.