libusb stress_mt.c conversion to python-libusb1

[mcuee]

Today I am trying out using ChatGPT to convert libusb tests/stress_mt.c to more portable codes.

Original pthread based C code. https://github.com/libusb/libusb/blob/master/tests/stress_mt.c Discussion:

• https://github.com/libusb/libusb/issues/1128

One exercise is to convert it to python-libusb1 (also pyusb). Here is the converted code after quite a few trial and errors from ChatGPT and it seems to run.

@vpelletier Two questions from my side. 1) Is the converted code correct? 2) Is the result correct or not?

import threading
import time
import usb1

class StressTest(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)
        self.dev = None

    def run(self):
        with usb1.USBContext() as context:
            dev_list = context.getDeviceList(skip_on_error=True)
            if not dev_list:
                print("No device found")
                return

            self.dev = dev_list[0]
            handle = self.dev.open()

            for i in range(5):
                try:
                    desc = self.dev.getProductDescriptor()
                    print(desc)
                except:
                    pass

            handle.close()

def main():
    num_threads = 4
    threads = []

    for i in range(num_threads):
        threads.append(StressTest())

    for thread in threads:
        thread.start()

    for thread in threads:
        thread.join()

if __name__ == "__main__":
    main()

Results:

(py310x64venv) PS C:\work\libusb\libusb_test\stress_mt_python> python .\stress_mt_usb1.py
Exception in thread Thread-2:
Traceback (most recent call last):
  File "C:\Users\xiaof\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner
Exception in thread Thread-4:
Traceback (most recent call last):
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Users\xiaof\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner
  File "C:\Users\xiaof\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner
Exception in thread Thread-3:
Traceback (most recent call last):
  File "C:\Users\xiaof\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "C:\work\libusb\libusb_test\stress_mt_python\stress_mt_usb1.py", line 18, in run
    self.run()
  File "C:\work\libusb\libusb_test\stress_mt_python\stress_mt_usb1.py", line 18, in run
    handle = self.dev.open()
  File "C:\work\python\py310x64venv\lib\site-packages\usb1\__init__.py", line 2055, in open
    self.run()
  File "C:\work\libusb\libusb_test\stress_mt_python\stress_mt_usb1.py", line 18, in run
    self.run()
  File "C:\work\libusb\libusb_test\stress_mt_python\stress_mt_usb1.py", line 18, in run
    mayRaiseUSBError(libusb1.libusb_open(self.device_p, byref(handle)))
  File "C:\work\python\py310x64venv\lib\site-packages\usb1\__init__.py", line 127, in mayRaiseUSBError
    handle = self.dev.open()
  File "C:\work\python\py310x64venv\lib\site-packages\usb1\__init__.py", line 2055, in open
    handle = self.dev.open()
  File "C:\work\python\py310x64venv\lib\site-packages\usb1\__init__.py", line 2055, in open
    handle = self.dev.open()
  File "C:\work\python\py310x64venv\lib\site-packages\usb1\__init__.py", line 2055, in open
    __raiseUSBError(value)
  File "C:\work\python\py310x64venv\lib\site-packages\usb1\__init__.py", line 119, in raiseUSBError
    mayRaiseUSBError(libusb1.libusb_open(self.device_p, byref(handle)))
  File "C:\work\python\py310x64venv\lib\site-packages\usb1\__init__.py", line 127, in mayRaiseUSBError
    mayRaiseUSBError(libusb1.libusb_open(self.device_p, byref(handle)))
    mayRaiseUSBError(libusb1.libusb_open(self.device_p, byref(handle)))
  File "C:\work\python\py310x64venv\lib\site-packages\usb1\__init__.py", line 127, in mayRaiseUSBError
  File "C:\work\python\py310x64venv\lib\site-packages\usb1\__init__.py", line 127, in mayRaiseUSBError
    raise __STATUS_TO_EXCEPTION_DICT.get(value, __USBError)(value)
    __raiseUSBError(value)
  File "C:\work\python\py310x64venv\lib\site-packages\usb1\__init__.py", line 119, in raiseUSBError
    __raiseUSBError(value)
  File "C:\work\python\py310x64venv\lib\site-packages\usb1\__init__.py", line 119, in raiseUSBError
usb1.USBErrorNotSupported: LIBUSB_ERROR_NOT_SUPPORTED [-12]
    __raiseUSBError(value)
  File "C:\work\python\py310x64venv\lib\site-packages\usb1\__init__.py", line 119, in raiseUSBError
    raise __STATUS_TO_EXCEPTION_DICT.get(value, __USBError)(value)
    raise __STATUS_TO_EXCEPTION_DICT.get(value, __USBError)(value)
    raise __STATUS_TO_EXCEPTION_DICT.get(value, __USBError)(value)
usb1.USBErrorNotSupported: LIBUSB_ERROR_NOT_SUPPORTED [-12]
usb1.USBErrorNotSupported: LIBUSB_ERROR_NOT_SUPPORTED [-12]
usb1.USBErrorNotSupported: LIBUSB_ERROR_NOT_SUPPORTED [-12]

[mcuee]

ChatGPT thinks python-libusb1 could be the problem. :-)

The error messages indicate that there is an issue with the underlying library libusb1. It's possible that the library is not fully compatible with your system or with the way you're using it. Without more information, it's difficult to pinpoint the exact cause of the errors.

You could try updating the libusb1 library to the latest version and see if that resolves the issue. You can also try running the code on a different machine to see if the problem persists.

If the issue persists, you could try using a different library for USB communication, such as pyusb.

FYI: here is the result for pyusb.

• https://github.com/pyusb/pyusb/issues/472

[mcuee]

Linux results seem to be strange as well.

mcuee@UbuntuSwift3:~/build/python/stress_mt_python$ sudo python3 stress_mt_usb1.py 
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2

[mcuee]

macOS result is the same as Linux, using Mac Mini M1.

(py310venv) mcuee@mcuees-Mac-mini stress_mt_python % python stress_mt_usb1.py 
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2

[mcuee]

Just want to state what I want to achieve through the chatgpt experiment.

1) Not to generate codes which will be merged -- only for testing purpose. This should avoid copyright concerns.

2) Reuse test cases across libusb and the two python bindings -- pyusb and python-libusb1. In this example, the libusb C code is the start. But often it may be easier to write a test case in Python than C.

3) Test method -- to verify the test results across Windows, macOS and Linux to see if there are potential problems. For example, if the results under Linux and macOS are the same, but there is a problem under Windows, then there is a potential problems under Windows. It may well be there is a valid reason why Windows performs different but it may also be possible that there is a real problem under Windows.

Specifically to the stress_mt.c tests, I suspect there is an issue with the libusb Windows backend when it comes to ref count. The converted C++11 codes seem to be able to expose the problem (to be confirmed).

[mcuee]

@vpelletier

Take note the above code does not match the intention of stress_mt.c which is to be a test "that creates and destroys contexts repeatedly".

The following code is generated by ChatGPT to match stress_mt.c and it seems to be work fine under Windows.

import usb1
import threading

NUM_THREADS = 16
NUM_ITERATIONS = 10000

def thread_func(thread_id, context):
    for i in range(NUM_ITERATIONS):
        try:
            with context:
                # Perform libusb operations using the context
                pass
        except usb1.USBError:
            print(f"Thread {thread_id} failed with error code {usb1.USBError.errno}")
            return

if __name__ == '__main__':
    threads = []

    for i in range(NUM_THREADS):
        with usb1.USBContext() as ctx:
            threads.append(threading.Thread(target=thread_func, args=(i, ctx)))

    for thread in threads:
        thread.start()

    for thread in threads:
        thread.join()

    print("All threads completed successfully")

(py310x64venv) PS C:\work\libusb\libusb_test\stress_mt_python> python .\stress_mt_usb1_context.py
All threads completed successfully

[vpelletier]

Sorry for not replying sooner.

1. Is the converted code correct?

At a glance, I see 3 differences:

• the biggest one, which you noted in your latest post: the python code goes beyond what the C code does: it not only creates (and tears down) the libusb context, but also lists present USB devices, opens one and fetches the product descriptor. The C code only creates and tears down the libusb context.
• the python code iterates much fewer times in each thread
• the python code spawns fewer threads

Then, there is a non-trivial limitation of the python interpreter: only one thread can be executing python code at any one time. "Non-python code" would be code from a C library, or more exotic things like waiting for IO. This means that a python-based example will be at a disadvantage when trying to trigger race-conditions, especially if the non-python code runs for only very little time on each iteration (compared to the time spent in python handling the loop). I cannot tell at a glance if this is the case, I'm just pointing this out as a possible source of discrepancies in the results.

2. Is the result correct or not?

There are 2 results:

On Windows: 4 exceptions get raised (one per thread) during open, with a consistent LIBUSB_ERROR_NOT_SUPPORTED error. I guess the open operation is not supported for whatever device is being opened. I'm going to guess this is the root hub. Of course, all stack traces are interleaved because of the multi-threading. As exceptions do not cross thread boundaries, all 4 threads get to the end of the error and then the main thread can join them and exit.

On Linux and OSX: the number 2 gets printed 20 times (4 threads times 5 iterations). This is the product descriptor, and my guess is that this is an USB2 root hub:

$ lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

So, to me the output looks reasonable, and not the sign of any bug (...if my assumptions are correct).

The following code is generated by ChatGPT to match stress_mt.c and it seems to be work fine under Windows.

This is indeed closer. The iteration and thread counts differs, but this is a minor divergence.

There is one unnecessary line: the thread creation loop can be simplified to:

    for i in range(NUM_THREADS):
          threads.append(threading.Thread(target=thread_func, args=(i, usb1.USBContext())))

but this should not matter: it is just that the main thread will be creating and then tearing down its own USB contexts.

Finally, just to clarify python's behaviour, the All threads completed successfully line will be printed whatever happened in the threads. So if the intended meaning is like "all threads could be joined" it is correct, but it does not mean that there were no error in any thread.

[mcuee]

Thanks a lot for the detail explanations.

[mcuee]

Commenting out the extra stuff and it seems to be running okay under Windows now.

(py310venv) C:\work\libusb\libusb_tests\stress_mt> cat .\stress_mt_usb1.py
import threading
import time
import usb1

class StressTest(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)
        self.dev = None

    def run(self):
        with usb1.USBContext() as context:
            dev_list = context.getDeviceList(skip_on_error=True)
            if not dev_list:
                print("No device found")
                return

def main():
    num_threads = 4
    threads = []

    for i in range(num_threads):
        threads.append(StressTest())

    for thread in threads:
        thread.start()

    for thread in threads:
        thread.join()

    print("Test ends here!")

if __name__ == "__main__":
    main()

(py310venv) C:\work\libusb\libusb_tests\stress_mt> python .\stress_mt_usb1.py
Test ends here!

[mcuee]

This may be a bit more similar to the new stress_mt.c here.

• https://github.com/libusb/libusb/pull/1252

(py310x64venv) PS C:\work\libusb\libusb_test\stress_mt_python> cat stress_mt_usb1_context_list.py
import usb1
import threading

NUM_THREADS = 4
NUM_ITERATIONS = 8

def thread_func(thread_id, context):
    for i in range(NUM_ITERATIONS):
        try:
            with context:
                # Perform libusb operations using the context
                dev_list = context.getDeviceList(skip_on_error=True)
                if not dev_list:
                    print("No device found")
                else:
                    print(f"Thread {thread_id} found", len(dev_list), " devices in iteration ", i)

        except usb1.USBError:
            print(f"Thread {thread_id} failed with error code {usb1.USBError.errno}")
            return

if __name__ == '__main__':
    threads = []

    for i in range(NUM_THREADS):
        with usb1.USBContext() as ctx:
            threads.append(threading.Thread(target=thread_func, args=(i, ctx)))

    for thread in threads:
        thread.start()

    for thread in threads:
        thread.join()

    print("Test finished!")
(py310x64venv) PS C:\work\libusb\libusb_test\stress_mt_python> python .\stress_mt_usb1_context_list.py
Thread 0 found 8  devices in iteration  0
Thread 2 found 8  devices in iteration  0
Thread 1 found 8  devices in iteration  0
Thread 3 found 8  devices in iteration  0
Thread 0 found 8  devices in iteration  1
Thread 2 found 8  devices in iteration  1
Thread 1 found 8  devices in iteration  1
Thread 3 found 8  devices in iteration  1
Thread 0 found 8  devices in iteration  2
Thread 2 found 8  devices in iteration  2
Thread 3 found 8  devices in iteration  2
Thread 1 found 8  devices in iteration  2
Thread 0 found 8  devices in iteration  3
Thread 2 found 8  devices in iteration  3
Thread 3 found 8  devices in iteration  3
Thread 1 found 8  devices in iteration  3
Thread 0 found 8  devices in iteration  4
Thread 3 found 8  devices in iteration  4
Thread 2 found 8  devices in iteration  4
Thread 1 found 8  devices in iteration  4
Thread 0 found 8  devices in iteration  5
Thread 3 found 8  devices in iteration  5
Thread 1 found 8  devices in iteration  5
Thread 2 found 8  devices in iteration  5
Thread 0 found 8  devices in iteration  6
Thread 3 found 8  devices in iteration  6
Thread 1 found 8  devices in iteration  6
Thread 2 found 8  devices in iteration  6
Thread 3 found 8  devices in iteration  7
Thread 0 found 8  devices in iteration  7
Thread 2 found 8  devices in iteration  7
Thread 1 found 8  devices in iteration  7
Test finished!