vpelletier / python-functionfs

Pythonic API for linux's functionfs
GNU General Public License v3.0
40 stars 13 forks source link

More examples #20

Closed mcuee closed 2 years ago

mcuee commented 2 years ago

It will be great to have more examples.

mcuee commented 2 years ago

Example from here: https://github.com/libusb/libusb/pull/965#issuecomment-887915009 It would be nice to have a Python example of the above shell script example.

#! /bin/bash

modprobe libcomposite
mount -t configfs none /sys/kernel/config
mkdir -p /sys/kernel/config/usb_gadget/g1
cd /sys/kernel/config/usb_gadget/g1

###############################
# Populate Device-Level Stuff #
###############################

#Setting device class/subclass/protocol to these values
# alerts the OS that this is a composite device with
# IADs in it's firmware.
# ref: https://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/usb-interface-association-descriptor
echo "0xEF" > bDeviceClass
echo "0x02" > bDeviceSubClass
echo "0x01" > bDeviceProtocol
echo "0x1d6b" > idVendor
echo "0x0104" > idProduct

mkdir strings/0x409
echo "1234567" > strings/0x409/serialnumber
echo "Some Manufacturer" > strings/0x409/manufacturer
echo "Some Product" > strings/0x409/product

#enable use of os_desc's (important for RNDIS & NCM enablement on Windows):
echo 1       > os_desc/use
echo 0xbc    > os_desc/b_vendor_code
echo MSFT100 > os_desc/qw_sign

#################################
# Populate Individual Functions #
#################################

#The order functions are populated here will be reflected in the
# order of descriptors written.

#########
# RNDIS #
#########
#Note! If RNDIS is enabled, it *has* to be the first function! Otherwise, Windows 10 will report error 10 (failed to start device).
# (It's unclear why this is the case..)
# https://docs.microsoft.com/en-us/answers/questions/474108/does-rndis-need-to-be-listed-as-the-first-function.html
# https://stackoverflow.com/questions/68365739/windows-rndis-compatible-device-does-rndis-need-to-be-listed-as-the-first-funct
if ((1))
then
    mkdir functions/rndis.usb0
    mkdir -p functions/rndis.usb0/os_desc/interface.rndis

    #Set compatible / sub-compatible id's so that Windows can match this
    # function to RNDIS6 driver more easily.
    echo RNDIS   > functions/rndis.usb0/os_desc/interface.rndis/compatible_id
    echo 5162001 > functions/rndis.usb0/os_desc/interface.rndis/sub_compatible_id

    mkdir -p configs/c.1
    mkdir -p configs/c.1/strings/0x409
    echo "conf1" > configs/c.1/strings/0x409/configuration
    ln -s functions/rndis.usb0 configs/c.1
    if [ ! -L os_desc/c.1 ]
    then
        ln -s configs/c.1 os_desc
    fi
fi

#########
# NCM   #
#########
#Usually I test with *either* RNDIS or NCM enabled, but not both, hence the if(0) here..
if ((0))
then
    mkdir functions/ncm.usb0
    mkdir -p functions/ncm.usb0/os_desc/interface.ncm
    #Set compatible id so that Windows 10 can match this function to
    # NCM driver more easily.
    echo WINNCM   > functions/ncm.usb0/os_desc/interface.ncm/compatible_id

    mkdir -p configs/c.1
    mkdir -p configs/c.1/strings/0x409
    echo "conf1" > configs/c.1/strings/0x409/configuration
    ln -s functions/ncm.usb0 configs/c.1
    if [ ! -L os_desc/c.1 ]
    then
         ln -s configs/c.1 os_desc
    fi
fi

#########
# ACM   #
#########
if ((1))
then
    mkdir -p functions/acm.GS0

    mkdir -p configs/c.1
    mkdir -p configs/c.1/strings/0x409
    ln -fs functions/acm.GS0 configs/c.1
    if [ ! -L os_desc/c.1 ]
    then
         ln -s configs/c.1 os_desc
    fi
fi

#bind..
# Chances are, you need to replace this line with the platform-specific UDC.
# Use 'ls /sys/class/udc' to see available UDCs
echo "using 12480000.hsotg for UDC.. you probably need to replace this.."
echo 12480000.hsotg > UDC
mcuee commented 2 years ago

Or things here: https://www.collabora.com/news-and-blog/blog/2019/02/18/modern-usb-gadget-on-linux-and-how-to-integrate-it-with-systemd-part-1/

And here: https://wiki.tizen.org/USB/Linux_USB_Layers/Configfs_Composite_Gadget/Usage_eq._to_g_zero.ko

An example gadget with Loopback and SourceSink functions:

$ modprobe libcomposite

$ mount none cfg -t configfs

$ mkdir cfg/usb_gadget/g1
$ cd cfg/usb_gadget/g1

$ mkdir configs/c.1
$ mkdir configs/c.2
$ mkdir functions/Loopback.0
$ mkdir functions/SourceSink.0

$ mkdir strings/0x409
$ mkdir configs/c.1/strings/0x409
$ mkdir configs/c.2/strings/0x409

$ echo 0x2d01 > idProduct
$ echo 0x04e8 > idVendor

$ echo my-serial-num > strings/0x409/serialnumber
$ echo my-manufacturer > strings/0x409/manufacturer
$ echo "Test gadget" > strings/0x409/product

$ echo "Conf 1" > configs/c.1/strings/0x409/configuration
$ echo "Conf 2" > configs/c.2/strings/0x409/configuration
$ echo 120 > configs/c.1/MaxPower
$ ln -s functions/Loopback.0 configs/c.1
$ ln -s functions/SourceSink.0 configs/c.2

$ echo s3c-hsotg > cfg/usb_gadget/g1/UDC
vpelletier commented 2 years ago

For these examples, I do not think it makes much sense to convert it to python: while the gadget setup could be converted into python code calling into functionfs.gadget, the rest is merely about loading USB functions which are fully implemented in-kernel, and not relying on functionfs to implement these in userland.

OTOH, I have been wondering how I would go to help a developer setup a mixed functionfs + (some in-kernel USB functions). To fit into the existing design, one would have to implement a class inheriting from functionfs.gadget.ConfigFunctionBase, implementing its API and exposing whatever methods & constructor arguments make sense for that USB function type, and provide it to functionfs.gadget.Gadget as part of its config_list constructor argument. I expect such class to be quite empty, and it would have to be updated whenever the kernel parameters for that function change, so it would not provide much added value and be a maintenance cost. Which is why none exist at the moment.

mcuee commented 2 years ago

Fair enough.

Personally I find there are very few good USB Gadgets examples and libraries. I was trying out libusbgx/gt (https://github.com/linux-usb-gadgets/gt) before I found this python library. And I think this library has very good potentials based on my quick testing.

One of the issue I have is usb_f_ss_lb module with regard to the SourceSink function. It seems to me the sourcesink function can have the isoc endpoint as well but I do not find any examples to enable isoc endpoints. https://www.kernel.org/doc/html/latest/usb/gadget-testing.html#sourcesink-function https://github.com/torvalds/linux/blob/master/drivers/usb/gadget/function/f_sourcesink.c

mcuee commented 2 years ago

Background: https://github.com/pyusb/pyusb/issues/235 https://github.com/LibUsbDotNet/LibUsbDotNet/issues/137#issuecomment-856803235

USB related open source projects will often need a test device. I think Linux USB Gadget can be a good choice.

Personally I have quite some USB devices including USB PIC and ARM MCU based boards, and Cypress FX2LP/FX3 based boards. I think the following Cypress FX3 board is excellent at US$49. https://www.cypress.com/documentation/development-kitsboards/cyusb3kit-003-ez-usb-fx3-superspeed-explorer-kit

But then not everyone is comfortable at using Cypress's IDE or ARM GCC to create USB FW. Linux USB Gadgets seem to be a good choices in that case. It is cool using shell scripts or python scripts to create USB devices (mostly FS/HS only, not so much superspeed like the FX3 but should be good enough for most cases).

vpelletier commented 2 years ago

USB related open source projects will often need a test device. I think Linux USB Gadget can be a good choice.

Indeed. And, at least when testing on Linux, they should be able to get away without any actual hardware token, but using the dummy-hcd driver. It is a very good idea indeed, making a unittest controlling the UDC on one hand and accessing with libusb on the other.

vpelletier commented 2 years ago

About examples, my main use of python-functionfs (and the reason I wrote it, actually, and then went on to write python-libaio) is in my userland implementation of the USB-CCID spec.

It is put into a gadget in my implementation of the OpenPGP smartcard specification, with two variants:

I do not know if this really qualifies as an example: on the one hand, it works. On the other hand, it may not be easy to follow as there is a lot going on.

mcuee commented 2 years ago

One of the issue I have is usb_f_ss_lb module with regard to the SourceSink function. It seems to me the sourcesink function can have the isoc endpoint as well but I do not find any examples to enable isoc endpoints.

I think I know the reason, my previous OrangePi Zero may not support this feature at all.

It works under my Raspberry Pi 400 with g_zero.


~ ❯ lsusb1 -vvv -d 0525:a4a0

Bus 002 Device 025: ID 0525:a4a0  
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass          255 
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  idVendor           0x0525 
  idProduct          0xa4a0 
  bcdDevice            5.11
  iManufacturer           1 
  iProduct                2 
  iSerial                 3 
  bNumConfigurations      2
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x0045
    bNumInterfaces          1
    bConfigurationValue     3
    iConfiguration          4 source and sink data
    bmAttributes         0xc0
      Self Powered
    MaxPower                2mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass       255 
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x01  EP 1 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       1
      bNumEndpoints           4
      bInterfaceClass       255 
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x01  EP 1 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x82  EP 2 IN
        bmAttributes            1
          Transfer Type            Isochronous
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               4
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            1
          Transfer Type            Isochronous
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               4
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x0020
    bNumInterfaces          1
    bConfigurationValue     2
    iConfiguration          5 loop input to output
    bmAttributes         0xc0
      Self Powered
    MaxPower                2mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass       255 
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 
      iInterface              6 loop input to output
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x01  EP 1 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass          255 
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  bNumConfigurations      2
can't get debug descriptor: No such file or directory
Device Status:     0x0001
  Self Powered
vpelletier commented 2 years ago

I think I know the reason, my previous OrangePi Zero may not support this feature at all.

I was just testing this on dummy_hcd, and if I set isoc_maxpacket to 64 I get IOError: [Errno 16] Device or resource busy when attaching the gadget to the bus. I guess this UDC does not support isochronous.

And to re-loop with the discussion above, I wrote a minimal gadget loading SourceSink, and the class is even shorter than I expected: if I do not implement any configuration, it is just:

class SourceSinkFunction(ConfigFunctionBase):
    type_name='SourceSink'

    def start(self, path):
        return

    def wait(self):
        return

    def kill(self):
        return

    def join(self):
        return

Of course, more "useful" functions may need more code: a USB network card could need to trigger some IP setup... But even then, maybe this can be done by (gadget-side) OS hotplug configuration functionalities, like NetworkManager noticing the interface appeared.

mcuee commented 2 years ago

For these examples, I do not think it makes much sense to convert it to python: while the gadget setup could be converted into python code calling into functionfs.gadget, the rest is merely about loading USB functions which are fully implemented in-kernel, and not relying on functionfs to implement these in userland.

Good point. Those examples I quoted including the following are using configFS. And probably shell scripts are good enough. https://wiki.tizen.org/USB/Linux_USB_Layers/Configfs_Composite_Gadget https://github.com/linux-usb-gadgets/gt

It seems to me there are very few examples for functionfs except yours and the following. https://github.com/linux-usb-gadgets/ptp-gadget (C based example for PTP gadget) https://github.com/Vogtinator/usbredir2phys ( C++ based example, to turn virtual USB connections usbredir into physical ones) https://www.collabora.com/news-and-blog/blog/2019/03/27/modern-usb-gadget-on-linux-and-how-to-integrate-it-with-systemd-part-2/

mcuee commented 2 years ago

I think I will close this issue. The bundled simpler examples, plus https://github.com/vpelletier/python-usb-f-ccid, should server as a good start for the users.

vpelletier commented 2 years ago

if I set isoc_maxpacket to 64 I get IOError: [Errno 16] Device or resource busy when attaching the gadget to the bus. I guess this UDC does not support isochronous.

For completeness (if someone gets this error and finds this bug report), this was fixed by b9d25398185b7c223db761956b10b1a7d1fd4ed1 .

mcuee commented 2 years ago

if I set isoc_maxpacket to 64 I get IOError: [Errno 16] Device or resource busy when attaching the gadget to the bus. I guess this UDC does not support isochronous.

For completeness (if someone gets this error and finds this bug report), this was fixed by b9d2539 .

Is it possible to include this isoc sourcesink sample then? There are not many known working isoc firmware.

vpelletier commented 2 years ago

I whipped up a quick example with improved functionfs.gadget-level ease of integration: c654d74ec65b62352dabffb1eb4141bb45a2dc7a .

It still needs to work with a more complex kernel-provided function before I can feel comfortable merging it. Contributions welcome ;) .

vpelletier commented 2 years ago

...and I went ahead and pushed examples for CDC NCM and mass storage. So far I feel these examples are rather boring (even though I did not expose all mass-storage features and instead went overboard with the NCM one). Boring can be a good sign: it means that python-functionfs is making the creation of gadgets easy. Hopefully.

mcuee commented 2 years ago

I tried the CDC NCM example and it seems to work. Windows 10 host needs extra step to use it. Tested with OrangePi Zero. https://docs.microsoft.com/en-us/answers/questions/52386/usb-cdc-ncm-devices-arenamp39t-assigned-usbncm-dri.html

mcuee commented 2 years ago

USB Mass Storage example works fine.


mcuee@orangepizero:~/gadget/python-functionfs_wip$ dd bs=1M count=64 if=/dev/zero of=../tmp/backingfile
64+0 records in
64+0 records out
67108864 bytes (67 MB, 64 MiB) copied, 0.660765 s, 102 MB/s
mcuee@orangepizero:~/gadget/python-functionfs_wip$ sudo python3 examples/kernel_mass_storage/device.py ../tmp/backingfile
mcuee commented 2 years ago

As for the Sourcesink demo, it seems to me that I can not get it to work with OrangePi Zero. Firstly I think it is the limitation of the OrangePi Zero that no isoc endpoints are available. That is kind of expected. However, reading does not seem to work. I tend to think it should work. Probably a python-libusb1 based example is good to have.

Host Code from https://github.com/vpelletier/python-libusb1/issues/72#issuecomment-913252143

(py39venv) C:\work\libusb\python_usb [master ≡ +10 ~0 -0 !]> python .\test_usb_gadget.py
Number of bytes written:  100
write: True
Traceback (most recent call last):
  File "C:\work\libusb\python_usb\test_usb_gadget.py", line 100, in <module>
    print("read:", usb.read())
  File "C:\work\libusb\python_usb\test_usb_gadget.py", line 75, in read
    data += handle.bulkRead(self.endpoint_in,
  File "C:\work\python\py39venv\lib\site-packages\usb1\__init__.py", line 1627, in bulkRead
    transferred = self._bulkTransfer(endpoint, data, length, timeout)
  File "C:\work\python\py39venv\lib\site-packages\usb1\__init__.py", line 1581, in _bulkTransfer
    return transferred.value
KeyboardInterrupt
Calling closing method to delete self
Closing USB
Close handle successfully
mcuee commented 2 years ago

The SourceSink example works better under Raspberry Pi (Raspberry Pi OS 32bit).

From my host side (x64 Ubuntu Linux 21.04). But the host test apps does not seem to do well.

Test code: https://github.com/torvalds/linux/blob/master/tools/usb/testusb.c

(mypy39venv) mcuee@ubuntu64:~/build/libusb/python_usb$ lsusb -vvv -d 1d6b:0104

Bus 001 Device 029: ID 1d6b:0104 Linux Foundation Multifunction Composite Gadget
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0 
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  idVendor           0x1d6b Linux Foundation
  idProduct          0x0104 Multifunction Composite Gadget
  bcdDevice            5.10
  iManufacturer           1 python-functionfs
  iProduct                2 SourceSink demo
  iSerial                 3 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x0045
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          4 SourceSink demo function
    bmAttributes         0x80
      (Bus Powered)
    MaxPower              500mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x01  EP 1 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       1
      bNumEndpoints           4
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x01  EP 1 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x82  EP 2 IN
        bmAttributes            1
          Transfer Type            Isochronous
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               4
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            1
          Transfer Type            Isochronous
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               4
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass            0 
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  bNumConfigurations      1
can't get debug descriptor: Resource temporarily unavailable
Device Status:     0x0000
  (Bus Powered)

(mypy39venv) mcuee@ubuntu64:~/build/libusb/python_usb$ sudo modprobe usbtest vendor=0x1d6b product=0x0104
(mypy39venv) mcuee@ubuntu64:~/build/libusb/python_usb$ sudo ./testusb -D /dev/bus/usb/001/029 
./testusb: /dev/bus/usb/001/029 may see only control tests
/dev/bus/usb/001/029 test 0,    0.000012 secs
/dev/bus/usb/001/029 test 9,    3.637028 secs
/dev/bus/usb/001/029 test 10,    2.069410 secs
mcuee commented 2 years ago

The legacy g_zero driver works much better with the host usbtest code (which uses usb_f_ss_lb as well).

(mypy39venv) mcuee@ubuntu64:~/build/libusb/python_usb$ sudo modprobe usbtest vendor=0x0525 product=0xa4a0
(mypy39venv) mcuee@ubuntu64:~/build/libusb/python_usb$ sudo ./testusb -D /dev/bus/usb/001/031 
unknown speed   /dev/bus/usb/001/031    0
/dev/bus/usb/001/031 test 0,    0.000011 secs
/dev/bus/usb/001/031 test 1,    0.047929 secs
/dev/bus/usb/001/031 test 2,    0.045963 secs
/dev/bus/usb/001/031 test 3,    0.044616 secs
/dev/bus/usb/001/031 test 4,    0.045561 secs
/dev/bus/usb/001/031 test 5,    1.222463 secs
/dev/bus/usb/001/031 test 6,    1.145457 secs
/dev/bus/usb/001/031 test 7,    1.128337 secs
/dev/bus/usb/001/031 test 8,    1.117154 secs
/dev/bus/usb/001/031 test 9,    3.701147 secs
/dev/bus/usb/001/031 test 10,    2.054418 secs
/dev/bus/usb/001/031 test 11,   20.614050 secs
/dev/bus/usb/001/031 test 12,   19.179533 secs
/dev/bus/usb/001/031 test 13,    1.323587 secs
/dev/bus/usb/001/031 test 14 --> 22 (Invalid argument)
/dev/bus/usb/001/031 test 17,    0.043113 secs
/dev/bus/usb/001/031 test 18,    0.042700 secs
/dev/bus/usb/001/031 test 19,    0.042244 secs
/dev/bus/usb/001/031 test 20,    0.044089 secs
/dev/bus/usb/001/031 test 21 --> 22 (Invalid argument)
/dev/bus/usb/001/031 test 24,    1.904141 secs
/dev/bus/usb/001/031 test 27,    1.001101 secs
/dev/bus/usb/001/031 test 28,    1.000669 secs
/dev/bus/usb/001/031 test 29,    0.184871 secs
vpelletier commented 2 years ago

I suspect these issues probably come from the kernel module and the hardware (UDCs seem to vary in features quite a lot): in these examples my code does virtually nothing once the gadget is attached to the bus, and before that I think it does very little which should be able to influence the functions (failure should rather result in the gadget not getting on the bus at all, or with unexpected device/configuration descriptors).

Maybe some options need to be set (in which case I should to make them available to the command line) ?

mcuee commented 2 years ago

Since I am using the same Raspberry Pi (Raspberry Pi OS 32bit) in the above test, I think there is a difference between g_zero and the python code.

g_zero is equivalent to loopback + sourcesink. I think you do not need to have that two configurations (which is not well supported by Windows), just single sourcesink feature. I am not so sure what is the difference between the python example versus the following shell script. Ref: https://wiki.tizen.org/USB/Linux_USB_Layers/Configfs_Composite_Gadget/Usage_eq._to_g_zero.ko

vpelletier commented 2 years ago

I stuck even closer to this example: ecdd288b2a7bb5a9c1736e19eb65100355e3ab82 .

Now both configurations are present, the manufacturer and device ids are the same, the sysfs tree should be almost identical (the difference I spot is that function symlinks are named function.$number in my case, while they are named $functiontype.$number in that example), and the USB descriptors should also be identical.

In the extract below, I am on dummy_hcd so there is no isochronous endpoint.

$ find /sys/kernel/config/usb_gadget/
/sys/kernel/config/usb_gadget/
/sys/kernel/config/usb_gadget/g1
/sys/kernel/config/usb_gadget/g1/os_desc
/sys/kernel/config/usb_gadget/g1/os_desc/qw_sign
/sys/kernel/config/usb_gadget/g1/os_desc/b_vendor_code
/sys/kernel/config/usb_gadget/g1/os_desc/use
/sys/kernel/config/usb_gadget/g1/strings
/sys/kernel/config/usb_gadget/g1/strings/0x409
/sys/kernel/config/usb_gadget/g1/strings/0x409/serialnumber
/sys/kernel/config/usb_gadget/g1/strings/0x409/product
/sys/kernel/config/usb_gadget/g1/strings/0x409/manufacturer
/sys/kernel/config/usb_gadget/g1/configs
/sys/kernel/config/usb_gadget/g1/configs/c.2
/sys/kernel/config/usb_gadget/g1/configs/c.2/function.0
/sys/kernel/config/usb_gadget/g1/configs/c.2/strings
/sys/kernel/config/usb_gadget/g1/configs/c.2/strings/0x409
/sys/kernel/config/usb_gadget/g1/configs/c.2/strings/0x409/configuration
/sys/kernel/config/usb_gadget/g1/configs/c.2/bmAttributes
/sys/kernel/config/usb_gadget/g1/configs/c.2/MaxPower
/sys/kernel/config/usb_gadget/g1/configs/c.1
/sys/kernel/config/usb_gadget/g1/configs/c.1/function.0
/sys/kernel/config/usb_gadget/g1/configs/c.1/strings
/sys/kernel/config/usb_gadget/g1/configs/c.1/strings/0x409
/sys/kernel/config/usb_gadget/g1/configs/c.1/strings/0x409/configuration
/sys/kernel/config/usb_gadget/g1/configs/c.1/bmAttributes
/sys/kernel/config/usb_gadget/g1/configs/c.1/MaxPower
/sys/kernel/config/usb_gadget/g1/functions
/sys/kernel/config/usb_gadget/g1/functions/SourceSink.0
/sys/kernel/config/usb_gadget/g1/functions/SourceSink.0/iso_qlen
/sys/kernel/config/usb_gadget/g1/functions/SourceSink.0/bulk_qlen
/sys/kernel/config/usb_gadget/g1/functions/SourceSink.0/bulk_buflen
/sys/kernel/config/usb_gadget/g1/functions/SourceSink.0/isoc_maxburst
/sys/kernel/config/usb_gadget/g1/functions/SourceSink.0/isoc_mult
/sys/kernel/config/usb_gadget/g1/functions/SourceSink.0/isoc_maxpacket
/sys/kernel/config/usb_gadget/g1/functions/SourceSink.0/isoc_interval
/sys/kernel/config/usb_gadget/g1/functions/SourceSink.0/pattern
/sys/kernel/config/usb_gadget/g1/functions/Loopback.0
/sys/kernel/config/usb_gadget/g1/functions/Loopback.0/bulk_buflen
/sys/kernel/config/usb_gadget/g1/functions/Loopback.0/qlen
/sys/kernel/config/usb_gadget/g1/max_speed
/sys/kernel/config/usb_gadget/g1/UDC
/sys/kernel/config/usb_gadget/g1/bcdUSB
/sys/kernel/config/usb_gadget/g1/bcdDevice
/sys/kernel/config/usb_gadget/g1/idProduct
/sys/kernel/config/usb_gadget/g1/idVendor
/sys/kernel/config/usb_gadget/g1/bMaxPacketSize0
/sys/kernel/config/usb_gadget/g1/bDeviceProtocol
/sys/kernel/config/usb_gadget/g1/bDeviceSubClass
/sys/kernel/config/usb_gadget/g1/bDeviceClass
$ sudo lsusb -vd 2d01:04e8
Bus 005 Device 021: ID 2d01:04e8 my-manufacturer Test gadget
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0 
  bDeviceSubClass         0
  bDeviceProtocol         0
  bMaxPacketSize0        64
  idVendor           0x2d01
  idProduct          0x04e8
  bcdDevice            5.10
  iManufacturer           1 my-manufacturer
  iProduct                2 Test gadget
  iSerial                 3 my-serial-num
  bNumConfigurations      2
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x0020
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          4 Conf 1
    bmAttributes         0x80
      (Bus Powered)
    MaxPower              120mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass      0
      bInterfaceProtocol      0
      iInterface              5 loop input to output
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x0020
    bNumInterfaces          1
    bConfigurationValue     2
    iConfiguration          6 Conf 2
    bmAttributes         0x80
      (Bus Powered)
    MaxPower                2mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass      0
      bInterfaceProtocol      0
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass            0
  bDeviceSubClass         0
  bDeviceProtocol         0
  bMaxPacketSize0        64
  bNumConfigurations      2
can't get debug descriptor: Resource temporarily unavailable
Device Status:     0x0000
  (Bus Powered)
mcuee commented 2 years ago

Good, I did a quick test on the OrangePi Zero which will not have the isoc endpoints. The host is Windows so only the first configuration will be there.

I want to check out the performance better so I use libusbk host cose and not python-libusb1 code. The test results are pretty decent with the total throughput (write + read) about 23MB/s.

(py39venv) C:\libusbK-dev-kit> .\kList.exe

Loading USB ID's maintained by Stephen J. Gowdy <linux.usb.ids@gmail.com>..

 1. Test gadget ((Undefined Vendor)) [Connected]
    Service              : WinUSB
    ClassGUID            : {88BAE032-5A81-49F0-BC3D-A4FF138216D6}
    DeviceID             : USB\VID_2D01&PID_04E8\MY-SERIAL-NUM
    DeviceInterfaceGUID  : {E0956A0A-2D87-ACD9-9ACD-AF4AAB206654}
    SymbolicLink         : \\?\usb#vid_2d01&pid_04e8#my-serial-num#{e0956a0a-2d87-acd9-9acd-af4aab206654}
    DevicePath           : \\?\usb#vid_2d01&pid_04e8#my-serial-num#{e0956a0a-2d87-acd9-9acd-af4aab206654}
    SerialNumber         : MY-SERIAL-NUM
    BusNumber            : 0
    DeviceAddress        : 3

Select device (1-1) :1

Loading driver api..
Getting descriptors..

-Device:
  bLength             :18
  bDescriptorType     :0x01
  bcdUSB              :0x0200
  bDeviceClass        :0x00 (Defined at Interface level)
  bDeviceSubClass     :0x00
  bDeviceProtocol     :0x00
  bMaxPacketSize0     :64
  idVendor            :0x2D01
  idProduct           :0x04E8
  bcdDevice           :0x0510
  iManufacturer       :1
  iProduct            :2
  iSerialNumber       :3
  bNumConfigurations  :2
!End Device
-Configuration:
  bLength             :9
  bDescriptorType     :0x02
  wTotalLength        :32
  bNumInterfaces      :1
  bConfigurationValue :0x01
  iConfiguration      :4
  bmAttributes        :0x80
  MaxPower            :60 (120ma)
 -Interface:
    bLength             :9
    bDescriptorType     :0x04
    bInterfaceNumber    :0x00
    bAlternateSetting   :0x00
    bNumEndpoints       :2
    bInterfaceClass     :0xFF (Vendor Specific Class)
    bInterfaceSubClass  :0x00
    bInterfaceProtocol  :0x00
    iInterface          :5
   -Endpoint:
      bLength             :7
      bDescriptorType     :0x05
      bEndpointAddress    :0x81
      bmAttributes        :0x02 (Bulk)
      wMaxPacketSize      :512
      bInterval           :0x00
   !End Endpoint
   -Endpoint:
      bLength             :7
      bDescriptorType     :0x05
      bEndpointAddress    :0x01
      bmAttributes        :0x02 (Bulk)
      wMaxPacketSize      :512
      bInterval           :0x00
   !End Interface
 !End Configuration

(py39venv) C:\libusbK-dev-kit> .\kBench.exe notestselect list buffersize=4096 buffercount=4 mode=async
device-count=1
1. Test gadget (USB\VID_2D01&PID_04E8\MY-SERIAL-NUM) [WinUSB]
Select device (1-1) :1

opened Test gadget (USB\VID_2D01&PID_04E8\MY-SERIAL-NUM)..
Loop Test Information
        Driver          : WinUSB
        Vid / Pid       : 2D01h / 04E8h
        DevicePath      : \\?\usb#vid_2d01&pid_04e8#my-serial-num#{e0956a0a-2d87-acd9-9acd-af4aab206654}
        Device Speed    : High
        Interface #     : 00h
        Alt Interface # : 00h
        Num Endpoints   : 2
        Priority        : 0
        Read Size       : 4096
        Write Size      : 4096
        Buffer Count    : 4
        Display Refresh : 1000 (ms)
        Transfer Timeout: 5000 (ms)
        Retry Count     : 0
        Verify Data     : Off

Bulk Read (Ep81h) Maximum Packet Size: 512
Bulk Write (Ep01h) Maximum Packet Size: 512

While the test is running:
Press 'Q' to quit
Press 'T' for test details
Press 'I' for status information
Press 'R' to reset averages

Press 'Q' to exit, any other key to begin..
Avg. Bytes/s: 22265856.00 Transfers: 5436 Bytes/s: 22265856.00
Avg. Bytes/s: 22422349.21 Transfers: 11036 Bytes/s: 22576377.95
Avg. Bytes/s: 22576891.25 Transfers: 16624 Bytes/s: 22888448.00
Avg. Bytes/s: 22613333.33 Transfers: 22260 Bytes/s: 22721511.81
Avg. Bytes/s: 22687542.13 Transfers: 27872 Bytes/s: 22986752.00
Avg. Bytes/s: 22770156.44 Transfers: 33616 Bytes/s: 23179728.08
Avg. Bytes/s: 22807813.11 Transfers: 39329 Bytes/s: 23031937.01
Avg. Bytes/s: 22801303.29 Transfers: 44968 Bytes/s: 22756003.94
Avg. Bytes/s: 22849140.09 Transfers: 50730 Bytes/s: 23229480.31
Avg. Bytes/s: 22840770.72 Transfers: 56377 Bytes/s: 22765858.27
Avg. Bytes/s: 22821623.37 Transfers: 61985 Bytes/s: 22630904.43
Avg. Bytes/s: 22851457.32 Transfers: 67645 Bytes/s: 23183360.00
Avg. Bytes/s: 22869774.90 Transfers: 73372 Bytes/s: 23088377.95
Avg. Bytes/s: 22873022.25 Transfers: 79056 Bytes/s: 22915023.62
Avg. Bytes/s: 22901604.28 Transfers: 84746 Bytes/s: 23306240.00
Avg. Bytes/s: 22915514.22 Transfers: 90476 Bytes/s: 23123231.53
Avg. Bytes/s: 22906429.60 Transfers: 96122 Bytes/s: 22761826.77
Avg. Bytes/s: 22905692.47 Transfers: 101795 Bytes/s: 22893209.85
waiting for Ep81h thread..
stopped Ep81h thread.   ExitCode=0
stopped Ep01h thread.   ExitCode=0
Loop Test Information
        Driver          : WinUSB
        Vid / Pid       : 2D01h / 04E8h
        DevicePath      : \\?\usb#vid_2d01&pid_04e8#my-serial-num#{e0956a0a-2d87-acd9-9acd-af4aab206654}
        Device Speed    : High
        Interface #     : 00h
        Alt Interface # : 00h
        Num Endpoints   : 2
        Priority        : 0
        Read Size       : 4096
        Write Size      : 4096
        Buffer Count    : 4
        Display Refresh : 1000 (ms)
        Transfer Timeout: 5000 (ms)
        Retry Count     : 0
        Verify Data     : Off

Bulk Read (Ep81h) Maximum Packet Size: 512
        Total Bytes     : 208478208
        Total Transfers : 50898
        Avg. Bytes/sec  : 11452958.74
        Elapsed Time    : 18.20 seconds

Bulk Write (Ep01h) Maximum Packet Size: 512
        Total Bytes     : 208478208
        Total Transfers : 50898
        Avg. Bytes/sec  : 11452958.74
        Elapsed Time    : 18.20 seconds

Press any key to exit..
mcuee commented 2 years ago

I think with dummy_hcd and python-functionfs, this does open up a lot of possibilities for testing for USB, without dedicated HW. Nice.

Ref: https://www.collabora.com/news-and-blog/blog/2019/06/24/using-dummy-hcd/

mcuee commented 2 years ago

I think we are all good now. You can commit the wip examples to the main branch. So I will close this issue. Thanks a lot for the help.