Extremely slow RNDIS throughput with esp-tinyusb (IEC-137)

wuyuanyi135 commented 1 month ago

Answers checklist.

[X] I have read the component documentation ESP-IDF Components and the issue is not addressed there.
[X] I am using target and esp-idf version as defined in component's idf_component.yml
[X] I have searched the issue tracker for a similar issue and not found any related issue.

Which component are you using? If you choose Other, provide details in More Information.

device/esp_tinyusb

ESP-IDF version.

v5.2

Development Kit.

ESP32-S3-WROOM-1-N8R8

Used Component version.

1.4.4 (Dirty)

More Information.

In my project, a USB dongle made of ESP32-S3 are used as communication gateway that creates an RNDIS interface on the PC for connecting with the other devices.

Observations

Current setup

I have been using the tinyusb and additions from esp-iot-bridge and the throughtput was satisfactory (max ~7Mbps). Because of the USB refactoring work, the esp-iot-bridge solution does not compile on esp-idf > 5.2.

Code difference

The iot-bridge's [tinyusb_install_driver](https://github.com/espressif/esp-iot-bridge/blob/master/components/usb/usb_device/additions/src/tinyusb.c) uses tons of hal code but the esp-tinyusb counterpart uses new API that is compatible with esp-idf>5.2.

However, from my observation, with esp_tinyusb, the throughtput was only about 400 Kbps, which is much slower than what I could get from esp-iot-bridge. Looking at the implementation of ECM_RNDIS in tinyusb, it seems there isn't any significant change from the versions.

Benchmark

idf	esp-tinyusb	tinyusb	net	speed (Mbps)	comment
5.2	esp-iot-bridge	0.11.0	RNDIS	~7	baseline
5.2	esp-tinyusb	0.15.0~10	RNDIS	~0.4
5.2	esp-tinyusb	0.15.0~10	NCM	~0.4	unstable, hang after some time. (upstream bug)
5.2	esp-tinyusb	master	RNDIS	~0.4
5.2	esp-tinyusb	master	NCM	~0.4	new driver no longer hang but still very slow
5.3-rc1	esp-tinyusb	0.15.0~10	RNDIS	~0.4
5.3-rc1	esp-tinyusb	0.15.0~10	NCM	~0.4	unstable, hang after some time. (upstream bug)
5.3-rc1	esp-tinyusb	master	RNDIS	~0.4
5.3-rc1	esp-tinyusb	master	NCM	~0.4	new driver no longer hang but still very slow

Note:

to make RNDIS work, the fix has to be applied.
upstream NCM driver is being rewritten

Conclusions

From my analysis, I found that

Using the legacy NCM driver in current esp-tinyusb will lead to very chopy network (ping vary from <1ms to seconds or timeout) and will no longer respond after heavy traffic.
The legacy RNDIS driver in esp-tinyusb does not show unstable traffic but the throughtput is very slow compared to the one in esp-iot-solution
Different upstream versions of tinyusb does not affect RNDIS. In fact, the codebase of RNDIS did not change significantly.
Different idf.py version does not affect the performance, but newer version of idf prevents the esp-iot-solution to compile, thus no comparison can be made

I guess the slowdown was not caused by tinyusb. Most probably the way how tinyusb driver are installed (see Code difference) caused the issue. But these code involves too much implementation details of USB peripheral so I am not able to further track down.

Can you please compare the network throughtput when using this project and see if the speed confirms my observations?

leeebo commented 1 month ago

@wuyuanyi135 The main difference between tinyusb 0.11.0 and 0.15.0~* is the latest one switch the dcd layer from espressif/esp32sx/dcd_esp32sx.c to synopsys/dwc2/dcd_dwc2.c

Would you please try again with esp-tinyusb 0.15.0~10 with this line https://github.com/espressif/tinyusb/blob/release/v0.15/CMakeLists.txt#L49 change to src/portable/espressif/esp32sx/dcd_esp32sx.c

wuyuanyi135 commented 1 month ago

@leeebo Hello, Thanks for your tips! I have tried your suggestion by commenting out "src/portable/synopsys/dwc2/dcd_dwc2.c" and add "src/portable/espressif/esp32sx/dcd_esp32sx.c" in CMakeLists.txt. It does not help improving the speed -- same performance @ ~0.4 Mbps

This is way slower than the implementation in esp-iot-bridge

lijunru-hub commented 3 weeks ago

I discovered that the new version of esp_tinyusb has low processing efficiency when handling tud_network_xmit in RNDIS. There's no effective notification mechanism to quickly send out data from the user layer. After replacing tinyusb_net.c in esp_tinyusb with the old version, the data transmission rate returned to normal.

espressif / esp-usb