UVC driver support

[ValkA]

It would be interesting to see how MODE_SYNCHRONOUS_FRONT_BUFFER_RENDERING will affect UVC device rendering. Currently, ROTG02 with GO FPV app have ~120ms latency. I've looked at GO FPV reversed code and it uses SurfaceTexture, with bitmaps. How complicated will be to feed your views with frames from a UVC device using this library https://github.com/saki4510t/UVCCamera instead of the frames coming from UDP ?

p.s, Can you recall how much latency FRONT_BUFFER_RENDERING reduced?

[Consti10]

Hello, I have reached out to the developer of the 'Go FPV' app, hopefully he can make his project open source. I tested the UVCCamera library but it did not work with the ROTG02. I do not know how the rendering pipeline of GO FPV works, but if I can shave off any delay I'l try my best.

[ValkA]

Cool 😃

[Consti10]

Unfortunately the dev of the GO FPV app hasn't repsponed yet. However, I did some research and after searching github for quite a while I found a fork of UVCCamer that compiled: repo

And the example nr 8 showed an image with my rotg02 receiver ( unfortunately it crashes quite a lot) The main takeaways from this repo are: a) It is in turn based on another repo - lubusb and libuvc. Both of them seem to be maintained and of good quality ( the UVCCamera example is not bad but a bit bloated for FPV ) b) It uses SW to transform the UVC camera frames (YUYV - and I don't think any FPV usb receiver provides MJPEG) to RGBA then copies that data into a ANativeWindow

[Consti10]

I have to correct myself - the example code configures the ROTG02 with MJPEG as frame format (if there is even anything to configure - I think it is more that the ROTG02 supplies MJPEG frames only

The encoding is probably where most latency comes from. Decoding & copying frames into ANativeWindow takes ~16ms on my ZTE Axon7. That could be maybe optimized to half (8ms) since it looks like some unncesseary copying it happening but a majority of the time is probably created by the encoder of the ROTG02.

The current side - by side mode of comparable apps like Go FPV does not do any undistortion so I assume Front buffer rendering In combination with distortion correction) can only match the latency of the curent go fpv app.

[ValkA]

It worth checking the datasheet of V399 chip which seem to be responsibe for USB, maybe its possible to change its configuration which will reduce latency.

AMT630A is responsible for converting analog video signal from the RX modules.

If a phone + 3d printed VR case + ROTG02 variation can replace a fatshark it will be a game changer.

[ValkA]

This looks like V399 datasheet.

[Consti10]

Thanks for the datasheets.

I have good news: I managed to get the ROTG02 to work with FPV_VR. I worked my way down from the UVCCamera project, stripping unneeded functionalities and trying to understand the magic behind saki4510t 's modified libuvc / libusb libraries (as it turned out, they don't have official support for android unfortunately) But while some stuff is still 'magic' both why it works and why it compiles :) I succesfully tested my uvcintegration on android 7 and 9. I was not yet able to do some latency testing since I am missing normal FPV goggles but it is definitely usable. What can be improved in the future: The project uses a really old release of libjpeg-turbo with SIMD disabled I think. At the time decoding takes ~20ms for a single 640x480 frame on latest smartphone HW. Feels like a bit too much for me, a proper install of the latest libjpeg-turbo release might reduce latency.

[ValkA]

Cool! Here's one way to measure end to end latency in case you'll get there

[Consti10]

closing since uvc is working fine now