search

fpv-wtf / voc-poc

A PoC for USB video out from the DJI FPV Goggles, inspired by The Video Out Club
MIT License
432 stars 80 forks source link

Streaming a video to DJI Goggles using USB #15

Closed usumfabricae closed 1 year ago

usumfabricae commented 2 years ago

Hi, I see that DJI Virtual Flight App is capable of showing a virtual environment within the DJI FPV Goggles, I think there should be another magic command that could enable streaming data to the goggles. Does anyone have tried to implement something similar?

mbundy commented 2 years ago

I've wondered the same. But it's also IOS only so who knows how it works...

j005u commented 2 years ago

Sorry for not replying earlier.

Short version: we've looked into it and it can't be done trivially due to Apple's MFI bullshit and iOS being the only supported platform.

kevincerro commented 1 year ago

Sorry for not replying earlier.

Short version: we've looked into it and it can't be done trivially due to Apple's MFI bullshit and iOS being the only supported platform.

Now DJI Virtual Flight is available on Android https://www.dji.com/es/downloads/products/avata

Maybe someone can decompile APK and check how it works

j005u commented 1 year ago

I already did some digging, and the short version is that it appears to use Android Open Accessory protocol, which isn't possible to emulate on a PC due to USB host vs. gadget roles.

Might be different on the new Goggles 2, but on V2 we will still need a custom solution.

mfkenson commented 1 year ago

It is still cool with Android phone as a relay/gateway.

j005u commented 1 year ago

Unless your Android device has two dedicated USB ports you'd be forced to stream over Wifi to the phone and the latency would absolutely suck.

MiniGod commented 1 year ago

Unless your Android device has two dedicated USB ports you'd be forced to stream over Wifi to the phone and the latency would absolutely suck.

Many android phones and tablets support USB-C dongles / hubs, with multiple USB ports, ethernet and even HDMI, so that's not a stopper.

mfkenson commented 1 year ago

Unless your Android device has two dedicated USB ports you'd be forced to stream over Wifi to the phone and the latency would absolutely suck.

Many android phones and tablets support USB-C dongles / hubs, with multiple USB ports, ethernet and even HDMI, so that's not a stopper.

I guess using usb hub does not work as they are not native dedicated USB ports.

However there are SBC boards satistying these requirements. https://wiki.radxa.com/Rock5 i got two units in hand. Its a android SBC (similar to Raspberry Pi but runs Android with play store).

j005u commented 1 year ago

Yes. The latest Raspis can also act as a gadget on the USB-C port with a firmware upgrade. Not very user friendly, but that'd work.

You'll need to write software to pretend to be the DJI Simulator app over the Android Open Accessory (AOA) protocol. The exact details aren't known, but from reverse engineering some of the Goggles side stuff what I can say is that there's likely a high bandwidth tunnel for DJI's proprietary DUML protocol delivering h264 frames which get fed to the RTOS. There's also a handful of setup procedures.

I would suggest a USB sniffer if you have access to one. Or the alternative is to forget the simulator protocol and use root access to implement an alternative.

i3v commented 1 year ago

Just for the reference: here's a tutorial that demonstrate a working video streaming from a Windows desktop to the Dji FPV Goggles V2.

I was even able to stream some desktop video myself, by just following this tutorial. The only glitch is that I had to run regsvr32 "C:\full\path\to\OCX\Button.ocx" and regsvr32 "C:\full\path\to\OCX\Frame.ocx" instead of running the RegOCX.bat. Also the streaming always failed after 1-30 seconds during this my first experiment.

Thanks, @j005u !

j005u commented 1 year ago

This is based on Knifa's work from the team. A proper public release should be on it's way soon-ish. The technical tl;dr is that instead of attempting to use the simulator video in pipeline, Knifa implemented his own streaming method based on root and moonlight.

The GUI you downloaded off of Youtube is cool, but it's made by a 3rd party and is closed source, so we can't guarantee it's safety.

Knifa's alpha is available on our Discord along with instructions, but I encourage you to hang tight as a release to the package manager is close.

j005u commented 1 year ago

See here for the release announcement: https://discord.com/channels/849931046883819550/977165066778587217/1094670102449897613

Or just head over to the repo and follow the README: https://github.com/fpv-wtf/dji-moonlight-shim

pierluigizagaria commented 4 months ago

Anyone knows if DJI Goggles 2 uses AOA like v2 for DJI Virtual Flight video input?