search

DragonX-cloud / dji_flutter_plugin

A Flutter plugin for DJI SDK.
BSD 3-Clause "New" or "Revised" License
34 stars 8 forks source link

Update on Video/Image Streaming/Downloading #4

Closed iyush1993 closed 2 years ago

iyush1993 commented 2 years ago

Any update on video/image streaming/downloading feature?

orenagiv commented 2 years ago

Hi @iyush1993

Last week I've updated the version with the ability to get the media list, download a high res media file and delete a file.

The development has been competed for iOS, but soon will be applied on Android too.

The next feature is to support streaming 💪 So I'll keep you posted 🙂

iyush1993 commented 2 years ago

Amazing. I appreciate it.

orenagiv commented 2 years ago

Quick update: I'm very close to finishing the video streaming.

I hope that within 1-2 weeks I'll be able to publish the new version.

FYI @iyush1993

mamphis commented 2 years ago

Hi @orenagiv do you have any updates on the video streaming?

I would love to include your flutter_plugin, and the video streaming would be a really nice to have!

I'm looking forward to implement your plugin into an app!

Great work!

orenagiv commented 2 years ago

Hey @mamphis I'm basically done with the bytes-stream from the DJI-SDK native side and into Flutter. And I'm able to transform them into an HLS format and view it on the Flutter side (using VLC Player).

All the above works already for me on iOS and I'll submit an update soon.

However, there's a serious latency issue. For some reason the VLC Player "caches" 5 seconds, and no matter what I've tried - it refuses to remove this buffer. This is why it takes so long...

If I won't be able to resolve the latency issue within 1-2 days - I'll submit a version with what I have so far anyway.

orenagiv commented 2 years ago

Hey,

Just a quick update FYI:

I've reached a stable state where the byte-stream gets converted to HLS, and played in real time by VLC Player, but unfortunately with a delay of 3-4 seconds.

It seems to be an issue with the VLC package itself, which causes the delay. There is an open issue here: https://github.com/solid-software/flutter_vlc_player/issues/327#issuecomment-1146840661

So after trying many other video player packages and hundreds of FFMPEG variations, I ended up with the 3-4 seconds delay, which is the best result I could achieve so far.

The DJI plugin serves the byte-stream in real-time, from the native platform using the DJI native SDK. The DJI SDK produces an H264 Raw byte-stream, and that's what I'm passing through into the Flutter side by the plugin. The Flutter App which will use the Flutter DJI plugin can do whatever it needs with it. The example and documentation will include the above variation which uses FFMPEG and VLC to preview the byte-stream on screen using either HLS format (for minimal latency).

I'm currently working on the Android side and once done I'll wrap it up, document, and publish the version update.

Will keep you posted.

orenagiv commented 2 years ago

Quick update: I've hit a snag - Flutter limitation on Android which causes a plugin conflict (because my example uses FFMPEG on the Flutter side, and it conflicts with the platform-channels of the DJI plugin). It works nicely on iOS, but fails on Android.

Related issues: https://github.com/tanersener/ffmpeg-kit/issues/367

I'm working on a solution.

orenagiv commented 2 years ago

Hey guys, Quick update: I'm very-very close a resolution here.

Will update soon.

orenagiv commented 2 years ago

Hey guys,

Yesterday I've released version 1.0.0-dev.18 which includes the Video Feed feature.

A YUV420p video byte-stream flows through the sendVideo() method, once the startVideoFeed() is triggered.

In the plugin example I'm using the ffmpeg-kit Flutter plugin to convert the raw-video YUV420p to HLS, and play it on screen using the Flutter VLC Player plugin.

The above works nicely on both iOS and Android. However, I couldn't overcome around 4-6 seconds latency. The latency seems to be a result of the VLC Player. But that's the only Player I could find that managed to play the output file while it's still being written (by using HLS format which breaks the output video into segments for smooth streaming).

I'm going to continue searching for ways to reduce the latency in the example, but the plugin itself is working as expected and produces the byte-stream in real time.

The same method (with the FFMPEG in the example) can be used to convert the video feed byte-stream to an .mp4 and save as a file or upload it as a stream to the cloud.

[!] Important Note When using the FFMPEG-KIT plugin on Android, you might encounter an "unhandled exception" error. If that happens, you will need to manually edit the GeneratedPluginRegistrant.java in the Android folder of your Flutter project, and move the ffmpeg-kit plugin to be the first plugin in that file. I've described the issue here: https://github.com/tanersener/ffmpeg-kit/issues/461 And the workaround here: https://github.com/tanersener/ffmpeg-kit/issues/461#issuecomment-1159565970

- May the force be with us 😉