Implement WebRTC streaming

kripper commented 1 year ago

Is your feature request related to a problem? Please describe.

The h264 streaming only works on some models. See: https://github.com/RosettaDrone/rosettadrone/issues/131

Describe the solution you'd like

Implement webRTC streaming. See: https://github.com/Andreas1331/android-webrtc-dji/issues/1#issuecomment-1635249369

Describe alternatives you've considered

No response

Additional context

No response

ricardohalfeld commented 5 months ago

Can you give us an update on this? I would very much like to get the video stream going with the Mini 2.

kripper commented 3 months ago

No one is working on this. Feel free to implement and share a pull request. Shouldn't be that difficult. Current video streaming code should be deprecated after implementing this issue.

angel-ayala commented 3 months ago

Any idea how to start? I'm new in Android development.

kripper commented 3 months ago

Test the code from https://github.com/Andreas1331/android-webrtc-dji (test receiving in the browser).
Adapt the code to work inside RD, ideally using the plugin interface to keep things clean.

For testing, I would suggest to receive the stream in the same way they do in https://github.com/Andreas1331/android-webrtc-dji. I'm afraid the current video streaming code should be kept in parallel until we confirm the webrtc implemenation supports all the same drone models.

ricardohalfeld commented 3 months ago

I am also not too familiar with Android development but I'll give a go.

angel-ayala commented 3 months ago

I'm still looking at the code and implementing a test case, have you any idea on how implement in the plugin the signaling server and the client, pointed out in the https://github.com/Andreas1331/android-webrtc-dji repo

kripper commented 3 months ago

The README on this repo seems to explains how to setup the servers and clients.

angel-ayala commented 3 months ago

Yes @kripper, indeed, my question was more based on the RD code in terms of classes. I don't exactly understand the plugin hierarchy.

kripper commented 3 months ago

Ok. That's quite easy:

PluginManager.java class: distributes some specific calls to all enabled plugins, ie. the main code invokes the methods of the manager, and the manager invokes the methods on all enabled plugins.
Plugin.java class: must be extended by plugins.
RawVideoStreamer.java class: an example of a plugin used to "stream" raw video frames (in an inefficient and non-standard format). Only usefull as a plugin implementation example, not as a video streaming implementation.

To keep code clean, you should:

create a WebRTCStreaming.java plugin with all your video streaming code.
move all the current buggy video streaming implementation from the main code to a LegacyVideoStreaming.java plugin.

I think for developing and testing, it's ok to start changing the code directly on the main code (without moving the code into plugins), but at the end, before doing a pull request, it would be better to move the new and old video streaming code into plugins to avoid cluttering the base code.

ricardohalfeld commented 3 months ago

@angel-ayala, I haven't been able to find the time to look into this, but if you share your branch I try my best to stay in the loop and bounce ideas with you.

kripper commented 3 months ago

All my changes are on the main branch.

angel-ayala commented 3 months ago

Hi,

I did some test with a simple WebRTC server-client examples and I'm trying to hook up RD into the networking. I implemented a really simple structure here just to check how a plugin should works. So far, so good. Nevertheless, when I add 'io.getstream:stream-webrtc-android:1.1.3' to the build.gradle file, I gets:

2024-08-07 19:05:00.126 15016-15016 libc                    sq.rogue.rosettadrone                A  Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 15016 (ue.rosettadrone), pid 15016 (ue.rosettadrone)

and I don't know why. If I comment that line, the code works again.

The idea now is to implement the SocketConnection class to work with the underlying code of DJIStreamer.

What do you think @kripper?

kripper commented 3 months ago

The error is thrown during the grade build process or during execution? Maybe it requires other dependencies.

angel-ayala commented 3 months ago

Sorry my bad, I was adding a wrong library, here is a updated first draft. I need to test yet, but at least is able to connect to my local signaling server.

RosettaDrone / rosettadrone