search

ThibaultBee / StreamPack

SRT/RTMP/RTMPS live streaming libraries for Android
https://thibaultbee.github.io/StreamPack/index.html
Apache License 2.0
175 stars 67 forks source link

OpenGL Filter support #52

Open themaxsmith opened 1 year ago

themaxsmith commented 1 year ago

Do you plan on exposing GL so you can use OpenGL to apply filters to the streaming video? Most similar libraries on Andriod (https://github.com/pedroSG94/rtmp-rtsp-stream-client-java) and HashinKit on iOS provide this feature or someway to filter/provide a way to modify the broadcast image. I specifically want this feature to be able to add an overlay image via a shader similar to this https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/4b85bce8475deb97380dad3bbe51aca6d24087ea/encoder/src/main/java/com/pedro/encoder/input/gl/render/filters/object/ImageObjectFilterRender.java

As a side note, I think this library has a lot of potentials and think it is heading in the right direction

ThibaultBee commented 1 year ago

Hi,

I am going to be straight: I don't plan to expose OpenGL filters. StreamPack is designed to focus on performance and video quality (like setting encoder profile and level, I plan to use bidirectionnal frame). So I had to made some compromises for this like having a rather close implementation to the way Android camera API (aka camera2) has been designed. This compromises make OpenGL filters hard to implement. The main drawback of this implementation is that camera preview is independant from the video stream, that means that the same filter can't be applied for both easily.

I guess you are looking to have an image overlay. Does it matter if it is not displayed on the camera preview? (if the camera is your source).

Moreover, my TODO list is enoooormouuuusss and my time is really limited!

If you have more questions, I will be happy to talk 👍

Glad to hear that you like this library :)

themaxsmith commented 1 year ago

Yeah, it does not need to be displayed on the camera preview just the SRT TS feed. I have experience with OpenGL and SRT streaming (Mostly on the CPP & iOS side). I can look into it if I have more free time. I might consider spending time on adding it to this library if SRT becomes more mainstream vs RTMP. I think it could be possible via encoderSurface. The problem like always is time and the Android user base / lower revenue per user makes it harder for me to justify the time vs iOS.

ThibaultBee commented 1 year ago

Hi, It would be great if you could have time for this but I totally understand that you have other priorities :) Thanks anyway for the idea.