search

pedroSG94 / RTSP-Server

Plugin of rtmp-rtsp-stream-client-java to stream directly to RTSP player.
Apache License 2.0
214 stars 66 forks source link

Take a frame in realtime, process, shift the outgoing frame by XY. #83

Closed envery closed 9 months ago

envery commented 1 year ago

Greetings. Please tell me what needs to be done in order to take a frame in realtime, process it with opencv, and shift the outgoing frame by XY coordinates to stabilize the camera shake. Best Regards !!!

pedroSG94 commented 1 year ago

Hello, OpenCV is not supported in this library. The only way to modify the output frames is using openGL with a filter: https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/wiki/Real-time-filters

You can use getGlInterface().takePhoto method to get frames as bitmap, convert it to argb byte array and proccess it with openCV. With the data you can use the filter to modify the image but I think that the first step will be really slow so I'm not sure it is a good idea.

Few devices support camera stabilization using camera2 api that maybe is the best way to handle with this problem. It is already implemented in the library using this methods:

rtmpCamera2.enableVideoStabilization()
rtmpCamera2.enableOpticalVideoStabilization()