Closed dhruvsingla273 closed 1 month ago
Hi @dhruvsingla273 , Could you share what you are trying to do with those frames? Do you require an input of several frames to produce an effect or what is the main goal with this request?
I want to upscale the frames using TFLite model. Refer this - https://github.com/androidx/media/issues/1422
So basically I want to preprocess some of the frames so that I have a buffer of upscaled frames which I can then render and keep processing other frames in background. So like say the player is paused at some point, then also exoplayer would be decoding the future frames to show, Now I want to get these future frames so that I can keep preprocessing the frames. i don't want to process the frames on the fly (like one frame comes, upscale it and then render it, rather I want to already have some upscaled frame ahead of rendering)
I will just pass these frames in a queue to my GL Program for rendering using .setVideoEffect (its a workaround of sorts)
Let me know if its still unclear. Thanks
Hi @droid-girl Any updates?
Thanks
Hi,
If I understand your request correctly, you want to have the following structure:
decoder -> upscaling effect -> queue of pending frames -> surface
Is that correct?
If so, I think the FrameCache effect can help. You could pass a list of effects to ExoPlayer.setVideoEffects() that contains your upscaling effect, followed by a FrameCache effect.
decoder -> upscaling effect -> frame cache effect -> surface
Hi, decoder -> upscaling effect -> frame cache effect -> surface
This is not the structure I want, I am looking for to get the frame in advance to process them (upscaling effect). If I am correctly understanding your method caches my upscaled frames. (let me know if I am wrong)
Rather I want to obtain raw frames earlier say 10 frames before I started rendering, then i can upscale these 10 frames in parallel and make a queue to render them.
With the frame cache, your upscaling effect should get the frames in advance.
If you don't add a frame cache between the upscaling effect and the surface, the upscaling effect will receive the frame right before it is rendered because there is no way to store a frame between the effect and the surface. The frames input to the effect are limited by back-pressure: no frame goes in if we can't handle the output.
If you have a cache of frames between the effect and the surface, then the effect will be able to receive frames earlier, because the output can be stored before it is rendered. Assuming decoding is faster than rendering, your frame cache should be full, and your effect should therefore receive frames in advance.
Does that make sense?
hi, i have some reqirements that how to get the decoded picture frame data from the decoder. in ffmpeg is equals the AVFrame->data[0], data[1], data[2](YUV data), i wanna get these YuvData and then give other program to deal with. can U help me?Thanks @droid-girl
Hi,
This seems unrelated to the current issue. Could you open another issue or clarify why it's related?
Thanks
ok
Hi everyone, I need to access video frames before they are displayed on the screen . Specifically, I want to retrieve future frames from ExoPlayer's buffer during video playback.
What I've Tried:
My Requirement: ExoPlayer must maintains a forward buffer of decoded frames(not sure). I need to access frames from this buffer before they are rendered. I want to process these frames and then render the processed frames using player.setVideoEffect.
Could anyone suggest how to achieve this or point me in the right direction?