pass information through to a post-processing stage - how?

[petercwallis]

The method LibCameraApp.Wait() waits for the next frame, but I want to modify what my post-processing-stage does with the frame. How should I pass data re the next frame to my post-processing-stage?

The obvious way to do that would be to have Wait() take an optional argument that looks like metadata, but perhaps I can adjust existing metdata? Perhaps AdjustConfig can be called before ... but that doesn't look as if it is meant for that. Perhaps I can get a reference to my PostProcessingStage that was created from the specification in the json file - then I could call my own public method there before doing the Wait()?

Before someone says I don't want to do that, I want to adjust the ROI for my fiducial finder based on a compass reading.

[naushir]

We designed the postprocessing API to be as simple as possible based on the example use cases we had implemented. As such, there is no way currently to pass data into the stages as you need here.

However, you should be able to achieve what you need if you take your compass readings (I assume they are electronically read?) in your postprocessing stage Process() function itself and adjusting your ROI directly from there?

If that is not possible, we need to consider an API update to allow access to the post processing stages from the application and a new method to pass arbitrary data into it.

[petercwallis]

If there was some way to access the CompletedRequestPtr before calling LibcameraApp.Wait(), I could add things to the metadata and pull it out in the post-processing stage...

[naushir]

Unfortunately there is no callback mechanism to do this. Could you not embed the compass reading within your stage directly?

[petercwallis]

I could, but then other things that access the compass will need to access the post-processing-stage buried in libcamera-app. It would be tidier to not use the stage mechanism. It would not be a callback would it? The 'app' object is sitting there - the call to wait is app.Wait() if I recall - a method like app.getMetaData() could return a pointer to the static (presumably) "response" metadata object. p

On Tue, 11 Jul 2023 at 12:45, naushir @.***> wrote:

Unfortunately there is no callback mechanism to do this. Could you not embed the compass reading within your stage directly?

— Reply to this email directly, view it on GitHub https://github.com/raspberrypi/libcamera-apps/issues/532#issuecomment-1630674617, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALY3LWJJALB7ETACGXD7TGTXPU4G5ANCNFSM6AAAAAA2CYM7DY . You are receiving this because you authored the thread.Message ID: @.***>

[naushir]

I could, but then other things that access the compass will need to access the post-processing-stage buried in libcamera-app. It would be tidier to not use the stage mechanism.

The mechanism I would suggest to account for this is to create a "compass" postprocessing stage that runs first. This reads the compass data, writes it to a metadata tag. Any other postprocessing stage downstream that wants compass data would read it from the appropriate metadata tag.

It would not be a callback would it? The 'app' object is sitting there - the call to wait is app.Wait() if I recall - a method like app.getMetaData() could return a pointer to the static (presumably) "response" metadata object.

Unfortunately it's a bit more complicated I think. The framebuffer flow is hidden within private members of the object. To hook into the correct place, i.e. after libcamera has returned the buffer but before the postprocessing stages have run would require a callback to ensure correct synchronisation. This is because the metadata is only available after libcamera has signalled a request has completed.

[petercwallis]

Hm. yes, I was expecting it to be possible to grab a reference to the metadata object from libcamera before it returns its next buffer - possible (presumably) if the metadata structure is re-used by libcamera.. and not cleared ofcourse..

On Tue, 11 Jul 2023 at 16:35, naushir @.***> wrote:

I could, but then other things that access the compass will need to access the post-processing-stage buried in libcamera-app. It would be tidier to not use the stage mechanism.

The mechanism I would suggest to account for this is to create a "compass" postprocessing stage that runs first. This reads the compass data, writes it to a metadata tag. Any other postprocessing stage downstream that wants compass data would read it from the appropriate metadata tag.

It would not be a callback would it? The 'app' object is sitting there - the call to wait is app.Wait() if I recall - a method like app.getMetaData() could return a pointer to the static (presumably) "response" metadata object.

Unfortunately it's a bit more complicated I think. The framebuffer flow is hidden within private members of the object. To hook into the correct place, i.e. after libcamera has returned the buffer but before the postprocessing stages have run would require a callback to ensure correct synchronisation. This is because the metadata is only available after libcamera has signalled a request has completed.

— Reply to this email directly, view it on GitHub https://github.com/raspberrypi/libcamera-apps/issues/532#issuecomment-1631050054, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALY3LWPKA53CUT6T3WTJA2TXPVXDTANCNFSM6AAAAAA2CYM7DY . You are receiving this because you authored the thread.Message ID: @.***>

[naushir]

If there is nothing more to discuss here, I'll close this down shortly.

[petercwallis]

That is fine. My use-case has been noted for future releases no doubt 🙂

On Tue, 18 Jul 2023, 09:20 naushir, @.***> wrote:

If there is nothing more to discuss here, I'll close this down shortly.

— Reply to this email directly, view it on GitHub https://github.com/raspberrypi/libcamera-apps/issues/532#issuecomment-1639743024, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALY3LWLJ7BHUU2BUQODFL2DXQZBK7ANCNFSM6AAAAAA2CYM7DY . You are receiving this because you authored the thread.Message ID: @.***>