Hermann-SW / fork-raspiraw

my raspiraw working fork for high framerate stuff (up to 750/1007 fps for v1/v2 camera)
BSD 3-Clause "New" or "Revised" License
90 stars 17 forks source link

Question: High speed motion vector output similiar to raspivid -x #7

Closed baesek closed 2 years ago

baesek commented 2 years ago

Hi, sorry that I ask my question here, in the issues, i did not find any other option. I am actually searching for ways to get the motion_vectors of the pi camera with the smallest possible latency for an interactive art installation.

I already asked here: https://github.com/waveform80/picamera/issues/725

I wonder if I can use this (fork-raspiraw) to modify one sensor modes framerate of either the pi camera v1 or v2 and then use picamera as an option to read the motion_vector data with the modified sensor mode? How does it work? Does the Data stay in the register after modification with raspiraw command or do I have to run the command everytime after booting to get the modified framerate? If I understood right it is also possible to just record every second line of the image to save data, would that help in getting the motion_vector / macroblock data as fast as possible? I just bought a camera v2, is this gonna be better than the v1 version for this, or does it make no difference? I already build a housing for the v1 camera, thats why i am asking.

Thank you for any help, this project made my hope grow that it is possible to have nice realtime motion_detection for the art installation.

best, Paul

vonj commented 2 years ago

I don’t think this is directly related. There may be SOME way to do what you want, but I don’t believe raspiraw has anything at all to do with motion vectors. It just reads bitmaps.

So you can’t use raspiraw I think, for this.

But it’s an interesting idea. Maybe the existing video code (not raspiraw) could be tweaked to increase framerate.

14 sep. 2022 kl. 09:37 skrev Flub @.***>:

 Hi, sorry that I ask my question here, in the issues, i did not find any other option. I am actually searching for ways to get the motion_vectors of the pi camera with the smallest possible latency for an interactive art installation.

I already asked here: waveform80/picamera#725

I wonder if I can use this (fork-raspiraw) to modify one sensor modes framerate of either the pi camera v1 or v2 and then use picamera as an option to read the motion_vector data with the modified sensor mode? How does it work? Does the Data stay in the register after modification with raspiraw command or do I have to run the command everytime after booting to get the modified framerate? If I understood right it is also possible to just record every second line of the image to save data, would that help in getting the motion_vector / macroblock data as fast as possible? I just bought a camera v2, is this gonna be better than the v1 version for this, or does it make no difference? I already build a housing for the v1 camera, thats why i am asking.

Thank you for any help, this project made my hope grow that it is possible to have nice realtime motion_detection for the art installation.

best, Paul

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

baesek commented 2 years ago

Mh. I just read this from 2014 https://www.raspberrypi.com/news/vectors-from-coarse-motion-estimation/ which is implemented here (it uses raspivid with the -x parameter). https://github.com/YggdrasiI/RPIMotionDetection/blob/f07e53ff9777c6ecb81319fd06e8f631c3354b0e/apps/raspicam/start I am still not 100% sure, but i have the feeling that your project could also be somehow used to get the motion vector data at high framerates and thus with low latency. Isn't it?

Hermann-SW commented 2 years ago

I learned from 6by9 that motion vectors can only be retrieved with legacy camera stack at that time. Perhaps libcamera can as well. Definitely raspiraw cannot, it just captures raw frames.