search

itberrios / CV_projects

General Computer Vision Projects
MIT License
40 stars 14 forks source link

Using rolling shutter in phase based video processing #3

Open alienhd opened 3 months ago

alienhd commented 3 months ago

Hi itberrios, I heard that for some applications like motion amplification or passive sound recovery using vibrations from sound waves the user has to have a high speed camera in order to be able to process the data. I also read about using the rolling shutter of CMOS cameras to leverage the effective sampling rate for such use cases. Could you help me to understand how to use or implement this effect?

For reference I found this article from the university of Ottawa talking about this topic: https://ruor.uottawa.ca/items/0d6100d4-fec2-4539-b1b6-fbf24a28c62c

itberrios commented 1 month ago

Hello, this sounds interesting. I might be able to help, do you have any specific questions? Or issues that I could look into?

alienhd commented 1 month ago

Well I just don't get the implementation to work, I have tried a lot of different github repos and even tried coding it by myself.

The project itself for the whole visual microphone project requires upwards of 2000 fps of footage because it translates to a resulting 2000 Hz of sound frequenzy. The CMOS sensor should be able to make normal footage usable because of the line delay.

itberrios commented 1 month ago

ok, I see. Can you send over any of the links to the repos? I can't seem to find anything useful on this.

alienhd commented 1 month ago

This one for example: https://github.com/jia-kai/hearv/tree/master