search

indrekluuk / LiveOV7670

A step-by-step guide to building the circuit for this project:
https://circuitjournal.com/arduino-ov7670-10fps
239 stars 91 forks source link

Synchronizing two cameras #11

Open kozuch opened 6 years ago

kozuch commented 6 years ago

Do you think it is possible to exactly "frame synchronize" two OV7670 sensors when used with Arduino (or compatible) board? Say there will be enough pins to connect the cameras (I may use only 4 bits per sensor). The idea is to feed a common XCLK input signal into both cameras.

However, there may be another issue in synchronizing the I2C communication to configure the cameras at exactly the same time - I have no idea how this could be done. The best may be of course 2 parallel and independent I2C interfaces on Arduino but even with that the CPU would need to be able to issue an I2C command on both buses simultaneously (within a single CPU clock cycle?).

indrekluuk commented 6 years ago

Probably the easiest way to sync the cameras is to do it after setup. Run one camera with XCLK until you get VSYNC. Then stop XCLK and run the other camera until you get its VSYNC and then start XCLK for both cameras.

Why do you need them to be synced?

kozuch commented 6 years ago

I need the sync because I do computer vision with a stereo camera in real time with a moving scene. The resolution can be quite low though.

The post setup sync is great idea. I wonder whether it can be really as simple as "stopping" the sensor (stopping XCLK) and starting it again without any negative side effects? The pause will probably be very minimal (up to one frame) so that it should probably not even affect the AEB etc. - in case it does, I can wait few frames to get the AEB right again after sync probably.

In the meantime I found that two sensors could probably be cofigured via the SCCB simutaneously (see this EE) given there are no ACKs. That is a subject to further research though.

Anyways, can you tell how to connect the two I2Cs even for the "after setup" case? Are the I2C addresses of the sensors hardcoded in the HW? Or can they be changed?

indrekluuk commented 6 years ago

I don't know if the address can be changed. Currently have hard coded 0x21 as the address. My current code also reads information from the camera if I want to set only one register bit. I read out a byte change one bit and then write the byte back. I guess it isn't actually really necessary, since first I write the entire default configuration to the camera. So you could just change the default config and then you don't have to read back anything.

kozuch commented 6 years ago

Well I have little experience with I2C but I am not so far yet.

Do you think a dual camera solution using the STM32 DCMI interface (like mentioned in #10) could also be synchronized by stopping one sensor and waiting for both VSYNCs to be in sync like you suggested? Will the DCMI play some role here? The camera's VSYNC is connected to dedicated DCMI interface input but the idea is that I could also simultaneously connect both VSYNC from cameras to general GPIO pins where I would watch the synchronization timing.

indrekluuk commented 6 years ago

Yes I believe you can do it like that.

kozuch commented 6 years ago

Thinking about the synchronization once again. My goal is to build frame synchronized stereo camera that will stay in sync for long time - I need to stream video from it where in each video frame both cameras expose the frame at exactly the same time (similar to the Stereolabs ZED camera). However I am afraid of one problem - different exposure time of both cameras.

My original idea with DCMI was to connect both cameras to the single DCMI interface - the prerequisite for this to work is that both sensors will behave exactly the same. And I think they wont unless the exposure time of the first sensor will be shared with the second sensor. From what I know about rolling shutter the sensor is exposed per line - with (even slightly) different exposure time, the same pixels will not read out at the same time from the sensor, am I right? This could probably be taken care of when not using DCMI but plain Arduino without DCMI - one would just watch PCLK from both cameras separately (like you do in this project). But without the external image sensor trigger the frame exposure moments would eventually drift away from sync (if the sensor can not be forced to a fixed FPS). I have a bunch of USB cameras and they lower the FPS with low light (the capture maybe 10 FPS only in the evening and 30 FPS during daylight).

Your idea with bringing the sensors to sync (syncing VSYNCs) will definitely work but I am afraid it may only for the first frame. I think (do not know for sure though) the ZED camera (and others) actually uses a trigger feature of the image sensors and they trigger both sensors at the same time at given FPS. The trigger (sometimes called RESET) is usually available only on the higher-end image sensors even though it probably works very simple just by blanking all sensor (=starting exposure).

I will highly appreciate your comment on this matter. Thanks!

indrekluuk commented 6 years ago

I really don't know much about the exposure times. I think you just have to experiment.

I have also noticed that OV7670 can slow down with low light and/or really rapid camera movement. But you can do the frame synchronization on every frame. If the cameras are already in sync then the wait time for other camera should be minimal.

Maggie275 commented 4 years ago

Hi, I am currently working on a project that uses stereo camera. I want to record the camera output to a file. I am able to record the output of each sensor individually using gstreamer to an mpeg4 file. Can you please suggest how to synchronize between the two sensors and record them simultaneously? Also, what are some uncompressed formats that I can use for recording videos for image processing?