External trigger timing not aligned with exposure start

[echoGee]

---

Required Info
Camera Model	{ D435i }
Firmware Version	5.12.15.50
Operating System & Version	{Win (8.1/10)
Platform	PC
SDK Version	realsense viewer

external trigger timing not aligned with frame start

1. Im referring https://dev.intelrealsense.com/docs/external-synchronization-of-intel-realsense-depth-cameras to sync an external light with the d435 camera running as a master. The trigger out(~50uS pulse) of the camera does not seem to be aligned with the start of the exposure, but the end of it as shown below(orangle line). Is it an error in the above link ?

2. We need to control the exposure time of the external light to align with the camera's exposure start-end duration. Is there a way to use any of the signals shown below to do this ?

[MartyG-RealSense]

Hi @echoGee Thanks very much for your questions.

1. The trigger diagram is correct. The section of the white-paper containing this diagram states that there is a delay of up to 1 frame between trigger and read-out when using Genlock mode (an Inter Cam Sync Mode value of 4+). However, the diagram and the accompanying text are describing triggering with a pulse of > 100us from an external signal generator device (such as an oscilloscope) instead of the 50us pulse from a RealSense camera set to Master.

It sounds though as you are not using a second 'slave' camera but are instead using the 50 us trigger pulse from the master camera to control an external light. Is that correct, please?

2. There is a project described at https://github.com/IntelRealSense/librealsense/issues/4574 for controlling an external RGB flash with hardware sync by setting Inter Cam Sync Mode to '1' (Master) and enabling the camera's trigger pulse, and using a voltage shifter (also known as a level shifter) to increase the 1.8v to a level compatible with an LED controller device. This may provide useful insights for your light-controlling project.

[echoGee]

Hi @echoGee Thanks very much for your questions.

1. The trigger diagram is correct. The section of the white-paper containing this diagram states that there is a delay of up to 1 frame between trigger and read-out when using Genlock mode (an Inter Cam Sync Mode value of 4+). However, the diagram and the accompanying text are describing triggering with a pulse of > 100us from an external signal generator device (such as an oscilloscope) instead of the 50us pulse from a RealSense camera set to Master.

It sounds though as you are not using a second 'slave' camera but are instead using the 50 us trigger pulse from the master camera to control an external light. Is that correct, please?

correct. To be precise, using the 50us trigger pulse to control an external light to illuminate the stereo cameras, not necessarily the rgb one. It seems like https://github.com/IntelRealSense/librealsense/issues/2474#issuecomment-430062038 have answered my question about timing

1.The trigger was generated when the image frame was read out from the image sensor. in another word, the end of the exposure.
2. When triggering externally, the trigger pulse aligns with the frame start

Regarding #2

2. We need to control the exposure time of the external light to align with the camera's exposure start and end duration. Is there a way to use any of the signals shown below to do this ? Could you provide documentation on the detailed functions of each of these pins in this image?

[MartyG-RealSense]

A simple design for an electronic circuit that responds to the trigger timing may be to connect a wire to Pin 5 (sync) and the other end to the external light, perhaps with a level shifter in the middle of the wire to increase the voltage enough to make the external light illuminate. When the trigger pulse was up the external light would illuminate and when the trigger was down the light would go out as the voltage falls to zero.

This is similar to the simple lightbulb circuit that kids who are learning electronics are taught, like the tutorial at the link below.

https://learning-center.homesciencetools.com/article/circuit-science-projects-for-elementary/

[echoGee]

A simple design for an electronic circuit that responds to the trigger timing may be to connect a wire to Pin 5 (sync) and the other end to the external light, perhaps with a level shifter in the middle of the wire to increase the voltage enough to make the external light illuminate. When the trigger pulse was up the external light would illuminate and when the trigger was down the light would go out as the voltage falls to zero.

The problem is that the trigger is turned high only when the frame has finished, like shown below . The trigger pulse doesn't show the exposure duration too.

[MartyG-RealSense]

If there will always be a delay of up to 1 frame between trigger and read-out when using Genlock mode then that will make it difficult to have the external light turn on-off exactly in line with exposure if the light will turn off when the trigger falls just before exposure starts.

You may be able to artificially add a delay in transmission of the voltage pulse to the external light by adding a transistor + capacitor to the wire in-between Pin 5 and the light, with the choice of capacitor determining the duration of the time delay. Such a circuit is described at the link below. The question is whether it is possible to generate a time delay small enough to be equivalent to 1 frame.

https://www.homemade-circuits.com/simple-delay-timer-circuits-explained/

[echoGee]

@MartyG-RealSense Could you provide a detailed specs on the individual pin outs?

[MartyG-RealSense]

The extent of the available public information about these pins is the table shown above.

[MartyG-RealSense]

Hi @echoGee Bearing in mind the above comment, do you require further assistance with this case, please? Thanks!

[echoGee]

Is there a particular intel support we could ask for these details ?

[MartyG-RealSense]

I will consult with my Intel RealSense colleagues about your pin information request.

[MartyG-RealSense]

Thanks very much @echoGee for your patience. My Intel RealSense colleagues have advised about your problem that if you want to start the trigger at the start of exposure then you can grab the LASER_PWM0 from the 9-pin connector (pin 6). Please see the screenshots below. However, you need to make the pulse burst to a single pulse to cover the entire exposure time. A one-shot timer can do this job.

[echoGee]

Thank you @MartyG-RealSense !

[MartyG-RealSense]

You are very welcome, @echoGee - I'm pleased that we could help. Thanks very much for the update!

[bart-adi]

Hi @MartyG-RealSense will this also work with the d415 ?

[MartyG-RealSense]

Hi @bart-adi Hardware sync for RealSense 400 Series cameras can be used with the D415 model, yes.

[bart-adi]

Hi @MartyG-RealSense , thanks for the quick response. I have been trying in vain to get this to work with 415. Is it possible that due to a different sensor type that the laser emitter PWM is always on , and not linked directly to the exposure time? This is what my experiment are indicating ... In contrast, i have been able to get this to work for 455.

[MartyG-RealSense]

You are correct. The projector on the D415 model is always on, whilst the projector in D435 / D435i and D455 pulses in line with exposure.

[bart-adi]

that is sad news! But thanks for passing it on. Is this information that can be found in the datasheets ?

[MartyG-RealSense]

It is not referenced in the data sheet but is referred to in the opening paragraph of section 9 of Intel's white-paper guide for the 400 Series cameras about projectors.

https://dev.intelrealsense.com/docs/projectors?_ga=2.155553791.534044615.1670336853-1041004708.1589641371#section-9-using-multiple-depth-cameras

"By default the D435 actually pulses its light to coincide with camera exposure. During these tests we turned that feature off to ensure that all projectors stay on all the time, as it is by default for the D415".

In the Stereo Module > Controls section of the options side-panel of the RealSense Viewer, the option to alternate the projector on-off is absent for the D415 camera model. D435 / D435i and D455 have options in this section to alternate the projector on-off or set it to Always On to replicate the behaviour of the D415 projector.

[echoGee]

I'm reopening this issue. but it is specifically for the HDR setting on. See the scope picture below where the HDR is set with two different exposure values:

1. 500us
2. 3000us

The blue signals are the PWM output from realsense and the purple one is the sync line(its really short blip around t-70ms). The issue is

1. the pwm is on for around 30ms
2. Both the frame are the same exposure. It should really be ~500us and ~3000us alternatively

[MartyG-RealSense]

Hi @echoGee You can check in the RealSense Viewer's metadata overlay (in the 'Actual Exposure' value of that overlay) if the exposure values are actually alternating after HDR has been enabled, even if the alternating does not appear to be represented on the scope screen. Instructions for doing so can be found in the section of the HDR white-paper guide linked to below.

https://dev.intelrealsense.com/docs/high-dynamic-range-with-stereoscopic-depth-cameras#31-hdr-with-intel-realsense-viewer

[echoGee]

Yes. It is alternating

[MartyG-RealSense]

If HDR is enabled and the exposure is alternating in the Viewer's metadata overlay then it likely is actually alternating but the scope cannot render the alternating for some reason. In that case, I would recommend trusting the Viewer's assertion that the exposure is actually alternating. HDR is not usually used with hardware sync, so there are not past precedents to refer to.

[echoGee]

I can see that the alternating frames have the expected exposure. The issue is that the sync and pwm output is not reflecting the exposures corresponding to the frames

[MartyG-RealSense]

I admittedly do not have sufficient knowledge of scopes to advise about your scope's output. There is a very detailed illustrated discussion about scopes and PWM readings though at https://github.com/IntelRealSense/librealsense/issues/8470 where a RealSense team member offers advice. That case may provide some useful insights.

[echoGee]

I believe we are electrically scoping the right pins and seeing the expected output in the pwm and sync pins during regular (non-hdr) settings. However, its doing unexpectedly during the HDR mode. Are you able to ask the firmware or electrical experts at realsense to verify the issue and how to solve this ?

[echoGee]

This issue is really blocking us. Could you tell us how to connect with the right support to solve this ? You could DM me

[MartyG-RealSense]

As I previously consulted with my Intel RealSense colleagues at https://github.com/IntelRealSense/librealsense/issues/10998#issuecomment-1309087400 and they provided a scope diagram, I will put your latest scope related situation to them.

[echoGee]

Thank you. Looking forward to it.

[MartyG-RealSense]

I have received a response from my colleagues. They confirmed my earlier advice at https://github.com/IntelRealSense/librealsense/issues/10998#issuecomment-1464970246 that HDR is not used with hardware sync.

The reason why HDR and hardware sync cannot be used together is that when running in HDR mode with multiple cameras, the different cameras may run at different exposure times at the same time. Hardware sync may be interrupted and so this usage is not recommended.

[echoGee]

Is there any other way to figure out(electronically) if frame is a short or long exposure ?

[MartyG-RealSense]

In regard to electronic based methods, the link below discusses possible mechanisms for testing the camera shutter.

https://www.largeformatphotography.info/forum/archive/index.php/t-148699.html

However, RealSense cameras have a fixed shutter speed and shutter speed is therefore indirectly controlled through exposure time. So I am not certain that the ideas in that guide will be relevant to RealSense.

In regard to SDK-based methods, if exposure must complete before capture initiates when in genlock mode (4+) then perhaps the time between capture activations will be shorter when the exposure period is shorter.

[MartyG-RealSense]

Hi @echoGee Was the information in the comment above helpful, please? Thanks!

[echoGee]

@MartyG-RealSense , We wanted to figure out the exposure time of the frame from signals in https://user-images.githubusercontent.com/518314/195955643-d2d26fcd-25ad-4abf-a6c7-b00abfe0e408.png Don't think this question is answered

[MartyG-RealSense]

I examined this case again carefully but cannot see an alternative means of measuring exposure electronically, unfortunately.

A common method of measuring exposure with an oscilloscope is to measure camera shutter speed (like in the link below) but RealSense cameras have a fixed-speed aperture.

https://www.picotech.com/library/experiment/camera-shutter-speed

[echoGee]

ok

[MartyG-RealSense]

Hi @echoGee Do you have an update about this case that you can provide, please? Thanks!

[echoGee]

No further questions :)

[MartyG-RealSense]

Thanks very much @echoGee for the update!