emitter automatically turning off when using python api

[dk-teknologisk-bgd]

I have an issue with the emitter turning off whenever the scene is bright. It turns on automatically when the scene is dark. But I want the emitter to always be enabled, just like the default settings when using the Realsense Viewer application.

This happens when using the python API, code to reproduce:

import numpy as np import cv2 import pyrealsense2 as rs

""" Setup """ fps = 5 # 5, 30 pipeline = rs.pipeline() config = rs.config()

config.enable_stream(rs.stream.color, 1280, 720, rs.format.bgr8, fps) config.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, fps) config.enable_stream(rs.stream.infrared, 1280, 720, rs.format.y8, fps)

pipeline_profile = pipeline.start(config)

depth_sensor = pipeline_profile.get_device().first_depth_sensor() depth_sensor.set_option(rs.option.emitter_enabled, 1) depth_sensor.set_option(rs.option.laser_power, 200) depth_sensor.set_option(rs.option.emitter_always_on, 1) depth_sensor.set_option(rs.option.emitter_on_off , 0)

""" Run """ running = True while running: frames = pipeline.wait_for_frames() depth_frame = frames.get_depth_frame() color_frame = frames.get_color_frame() ir_frame = frames.get_infrared_frame() if not depth_frame or not color_frame or not ir_frame: print("failed to get color, depth or IR, skipping") continue

depth = np.asanyarray(depth_frame.get_data())
color = np.asanyarray(color_frame.get_data())
ir = np.asanyarray(ir_frame.get_data())

depth_colormap = cv2.applyColorMap(cv2.convertScaleAbs(depth, alpha=0.03), cv2.COLORMAP_JET)
ir_display = np.repeat(ir[..., np.newaxis], 3, axis=2)
image = np.hstack((color, ir_display, depth_colormap))
image = cv2.resize(image, (0,0), fx=0.7, fy=0.7)
cv2.imshow('img', image)
k = cv2.waitKey(1)

D456, firmware 5.16.0.1 librealsense built from current Master.

[MartyG-RealSense]

Hi @dk-teknologisk-bgd The Emitter Always On option relates to a difference between the emitter of the D415 camera model and the emitter on other RealSense 400 Series models.

On the D415 model only, the emitter is continually in an 'on' state, whilst on the other models the emitter pulses in line with exposure. Enabling Emitter Always On enables cameras other than D415 to emulate the D415 emitter's always-on state instead of pulsing the emitter.

Which camera firmware driver version is currently installed in your camera, please? You can find this information by clicking on the 'Info' button at the top of the RealSense Viewer's options side-panel.

There is an emitter mode called 'Laser Auto' that sounds like the behaviour you describe, where the emitter can turn on or off automatically depending on the lighting conditions. That mode should only activate when rs.option.emitter_enabled is set to '2' though.

What happens if emitter_enabled is set to '0' in your script, please? It should disable the emitter, but if there is an issue that was causing the emitter to use a mode that is 1 higher than the defined one then setting '0' might cause mode '1' to be used, keeping the emitter enabled.

[dk-teknologisk-bgd]

@MartyG-RealSense thanks for your reply. The camera is running firmware 5.16.0.1. I just tried setting: depth_sensor.set_option(rs.option.emitter_enabled, 0) AND moving depth_sensor.set_option(rs.option.laser_power, 200) above this line, in this case the emitter is contantly off, as expected.

I also tried again, without disconnecting the camera, to switch to Realsense Viewer. Again, it works well in the Viewer.

[MartyG-RealSense]

If depth_sensor.set_option(rs.option.emitter_enabled, 1) is moved to the line after while running; so that it is in a loop then continually setting the emitter to '1' might force it to automatically re-enable as soon as it is disabled.

The same should occur if depth_sensor.set_option(rs.option.laser_power, 200) is placed in the loop, since setting Laser Power to a value higher than '0' should automatically turn the emitter on.

[dk-teknologisk-bgd]

@MartyG-RealSense negativ, I tried moving both the depth_sensor.set_option(rs.option.laser_power, 200) and emitter_enabled into the while-loop. It does not resolve the issue. Again, the issue is that the emitter is ONt in the first ~10frames, and then it turns off if I point the camera into a high-illuminated scene. If I then close my small python-script and open RealsenseViewer (without moving the camera), the emiter is constantly on. So it seems to be an issue with the python wrapper imo

[MartyG-RealSense]

Reading again the description of emitter behaviour in the first couple of lines at the top of this discussion, I wonder whether the emitter is actually constantly on but the dots simply are not visible on the infrared image when the camera is pointed at bright light.

You could test this by setting Laser Power to its maximum of '360', as the dot pattern becomes more visible to the camera as the Laser Power value is increased.

You could also confirm whether the emitter is on or off by printing the emitter's current enabled status of '0' (off) or '1' (enabled).

emitter = depth_sensor.get_option(rs.option.emitter_enabled)
print("emitter = ", emitter)

[dk-teknologisk-bgd]

No that is not the issue, see attached. I set the emiter power to 360 in Viewer, and the emitter stays on. When I close the viewer (without disconnecting the camera), and I change my python code to not modify ANY settings, the turns off when facing the ceiling / lamp.

This is getting a bit frustrating. Do you have access to a D456 that you could possibly use to recreate the issue?

[MartyG-RealSense]

I do not have access to a D456 but the D455 I have is the same as a D456 aside from the difference in the outer casing.

I tested the D455 in the Viewer when pointing the camera at a bright ceiling light like you did and like in your test, the emitter pattern remained enabled.

My computer is not equipped to compile and run Python code and so I cannot test the emitter in Python myself.

What happens if you ensure that auto-exposure is enabled like it is by default in the Viewer?

depth_sensor.set_option(rs.option.enable_auto_exposure, 1)

[dk-teknologisk-bgd]

Okay, its unfortunte that you cannot test the python api, as this would have been a lot easier. When I explicitly set depth_sensor.set_option(rs.option.enable_auto_exposure, 1) , the issue is still there.

Please consider this slightly revised python-code, in which the issue is obviously still there:

""" Setup """
fps = 5 # 5, 30
maxExposureTime = 6500

""" Configure """
pipeline = rs.pipeline()
config = rs.config()

config.enable_stream(rs.stream.color, 1280, 720, rs.format.bgr8, fps)
config.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, fps)
config.enable_stream(rs.stream.infrared, 1280, 720, rs.format.y8, fps)

pipeline_profile = config.resolve(pipeline)

depth_sensor = pipeline_profile.get_device().first_depth_sensor()
depth_sensor.set_option(rs.option.auto_exposure_limit, maxExposureTime)
depth_sensor.set_option(rs.option.auto_exposure_limit_toggle, 1)
depth_sensor.set_option(rs.option.enable_auto_exposure, True)
depth_sensor.set_option(rs.option.laser_power, 200)
depth_sensor.set_option(rs.option.emitter_enabled, 1) 
# 0 = frame_metadata_value.frame_laser_power: 0
# 1 = frame_metadata_value.frame_laser_power: 200
# 2 = frame_metadata_value.frame_laser_power: 0

pipeline.start(config)

""" Run """
running = True
n = 0
while running:
    frames = pipeline.wait_for_frames()
    depth_frame,color_frame,ir_frame = frames.get_depth_frame(), frames.get_color_frame(), frames.get_infrared_frame()
    if not depth_frame or not color_frame or not ir_frame:
        print("failed to get color, depth or IR, skipping")
        continue

    depth = np.asanyarray(depth_frame.get_data())
    color = np.asanyarray(color_frame.get_data())
    ir = np.asanyarray(ir_frame.get_data())

When I alter the depth_sensor.set_option(rs.option.emitter_enabled, 1) I get the three different frame_metadata_value.frame_laser_power values, however the emitter is constantly off when I set 1. So this is also an error, that the reported metadata laser power is wrong. I suspect that something in the python api wrongly uses emitter-auto mode even though this is not intended

[dk-teknologisk-bgd]

I have just found, that when I change fps to 30, the issue disappears.

[MartyG-RealSense]

It's excellent to hear that you have found a setting that helps.

[dk-teknologisk-bgd]

however, I still think the issue is valid. I would prefer to only run my camera at 5fps, as I dont need more fps. and also to enable slightly higher exposure time. so any better solution to this issue is prefered

[MartyG-RealSense]

If you would prefer to use 5 FPS, https://github.com/IntelRealSense/librealsense/issues/3169 has a Python script that only uses every "nth" frame. So if you set 30 FPS but only use every 6th frame then you get an effective speed of 5 FPS.

[MartyG-RealSense]

Hi @dk-teknologisk-bgd Do you require further assistance with this case, please? Thanks!

[dk-teknologisk-bgd]

@MartyG-RealSense no I will be using the 15-fps as a workaround, however I would prefer to run at 5fps if the bug is resolved at some point. But we can close this issue

[MartyG-RealSense]

Thanks very much, @dk-teknologisk-bgd - as you suggested, I will close this issue. Thanks again!