search

ouster-lidar / ouster-sdk

Ouster, Inc. sample code
Other
464 stars 438 forks source link

save out near_ir images for labelling #417

Closed bensail-ai closed 2 years ago

bensail-ai commented 2 years ago

I would like to save out near_ir images from the scans for computer vision labelling. However the near_ir sensor is not equalised to 0-255 standard for images. I have tried converting to uint8 and 16 with no success. Example outputs are attached.

example of my code below: scans=client.Scans(source) scan = nth(scans,84) IR=scan.field(client.ChanField.NEAR_IR) IR_img = IR_destaggered.astype(np.uint16) cv2.imwrite('test2',IR_img) test2

kairenw commented 2 years ago

Yes, that's correct. Unfortunately NEAR_IR is not equalized. Not sure what the original scene was but if it was super bright outside or pointed at the sun, you might get images like that for your NEAR_IR. If you want something equalized, you could try the REFLECTIVITY (if your sensor has calibrated reflectivity which you can check with the tcp command get_calibration_status).

bensail-ai commented 2 years ago

I thought that might be the case. However I tested with the sample data. In the Ouster-viz programme the near_ir images look great. However when I try to replicate with my own script I get the above image. How is it done in ouster-viz?

kairenw commented 2 years ago

Sorry -- I looked at the image and while I read the word "equalized" I was thinking it was "overexposed" (as you can see from the answer I gave about brightness outside/the sun).

NEAR_IR is also not equalized on the sensor, but you can apply a bit of image processing after the fact, and we in fact apply some NEAR_IR in the default python viz. These APIs are subject to change, but try, in ouster._utils, the AutoExposure and BeamUniformityCorrector. These are just bindings to the same C++ classes which are implemented here.

You can see how we're applying them in the code for the ouster-viz.

kairenw commented 2 years ago

The AutoExposure does help with overexposed as well, unless the image has just lost too much information. What I'm seeing is an almost all-white image, so I'm not sure it will help with the areas that look like that, but in other places it will scale the contrast so that the bulk of it ends up between the specified percentiles. Useful if you want to do labeling or any CV work on it for sure.

kairenw commented 2 years ago

Closing this out now. Feel free to re-open with questions.