search

slightech / MYNT-EYE-S-SDK

MYNT EYE S SDK (MYNT® EYE Standard Camera)
http://www.myntai.com/mynteye/standard
Other
252 stars 145 forks source link

Bad quality pointcloud #102

Open songshan0321 opened 4 years ago

songshan0321 commented 4 years ago

Device: MYNT-EYE-S1040

PointCloud quality is very bad, it seems like not even working properly. Any idea what error is this? How to solve it?

This is the output I get from running using ros wrapper in this SDK.

disparity_computing_method: 1 (bm) mynt-eye-s-indoor-bm

disparity_computing_method: 0 (sgbm) mynt-eye-s-indoor

tiagonovo0 commented 3 years ago

Try to use mynteye-s-calibrator. It only works with Ubuntu 16, but it was the only way I found to correctly calibrate my S1030.

Andrew-rw commented 3 years ago

Hello, I have a similar issue as @songshan0321 , but when I try to run calibration I have the following error:

root@a646dcd58242:/opt/mynt-eye-s-sdk/samples# mynteye-s-calibrator -x 11 -y 7 -s 0.036
I/main.cc:68 Calibrate MYNT EYE S device with chessboard 11x7, size: 0.036m, num: 11
I/utils.cc:35 Detecting MYNT EYE devices
F/channels.cc:477 Check failed: n == file_size The firmware not support getting device info, you could upgrade to latest
Aborted (core dumped)

Below is a device info:

root@a646dcd58242:/opt/mynt-eye-s-sdk/samples# ./get_device_info 
I/utils.cc:35 Detecting MYNT EYE devices
E/device.cc:670 Image params not found, but we need it to process the images. Please `make tools` and use `img_params_writer` to write the image params. If you update the SDK from 1.x, the `SN*.conf` is the file contains them. Besides, you could also calibrate them by yourself. Read the guide doc (https://github.com/slightech/MYNT-EYE-SDK-2-Guide) to learn more.
W/device.cc:677 Intrinsics & extrinsics not exist. Use default intrinsics.
I/device.cc:678 camera calib model: unknow, use default pinhole data
I/utils.cc:48 MYNT EYE devices:
I/utils.cc:51   index: 0, name: MYNT-EYE-S1040, sn: xxMANUALLY_EDITEDxxx, firmware: 0.0
I/utils.cc:60 Only one MYNT EYE device, select index: 0
I/synthetic.cc:49 camera calib model: pinhole
E/disparity_processor.cc:57 BM not supported in opencv 2.x, use sgbm
2.5.0W/version_checker.cc:158 You're using a custom mynteye device
I/get_device_info.cc:23 Device name: MYNT-EYE-S1040
I/get_device_info.cc:24 Serial number: xxMANUALLY_EDITEDxxx
I/get_device_info.cc:25 Firmware version: 0.0
I/get_device_info.cc:26 Hardware version: 3.0
I/get_device_info.cc:27 Spec version: 1.3
I/get_device_info.cc:28 Lens type: 0000
I/get_device_info.cc:29 IMU type: 0000
I/get_device_info.cc:30 Nominal baseline: 0
I/get_device_info.cc:31 Auxiliary chip version: 0.0
I/get_device_info.cc:33 Isp version: 0

I'm using binaries from ppa:slightech/mynt-eye-s-sdk @tiagonovo0 could you tell me do I need to update the firmware? Thank you

tiagonovo0 commented 3 years ago

Hi, I had the same error when running on Ubuntu 16 inside docker. I know that virtual machines are not recommended too because of the usb 3.0. For me, it only worked when I installed ubuntu 16 from ground.

I do not know the S1040 model but from what I see in your device info, it is look like you do not have any parameter defined so you trully need to calibrate it. Try to update the firmware and calibrate it.

Bests

Andrew-rw commented 3 years ago

Hi @tiagonovo0 , Thank you for the quick reply. I've updated firmware (the latest what I've found was 2.7) by using Windows application. After firmware upgrade camera is recognizied as s1030: 

user@DEV-PC:~/MYNT-EYE-S-SDK/samples/_output/bin$ ./get_device_info 
I/utils.cc:35 Detecting MYNT EYE devices
I/utils.cc:48 MYNT EYE devices:
I/utils.cc:51   index: 0, name: MYNT-EYE-S1030, sn: xxMANUALLY_EDITEDxxx, firmware: 2.7
I/utils.cc:60 Only one MYNT EYE device, select index: 0
I/synthetic.cc:49 camera calib model: pinhole
2.5.0I/get_device_info.cc:23 Device name: MYNT-EYE-S1030
I/get_device_info.cc:24 Serial number: xxMANUALLY_EDITEDxxx
I/get_device_info.cc:25 Firmware version: 2.7
I/get_device_info.cc:26 Hardware version: 0.0
I/get_device_info.cc:27 Spec version: 1.3
I/get_device_info.cc:28 Lens type: 0000
I/get_device_info.cc:29 IMU type: 0000
I/get_device_info.cc:30 Nominal baseline: 0
I/get_device_info.cc:31 Auxiliary chip version: 0.0
I/get_device_info.cc:33 Isp version: 0

I also tried to set default configurations for image and imu using files provided in SDK. But the problem is that after firmware update I can get mono and diparity images neither in ROS wrapper nor using SDK samples. Running ROS node does not provide any error except TF error:

user@DEV-PC:~$ roslaunch mynt_eye_ros_wrapper display.launch 
... logging to /home/user/.ros/log/e6a18944-1fa2-11eb-a463-346895594943/roslaunch-DEV-PC-3051.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://127.0.0.1:35515/

SUMMARY
========

PARAMETERS
 * /mynteye/depth/image_raw/disable_pub_plugins: ['image_transport...
 * /mynteye/disparity/image_norm/disable_pub_plugins: ['image_transport...
 * /mynteye/disparity/image_raw/disable_pub_plugins: ['image_transport...
 * /mynteye/left/image_mono/disable_pub_plugins: ['image_transport...
 * /mynteye/left/image_raw/disable_pub_plugins: ['image_transport...
 * /mynteye/left_rect/image_rect/disable_pub_plugins: ['image_transport...
 * /mynteye/left_rect_mono/disable_pub_plugins: ['image_transport...
 * /mynteye/mynteye_wrapper_node/base_frame_id: mynteye_link
 * /mynteye/mynteye_wrapper_node/depth_frame_id: mynteye_depth_frame
 * /mynteye/mynteye_wrapper_node/depth_topic: depth/image_raw
 * /mynteye/mynteye_wrapper_node/depth_type: 0
 * /mynteye/mynteye_wrapper_node/disparity_computing_method: 1
 * /mynteye/mynteye_wrapper_node/disparity_frame_id: mynteye_disparity...
 * /mynteye/mynteye_wrapper_node/disparity_norm_frame_id: mynteye_disparity...
 * /mynteye/mynteye_wrapper_node/disparity_norm_topic: disparity/image_norm
 * /mynteye/mynteye_wrapper_node/disparity_topic: disparity/image_raw
 * /mynteye/mynteye_wrapper_node/enable_depth: False
 * /mynteye/mynteye_wrapper_node/enable_disparity: False
 * /mynteye/mynteye_wrapper_node/enable_disparity_norm: False
 * /mynteye/mynteye_wrapper_node/enable_left_rect: False
 * /mynteye/mynteye_wrapper_node/enable_points: False
 * /mynteye/mynteye_wrapper_node/enable_right_rect: False
 * /mynteye/mynteye_wrapper_node/gravity: 9.8
 * /mynteye/mynteye_wrapper_node/imu_topic: imu/data_raw
 * /mynteye/mynteye_wrapper_node/is_multiple: False
 * /mynteye/mynteye_wrapper_node/left_frame_id: mynteye_left_frame
 * /mynteye/mynteye_wrapper_node/left_mono_topic: left/image_mono
 * /mynteye/mynteye_wrapper_node/left_rect_frame_id: mynteye_left_rect...
 * /mynteye/mynteye_wrapper_node/left_rect_topic: left_rect/image_rect
 * /mynteye/mynteye_wrapper_node/left_topic: left/image_raw
 * /mynteye/mynteye_wrapper_node/model_positioin_x: 0.0
 * /mynteye/mynteye_wrapper_node/model_positioin_y: -0.176
 * /mynteye/mynteye_wrapper_node/model_positioin_z: 0.0
 * /mynteye/mynteye_wrapper_node/model_rotation_x: 1.5708
 * /mynteye/mynteye_wrapper_node/model_rotation_y: 0.0
 * /mynteye/mynteye_wrapper_node/model_rotation_z: 1.5708
 * /mynteye/mynteye_wrapper_node/points_frame_id: mynteye_points_frame
 * /mynteye/mynteye_wrapper_node/points_topic: points/data_raw
 * /mynteye/mynteye_wrapper_node/right_frame_id: mynteye_right_frame
 * /mynteye/mynteye_wrapper_node/right_mono_topic: right/image_mono
 * /mynteye/mynteye_wrapper_node/right_rect_frame_id: mynteye_right_rec...
 * /mynteye/mynteye_wrapper_node/right_rect_topic: right_rect/image_...
 * /mynteye/mynteye_wrapper_node/right_topic: right/image_raw
 * /mynteye/mynteye_wrapper_node/s1030_mesh_file: S1030-0315.obj
 * /mynteye/mynteye_wrapper_node/serial_number: 
 * /mynteye/mynteye_wrapper_node/standard/accel_range: -1
 * /mynteye/mynteye_wrapper_node/standard/brightness: -1
 * /mynteye/mynteye_wrapper_node/standard/contrast: -1
 * /mynteye/mynteye_wrapper_node/standard/desired_brightness: -1
 * /mynteye/mynteye_wrapper_node/standard/exposure_mode: -1
 * /mynteye/mynteye_wrapper_node/standard/frame_rate: -1
 * /mynteye/mynteye_wrapper_node/standard/gain: -1
 * /mynteye/mynteye_wrapper_node/standard/gyro_range: -1
 * /mynteye/mynteye_wrapper_node/standard/hdr_mode: -1
 * /mynteye/mynteye_wrapper_node/standard/imu_frequency: -1
 * /mynteye/mynteye_wrapper_node/standard/ir_control: 80
 * /mynteye/mynteye_wrapper_node/standard/max_exposure_time: -1
 * /mynteye/mynteye_wrapper_node/standard/max_gain: -1
 * /mynteye/mynteye_wrapper_node/standard/request_index: 0
 * /mynteye/mynteye_wrapper_node/standard2/accel_low_filter: -1
 * /mynteye/mynteye_wrapper_node/standard2/accel_range: -1
 * /mynteye/mynteye_wrapper_node/standard2/brightness: -1
 * /mynteye/mynteye_wrapper_node/standard2/desired_brightness: -1
 * /mynteye/mynteye_wrapper_node/standard2/exposure_mode: -1
 * /mynteye/mynteye_wrapper_node/standard2/gyro_low_filter: -1
 * /mynteye/mynteye_wrapper_node/standard2/gyro_range: -1
 * /mynteye/mynteye_wrapper_node/standard2/imu_process_mode: 2
 * /mynteye/mynteye_wrapper_node/standard2/max_exposure_time: -1
 * /mynteye/mynteye_wrapper_node/standard2/max_gain: -1
 * /mynteye/mynteye_wrapper_node/standard2/min_exposure_time: -1
 * /mynteye/mynteye_wrapper_node/standard2/request_index: 2
 * /mynteye/mynteye_wrapper_node/standard200b/accel_low_filter: -1
 * /mynteye/mynteye_wrapper_node/standard200b/accel_range: -1
 * /mynteye/mynteye_wrapper_node/standard200b/brightness: -1
 * /mynteye/mynteye_wrapper_node/standard200b/desired_brightness: -1
 * /mynteye/mynteye_wrapper_node/standard200b/exposure_mode: -1
 * /mynteye/mynteye_wrapper_node/standard200b/gyro_low_filter: -1
 * /mynteye/mynteye_wrapper_node/standard200b/gyro_range: -1
 * /mynteye/mynteye_wrapper_node/standard200b/max_exposure_time: -1
 * /mynteye/mynteye_wrapper_node/standard200b/max_gain: -1
 * /mynteye/mynteye_wrapper_node/standard200b/min_exposure_time: -1
 * /mynteye/mynteye_wrapper_node/standard210a/accel_low_filter: -1
 * /mynteye/mynteye_wrapper_node/standard210a/accel_range: -1
 * /mynteye/mynteye_wrapper_node/standard210a/brightness: -1
 * /mynteye/mynteye_wrapper_node/standard210a/desired_brightness: -1
 * /mynteye/mynteye_wrapper_node/standard210a/exposure_mode: -1
 * /mynteye/mynteye_wrapper_node/standard210a/gyro_low_filter: -1
 * /mynteye/mynteye_wrapper_node/standard210a/gyro_range: -1
 * /mynteye/mynteye_wrapper_node/standard210a/iic_address_setting: -1
 * /mynteye/mynteye_wrapper_node/standard210a/max_exposure_time: -1
 * /mynteye/mynteye_wrapper_node/standard210a/max_gain: -1
 * /mynteye/mynteye_wrapper_node/standard210a/min_exposure_time: -1
 * /mynteye/mynteye_wrapper_node/temperature_frame_id: mynteye_temperatu...
 * /mynteye/mynteye_wrapper_node/temperature_topic: temperature/data_raw
 * /mynteye/right/image_mono/disable_pub_plugins: ['image_transport...
 * /mynteye/right/image_raw/disable_pub_plugins: ['image_transport...
 * /mynteye/right_rect/image_rect/disable_pub_plugins: ['image_transport...
 * /mynteye/right_rect_mono/disable_pub_plugins: ['image_transport...
 * /rosdistro: melodic
 * /rosversion: 1.14.9

NODES
  /
    rviz (rviz/rviz)
  /mynteye/
    mynteye_wrapper_node (mynt_eye_ros_wrapper/mynteye_wrapper_node)

auto-starting new master
process[master]: started with pid [3070]
ROS_MASTER_URI=http://127.0.0.1:11311

setting /run_id to e6a18944-1fa2-11eb-a463-346895594943
process[rosout-1]: started with pid [3081]
started core service [/rosout]
process[mynteye/mynteye_wrapper_node-2]: started with pid [3084]
process[rviz-3]: started with pid [3086]
[ INFO] [1604607009.024516090]: Initializing nodelet with 8 worker threads.
[ INFO] [1604607009.254807367]: rviz version 1.13.13
[ INFO] [1604607009.254839585]: compiled against Qt version 5.9.5
[ INFO] [1604607009.254848274]: compiled against OGRE version 1.9.0 (Ghadamon)
[ INFO] [1604607009.263145662]: Forcing OpenGl version 0.
[ INFO] [1604607009.324598596]: Detecting MYNT EYE devices
[ INFO] [1604607009.551353042]: MYNT EYE devices:
[ INFO] [1604607009.551455111]:   index: 0, name: MYNT-EYE-S1030, serial number: xxMANUALLY_EDITEDxxx
[ INFO] [1604607009.552625603]: Only one MYNT EYE device, select index: 0
I/synthetic.cc:49 camera calib model: pinhole
[ INFO] [1604607009.648890297]: Option::GAIN: 24
[ INFO] [1604607009.649482280]: Option::BRIGHTNESS: 120
[ INFO] [1604607009.650052379]: Option::CONTRAST: 116
[ INFO] [1604607009.650736638]: Option::FRAME_RATE: 25
[ INFO] [1604607009.651427797]: Option::IMU_FREQUENCY: 200
[ INFO] [1604607009.652108104]: Option::EXPOSURE_MODE: 0
[ INFO] [1604607009.652789434]: Option::MAX_GAIN: 48
[ INFO] [1604607009.653470216]: Option::MAX_EXPOSURE_TIME: 240
[ INFO] [1604607009.654242995]: Option::DESIRED_BRIGHTNESS: 192
[ INFO] [1604607009.654712457]: Set standard/ir_control to 80
[ INFO] [1604607009.655088089]: Option::IR_CONTROL: 0
[ INFO] [1604607009.655773511]: Option::HDR_MODE: 0
[ INFO] [1604607009.656455676]: Option::ACCELEROMETER_RANGE: 8
[ INFO] [1604607009.657141495]: Option::GYROSCOPE_RANGE: 1000
[ INFO] [1604607009.679748311]: Advertized on topic left/image_raw
[ INFO] [1604607009.689219475]: Advertized on topic right/image_raw
[ INFO] [1604607009.698676467]: Advertized on topic left_rect/image_rect
[ INFO] [1604607009.708131288]: Advertized on topic right_rect/image_rect
[ INFO] [1604607009.717614335]: Advertized on topic disparity/image_raw
[ INFO] [1604607009.726768322]: Advertized on topic disparity/image_norm
[ INFO] [1604607009.736261487]: Advertized on topic depth/image_raw
[ INFO] [1604607009.736623171]: Advertized on topic points/data_raw
[ INFO] [1604607009.737439131]: Advertized on topic imu/data_raw
[ INFO] [1604607009.737815449]: Advertized on topic temperature/data_raw
[ INFO] [1604607009.741974599]: Advertized service get_info
[ INFO] [1604607010.068299909]: Stereo is NOT SUPPORTED
[ INFO] [1604607010.068457452]: OpenGl version: 3 (GLSL 1.3).
[ERROR] [1604607010.252130082]: Ignoring transform for child_frame_id "camera_imu_frame" from authority "unknown_publisher" because of an invalid quaternion in the transform (0.000000 0.000000 -0.000000 0.500000)

And running, for example, get_disparity sample returns "Timeout waiting for key frames" error:

user@DEV-PC:~/MYNT-EYE-S-SDK/samples/_output/bin$ ./get_disparity 
I/utils.cc:35 Detecting MYNT EYE devices
I/utils.cc:48 MYNT EYE devices:
I/utils.cc:51   index: 0, name: MYNT-EYE-S1030, sn: xxMANUALLY_EDITEDxxx, firmware: 2.7
I/utils.cc:60 Only one MYNT EYE device, select index: 0
I/synthetic.cc:49 camera calib model: pinhole
2.5.0I/utils.cc:99 MYNT EYE requests:
I/utils.cc:102   index: 0, request: width: 752, height: 480, format: Format::YUYV, fps: 25
I/utils.cc:102   index: 1, request: width: 376, height: 240, format: Format::YUYV, fps: 25
I/utils.cc:113 There are 2 stream requests, select index: 
0
disparity fps:
F/streams.cc:165 Timeout waiting for key frames. Please use USB 3.0, and not in virtual machine.
Aborted

I'm using USB 3.0 and it worked on previous unknown version of firmware. But still I can get stereo image using Windows application or by getting video stream directly from device (/dev/videoX) @tiagonovo0 , maybe you faced with similar issue before?

Thank you

tiagonovo0 commented 3 years ago

Actually, I did not have those issues, even after I update the firmware to 2.7.

Have you tried to calibrate the camera by manually? Because I think they already are calibrated with the equidistant model, Kannala Brandt. The other difference I have is OpenGl version: 4.6 (GLSL 4.6).

For the command roslaunch mynt_eye_ros_wrapper display.launch, try to check the TFs on ROS.

Note that your hardware version has changed. That did not happen to me either. Mine is 2.0.

yashart commented 3 years ago

@Andrew-rw , I have had a similar issue with mynteye s1040 and firmware 2.7. It is not a good fix, but I commented out checks on header and checksum for image package from camera, and it is working. For me, header is allways 0 (not 0x3B) and checksum is bad. I think, it is a firmware problem. fix-s1040.patch.txt

randohinn commented 3 years ago

Seeing this as well, sdk fix helped get a stream running again, but the pointcloud looks awful. mynteye-s-calibrator on 16.04 can't talk to the wiped hardware version firmware, and the ros camera calibrator assertion errors on calibrating..

liu83 commented 2 years ago

@Andrew-rw @yashart please refer to #116 where the issue can be solved by updating the firmware.

Andrew-rw commented 2 years ago

@liu83 may I ask you to share the firmware image? Unfortunately I have difficulties downloading it from the resource which @gh18037581263 has mentioned:( Maybe put it as a pull request to the https://github.com/Andrew-rw/MYNT_EYE_TESTS ?

Thank you!