search

jsk-ros-pkg / jsk_robot

jsk-ros-pkg/jsk_robot
https://github.com/jsk-ros-pkg/jsk_robot
73 stars 97 forks source link

D405をROSで使用する際の問題と解決策? #1827

Open softyanija opened 1 year ago

softyanija commented 1 year ago

cc: @sktometometo

問題

使っていてあたった問題

PCの環境

amabe@comet:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.6 LTS
Release:    20.04
Codename:   focal

Realsense 周りでインストールされているパッケージ

amabe@comet:~$ apt list | grep realsense

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

librealsense2-dbg/focal 2.54.1-0~realsense.9590 amd64
librealsense2-dev/focal 2.54.1-0~realsense.9590 amd64 [upgradable from: 2.53.1-0~realsense0.8250]
librealsense2-dkms/focal,now 1.3.19-0ubuntu1 all [installed]
librealsense2-gl-dbg/focal 2.54.1-0~realsense.9590 amd64
librealsense2-gl-dev/focal 2.54.1-0~realsense.9590 amd64
librealsense2-gl/focal 2.54.1-0~realsense.9590 amd64
librealsense2-net-dbg/focal 2.53.1-0~realsense0.8250 amd64
librealsense2-net-dev/focal 2.53.1-0~realsense0.8250 amd64
librealsense2-net/focal 2.53.1-0~realsense0.8250 amd64
librealsense2-udev-rules/focal 2.54.1-0~realsense.9590 amd64 [upgradable from: 2.53.1-0~realsense0.8250]
librealsense2-utils/focal 2.54.1-0~realsense.9590 amd64
librealsense2/focal 2.54.1-0~realsense.9590 amd64 [upgradable from: 2.53.1-0~realsense0.8250]
ros-noetic-librealsense2-dbgsym/focal 2.50.0-1focal.20211115.134455 amd64
ros-noetic-librealsense2/focal,now 2.50.0-1focal.20211115.134455 amd64 [installed]
ros-noetic-realsense2-camera-dbgsym/focal 2.3.2-1focal.20230215.235752 amd64
ros-noetic-realsense2-camera/focal 2.3.2-1focal.20230215.235752 amd64
ros-noetic-realsense2-description/focal 2.3.2-1focal.20230524.152515 amd64

Workspace configuration

amabe@comet:~/robothand_dataset_ws/src$ wstool export
# THIS IS AN AUTOGENERATED FILE, LAST GENERATED USING wstool ON 2023-06-26
- git:
    local-name: DREAM
    uri: git@github.com:softyanija/DREAM.git
    version: amabe_dev
- git:
    local-name: chainer-openpose
    uri: git@github.com:iory/chainer-openpose.git
    version: master
- git:
    local-name: pr2_common
    uri: git@github.com:softyanija/pr2_common.git
    version: add_pr2hand
- git:
    local-name: realsense-ros
    uri: https://github.com/IntelRealSense/realsense-ros.git
    version: development < d405/development
- git:
    local-name: robokey
    uri: git@github.com:softyanija/robokey.git
    version: add_pr2hand
- git:
    local-name: ros-drivers/rosserial
    uri: https://github.com/708yamaguchi/rosserial.git
    version: m5stack-ros-noetic
- git:
    local-name: scikit-robot
    uri: git@github.com:softyanija/scikit-robot.git
    version: add_pr2handmodel
- git:
    local-name: scikit-robot-models
    uri: git@github.com:softyanija/scikit-robot-models.git
    version: pr2hand_dev

realsense-ros の checkout している commit は

ommit bbe22d7d51937120453c9fded272de97105b30ad (HEAD -> development, d405/development)
Author: rjwb1 <rbelshaw@sagarobotics.com>
Date:   Wed Mar 23 19:27:27 2022 +0000

    update d405 HWID

Case1 フレームがおかしくなる

realsense-viewer では以下のように正常に画像が表示されている

realsense-viewer

にもかかわらず, 以下のコマンドで roslaunch すると, /camera_infoにおける解像度が反映されない + 表示される画像がおかしい ( ラインがずれている?)

roslaunch realsense2_camera rs_rgbd.launch color_width:=1280 color_height:=720 color_fps:=15

TODO

記録する前にデバッグしてしまったため詳細は記録できなかった。

参考リンク: https://github.com/IntelRealSense/librealsense/issues/10967 このissueではlibrealsenseのバージョンが2.51,2.52であり、原因になっている？

Case 2

一度解決して通信できるようになったあとに表示された。 以下のエラーが表示されて画像が全くpublishされてなかった。

[ INFO] [1687758910.556341104]: RealSense Node Is Up!
[ WARN] [1687758911.019078812]: 
 26/06 14:55:11,020 WARNING [139882224375552] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 26/06 14:55:11,071 WARNING [139882224375552] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
[ WARN] [1687758911.889231598]: Hardware Notification:RT MIPI error,1.68776e+12,Error,Hardware Error
 26/06 14:55:12,465 ERROR [139881963370240] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 132
[ WARN] [1687758912.890104601]: Hardware Notification:RT MIPI error,1.68776e+12,Error,Hardware Error
^C[camera/points_xyzrgb_hw_registered-4] killing on exit

このケースではカメラICとの通信に失敗していそう

解決策

ros-noetic-librealsense が入っていたので purge

特定のバージョンでは動作していたはずという読みで librealsense のバージョンを確認

amabe@comet:~/robothand_dataset_ws/src/realsense-ros/realsense2_camera$ apt list librealsense2-dev -a
Listing... Done
librealsense2-dev/focal 2.54.1-0~realsense.9590 amd64 [upgradable from: 2.53.1-0~realsense0.8250]
librealsense2-dev/focal 2.54.1-0~realsense.9587 amd64
librealsense2-dev/focal,now 2.53.1-0~realsense0.8250 amd64 [installed,upgradable to: 2.54.1-0~realsense.9590]
librealsense2-dev/focal 2.51.1-0~realsense0.7526 amd64
librealsense2-dev/focal 2.51.1-0~realsense0.7520 amd64
librealsense2-dev/focal 2.50.0-0~realsense0.6128 amd64
librealsense2-dev/focal 2.49.0-0~realsense0.5307 amd64
librealsense2-dev/focal 2.49.0-0~realsense0.5268 amd64
librealsense2-dev/focal 2.49.0-0~realsense0.5199 amd64
librealsense2-dev/focal 2.48.0-0~realsense0.4976 amd64
librealsense2-dev/focal 2.47.0-0~realsense0.4806 amd64
librealsense2-dev/focal 2.45.0-0~realsense0.4552 amd64
librealsense2-dev/focal 2.44.0-0~realsense0.4390 amd64
librealsense2-dev/focal 2.43.0-0~realsense0.4278 amd64
librealsense2-dev/focal 2.42.0-0~realsense0.4060 amd64
librealsense2-dev/focal 2.41.0-0~realsense0.3963 amd64
librealsense2-dev/focal 2.40.0-0~realsense0.3815 amd64

最新版にアップグレード

sudo apt librealsense2-dev=2.54.1-0~realsense.9590 librealsense2=2.54.1-0~realsense.9590

realsense2 の build し直し

roscd realsense2_camera
catkin clean realsense2_camera
catkin build realsense2_camera

その後, sourceして

roslaunch realsense2_camera rs_rgbd.launch color_width:=1280 color_height:=720 color_fps:=15

で正常に画像がpublishされている

<TODO 画像> ok_rqt_viewer

amabe@comet:~$ rostopic echo -n 1 /camera/color/camera_info
header: 
  seq: 364
  stamp: 
    secs: 1687761054
    nsecs: 496928215
  frame_id: "camera_color_optical_frame"
height: 720
width: 1280
distortion_model: "plumb_bob"
D: [-0.053694773465394974, 0.05958225205540657, 0.00016518920892849565, 7.685108357691206e-06, -0.018904663622379303]
K: [646.7379150390625, 0.0, 648.179443359375, 0.0, 645.7910766601562, 353.37396240234375, 0.0, 0.0, 1.0]
R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
P: [646.7379150390625, 0.0, 648.179443359375, 0.0, 0.0, 645.7910766601562, 353.37396240234375, 0.0, 0.0, 0.0, 1.0, 0.0]
binning_x: 0
binning_y: 0
roi: 
  x_offset: 0
  y_offset: 0
  height: 0
  width: 0
  do_rectify: False
---
amabe@comet:~$ rostopic echo -n 1 /camera/color/camera_info
header: 
  seq: 364
  stamp: 
    secs: 1687761054
    nsecs: 496928215
  frame_id: "camera_color_optical_frame"
height: 720
width: 1280
distortion_model: "plumb_bob"
D: [-0.053694773465394974, 0.05958225205540657, 0.00016518920892849565, 7.685108357691206e-06, -0.018904663622379303]
K: [646.7379150390625, 0.0, 648.179443359375, 0.0, 645.7910766601562, 353.37396240234375, 0.0, 0.0, 1.0]
R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
P: [646.7379150390625, 0.0, 648.179443359375, 0.0, 0.0, 645.7910766601562, 353.37396240234375, 0.0, 0.0, 0.0, 1.0, 0.0]
binning_x: 0
binning_y: 0
roi: 
  x_offset: 0
  y_offset: 0
  height: 0
  width: 0
  do_rectify: False
---
sktometometo commented 1 year ago

@softyanija Can you add how to use D405 to https://github.com/jsk-ros-pkg/jsk_recognition/blob/master/doc/install_realsense_camera.rst ?

softyanija commented 1 year ago

OK. I will do it next Monday.