/fiducial_transform not publishing when using USB camera

[roaz1801]

Hello, I'm trying to get the transform of a marker that has been detected. Using ROS Noetic, a USB camera with the cv_camera package, and the camera is attached to a turtlebot. Using a USB camera because I couldn't get the raspi cam to work.

The USB camera works and is detecting a marker when checking the rqt image, and the when echoing the /fiducial_vertices topic you get feedback from it. The /fiducial_transform however does not print anything when using echo, no fiducial shows up in the tf_tree. This all worked well in the simulations, but now after moving to hardware and changing to a USB camera it is not working.

This is the launch file for the turtlebot:

<launch>
   <include file="$(find turtlebot3_bringup)/launch/turtlebot3_robot.launch" ns="tb3_0">
  </include>

  <arg name="model" default="waffle_pi"/>
  <arg name="first_tb3" default="tb3_0"/>

  <node pkg = "cv_camera" type="cv_camera_node" name="cv_camera" output="screen">
   <param name="image_width" value="640"/>
   <param name="image_heigth" value="480"/>
   <param name="rate" value="60"/>
   </node>

  <node pkg="tf" type="static_transform_publisher" name="camera_frames_pub" args="0.05 0.0 0.1 0 0 0 tb3_0/base_link /camera 35"/>

     <!-- Start marker detector-->
 <node name="aruco_detect" pkg="aruco_detect" type="aruco_detect">
    <param name="image_transport" value="compressed" />
    <param name="publish_images" value="true" />
    <param name="fiducial_len" value="0.15" />
    <param name="dictionary" value="0" />
    <param name="do_pose_estimation" value="true" />
    <param name="ignore_fiducials" value="" />
    <param name="fiducial_len_override" value="" />
    <remap from="/camera/compressed" to="/cv_camera/image_raw/compressed"/>
    <remap from="/camera_info" to="/cv_camera/camera_info"/>
  </node>

 <group ns = "$(arg first_tb3)">
    <param name="robot_description" command="$(find xacro)/xacro --inorder '$(find turtlebot3_description)/urdf/turtlebot3_$(arg model).urdf.xacro' botname:=$(arg firs>

     <node pkg="robot_state_publisher" type="robot_state_publisher" name="robot_state_publisher" output="screen">
      <param name="publish_frequency" type="double" value="50.0" />
      <param name="tf_prefix" value="$(arg first_tb3)" />
    </node>
 </group>
 </launch>

where the main difference between this and the launch file used in the simulations is the remapping of the camera/compressed and caemra/info.

Also asked the same question here: https://answers.ros.org/question/392521/aruco_detect-fiducial_transform-not-giving-any-transforms-when-using-usb-camera/

Thanks in advance

[anfederman]

I have gotten Aruco detect to work on a USB cam in Noetic. I think you are correct in thinking it is a transform error. I will spin up a USB camera and send you my launch files.

On Thu, Dec 9, 2021, 7:35 AM roaz1801 @.***> wrote:

Hello, I'm trying to get the transform of a marker that has been detected. Using ROS Noetic, a USB camera with the cv_camera package, and the camera is attached to a turtlebot. Using a USB camera because I couldn't get the raspi cam to work.

The USB camera works and is detecting a marker when checking the rqt image, and the when echoing the /fiducial_vertices topic you get feedback from it. The /fiducial_transform however does not print anything when using echo, no fiducial shows up in the tf_tree. This all worked well in the simulations, but now after moving to hardware and changing to a USB camera it is not working.

This is the launch file for the turtlebot:

where the main difference between this and the launch file used in the simulations is the remapping of the camera/compressed and caemra/info.

Also asked the same question here: https://answers.ros.org/question/392521/aruco_detect-fiducial_transform-not-giving-any-transforms-when-using-usb-camera/

Thanks in advance

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/UbiquityRobotics/fiducials/issues/263, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHZ372EQUKOH4JHM3VU6ZDUQDEF3ANCNFSM5JWY2EMA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[anfederman]

PS Raspicam node works in both Arm32 and ARM64 in noetic.

On Thu, Dec 9, 2021, 7:35 AM roaz1801 @.***> wrote:

Hello, I'm trying to get the transform of a marker that has been detected. Using ROS Noetic, a USB camera with the cv_camera package, and the camera is attached to a turtlebot. Using a USB camera because I couldn't get the raspi cam to work.

The USB camera works and is detecting a marker when checking the rqt image, and the when echoing the /fiducial_vertices topic you get feedback from it. The /fiducial_transform however does not print anything when using echo, no fiducial shows up in the tf_tree. This all worked well in the simulations, but now after moving to hardware and changing to a USB camera it is not working.

This is the launch file for the turtlebot:

where the main difference between this and the launch file used in the simulations is the remapping of the camera/compressed and caemra/info.

Also asked the same question here: https://answers.ros.org/question/392521/aruco_detect-fiducial_transform-not-giving-any-transforms-when-using-usb-camera/

Thanks in advance

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/UbiquityRobotics/fiducials/issues/263, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHZ372EQUKOH4JHM3VU6ZDUQDEF3ANCNFSM5JWY2EMA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[roaz1801]

I did try making the rospicam work on noetic but at some point I decided it was just too much effort when a USB camera could work too. Tried to make it work with both the image given in the https://emanual.robotis.com/docs/en/platform/turtlebot3/sbc_setup/#sbc-setup and the default image from Raspberry Pi Imager. Never got raspistill to work there.

[anfederman]

Try:

Add camera support libraries and test:

sudo apt install libraspberrypi-bin libraspberrypi-dev

this worked on ARM64 Noetic Pi 4 Ubuntu 20.04 built from the server image.

then raspistill -o test.jpg

On Thu, Dec 9, 2021 at 9:31 AM roaz1801 @.***> wrote:

I did try making the rospicam work on noetic but at some point I decided it was just too much effort when a USB camera could work too. Tried to make it work with both the image given in the https://emanual.robotis.com/docs/en/platform/turtlebot3/sbc_setup/#sbc-setup and the default image from Raspberry Pi Imager. Never got raspistill to work there.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/UbiquityRobotics/fiducials/issues/263#issuecomment-990065967, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHZ37ZP7N5UKKOM4W3ILDLUQDRXVANCNFSM5JWY2EMA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[roaz1801]

Try: Add camera support libraries and test: sudo apt install libraspberrypi-bin libraspberrypi-dev this worked on ARM64 Noetic Pi 4 Ubuntu 20.04 built from the server image. then raspistill -o test.jpg … On Thu, Dec 9, 2021 at 9:31 AM roaz1801 @.***> wrote: I did try making the rospicam work on noetic but at some point I decided it was just too much effort when a USB camera could work too. Tried to make it work with both the image given in the https://emanual.robotis.com/docs/en/platform/turtlebot3/sbc_setup/#sbc-setup and the default image from Raspberry Pi Imager. Never got raspistill to work there. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#263 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHZ37ZP7N5UKKOM4W3ILDLUQDRXVANCNFSM5JWY2EMA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Thank you, will try this next time I get to use the physical turtlebot and report back, but it might take a few days until next time I get to use the hardware.

[roaz1801]

Also, I have now tested using only a USB camera not connected to anything, so I don't think this issue has anything to do with the turtlebot at all, might just be that the aruco_detect node is not writing to the right topic. From the /fiducial_vertices I get:

header: 
  seq: 6092
  stamp: 
    secs: 1639076096
    nsecs: 909698461
  frame_id: ''
image_seq: 6093
fiducials: 
  - 
    fiducial_id: 0
    direction: 0
    x0: 499.4951171875
    y0: 222.32830810546875
    x1: 552.80908203125
    y1: 223.65765380859375
    x2: 549.2293701171875
    y2: 277.81787109375
    x3: 495.5509033203125
    y3: 274.6446228027344
---

And in the rqt image viewer the marker is being detected, but no /fiducial_transform output or fiducial_0 transform in tf tree

Edit: Tested the simulation I have that actually works and has a /fiducial_transform output and fuducial_0 in the tf tree, seems I'm missing a frame_id in the hardware implementation, not sure where you set that.

header: 
  seq: 6090
  stamp: 
    secs: 615
    nsecs: 406000000
  frame_id: "tb3_1/camera_rgb_optical_frame"
image_seq: 6090
fiducials: 
  - 
    fiducial_id: 0
    direction: 0
    x0: 272.4456481933594
    y0: 151.20103454589844
    x1: 368.3885192871094
    y1: 151.1510467529297
    x2: 368.7546081542969
    y2: 251.49407958984375
    x3: 272.0
    y3: 251.0

[anfederman]

This is the aruco launch filefor raspicam on a Magni robot:

----- If it works,in rqt_image_view you should see both camera and fiducial image topics. On Thu, Dec 9, 2021 at 10:56 AM roaz1801 ***@***.***> wrote: > Also, I have now tested using only a USB camera not connected to anything, > so I don't think this issue has anything to do with the turtlebot at all, > might just be that the aruco_detect node is not writing to the right topic. > From the /fiducial_vertices I get: > header: > seq: 6092 > stamp: > secs: 1639076096 > nsecs: 909698461 > frame_id: '' > image_seq: 6093 > fiducials: > > fiducial_id: 0 > direction: 0 > x0: 499.4951171875 > y0: 222.32830810546875 > x1: 552.80908203125 > y1: 223.65765380859375 > x2: 549.2293701171875 > y2: 277.81787109375 > x3: 495.5509033203125 > y3: 274.6446228027344 > > ------------------------------ > > And in the rqt image viewer the marker is being detected. > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > , > or unsubscribe > > . > Triage notifications on the go with GitHub Mobile for iOS > > or Android > . > >

[roaz1801]

Is there no way of making this work without a raspicam/ raspicam_node? Would like to try it it with a USB camera. Also tried with this part:

  <!-- Fiducial detection -->
  <include file="$(find aruco_detect)/launch/aruco_detect.launch">
    <arg name="camera" value="/raspicam_node"/>
    <arg name="image" value="image"/>
    <arg name="transport" value="compressed"/>

but changed /raspicam_node to /cv_camera, didn't work though because the

    <remap from="/camera/compressed" to="/cv_camera/image_raw/compressed"/>
    <remap from="/camera_info" to="/cv_camera/camera_info"/>

are needed

[anfederman]

I had it working with an Xtion (3d camera, using the camera_rgb image (this is via USB)in both Kinetic and Noetic. I am not positive, but I think I had it work both on a laptop and a Jetson. I think you will need to have a calibrated camera however.

On Thu, Dec 9, 2021 at 12:03 PM roaz1801 @.***> wrote:

Is there no way of making this work without a raspicam/ raspicam_node? Would like to use it it with a USB camera

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/UbiquityRobotics/fiducials/issues/263#issuecomment-990200468, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHZ377WD2Y3WSUSBYCFJIDUQEDRBANCNFSM5JWY2EMA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[anfederman]

in the launch file I sent

change the camera by the include file to find "usb_cam" (or whatever the camera node is.

in the fiducial detection section change the camera value.

On Thu, Dec 9, 2021 at 12:47 PM Alan Federman @.***> wrote:

I had it working with an Xtion (3d camera, using the camera_rgb image (this is via USB)in both Kinetic and Noetic. I am not positive, but I think I had it work both on a laptop and a Jetson. I think you will need to have a calibrated camera however.

On Thu, Dec 9, 2021 at 12:03 PM roaz1801 @.***> wrote:

Is there no way of making this work without a raspicam/ raspicam_node? Would like to use it it with a USB camera

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/UbiquityRobotics/fiducials/issues/263#issuecomment-990200468, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHZ377WD2Y3WSUSBYCFJIDUQEDRBANCNFSM5JWY2EMA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[roaz1801]

I've got the launch code working, still no luck with the transform though. Here is the working launch code with the usb camera:

<launch>
  <arg name="mapping_mode" default="false"/>
  <arg name="fiducial_len" default="0.15"/>

  <!-- camera -->
  <node pkg = "cv_camera" type="cv_camera_node" name="cv_camera" output="screen">
   <param name="image_width" value="640"/>
   <param name="image_heigth" value="480"/>
   </node>

  <!-- Fiducial detection -->
  <include file="$(find aruco_detect)/launch/aruco_detect.launch">
    <arg name="camera" value="cv_camera"/>
    <arg name="image" value="image_raw"/>
    <arg name="transport" value="compressed"/>
    <arg name="dictionary" value="0"/>

    <!-- fiducial_len -->
    <arg name="fiducial_len" value="$(arg fiducial_len)" />
  </include>

    <!-- Fiducial slam -->
  <include file="$(find fiducial_slam)/launch/fiducial_slam.launch">
    <arg name="map_frame" value="map"/>
    <arg name="odom_frame" value="odom"/>
    <arg name="base_frame" value="base_footprint"/>
    <arg name="future_date_transforms" value="0.5"/>
  </include>

  </launch>

but the fiducial transform still does not show up in the tf tree

EDIT: It still seems weird to me that the /fiducial_vertices has an empty frame_id, because it is supposed to get the frame id from the camera_info, from what I understand, and doing a echo on /cv_camera/camera_info gives the frame_id = "/camera" as shown here:

header: 
  seq: 9886
  stamp: 
    secs: 1639087170
    nsecs: 640330259
  frame_id: "/camera"
height: 480
width: 640
distortion_model: ''
D: []
K: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
R: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
P: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
binning_x: 0
binning_y: 0
roi: 
  x_offset: 0
  y_offset: 0
  height: 0
  width: 0
  do_rectify: False

but the /fidcual_vertice still gets:

header: 
  seq: 6092
  stamp: 
    secs: 1639076096
    nsecs: 909698461
  frame_id: ''
image_seq: 6093
fiducials: 
  - 
    fiducial_id: 0
    direction: 0
    x0: 499.4951171875
    y0: 222.32830810546875
    x1: 552.80908203125
    y1: 223.65765380859375
    x2: 549.2293701171875
    y2: 277.81787109375
    x3: 495.5509033203125
    y3: 274.6446228027344
---

[anfederman]

So rostopic list

Rostopic echo / camera image topic / "Fiducial_image topic'

See if it works, then view it with rqt_image_view.

If you can list the topic, but not see when echoing, the issue is networking, most likely how hostnames and IPs are specified.

On Thu, Dec 9, 2021, 1:26 PM roaz1801 @.***> wrote:

This is what I've done with the launch code. The cv_camera has no launch file so the node is directly put in, and the camera value has been changed. Not sure if this is what the camera value should be changed into, but I don't know what else it could be.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/UbiquityRobotics/fiducials/issues/263#issuecomment-990275749, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHZ37YWLMHMSQEXMPCEZLTUQENGRANCNFSM5JWY2EMA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[anfederman]

Well there is a cv launch file, it probably is sitting in /opt/ros/noetic/share.

Again, is raspistill working? Since you are getting marker vertices, it seems the data is there and working fine, but you can't view it?

Of this still is an issue we can set up a zoom call sometime tomorrow, and we can share screens, I will try to duplicate your issue.

On Thu, Dec 9, 2021, 1:26 PM roaz1801 @.***> wrote:

This is what I've done with the launch code. The cv_camera has no launch file so the node is directly put in, and the camera value has been changed. Not sure if this is what the camera value should be changed into, but I don't know what else it could be.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/UbiquityRobotics/fiducials/issues/263#issuecomment-990275749, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHZ37YWLMHMSQEXMPCEZLTUQENGRANCNFSM5JWY2EMA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[roaz1801]

Alright, let me give all the info I can: rostopic list:

/aruco_detect/compressed/parameter_descriptions
/aruco_detect/compressed/parameter_updates
/aruco_detect/parameter_descriptions
/aruco_detect/parameter_updates
/cv_camera/camera_info
/cv_camera/image_raw
/cv_camera/image_raw/compressed
/cv_camera/image_raw/compressed/parameter_descriptions
/cv_camera/image_raw/compressed/parameter_updates
/cv_camera/image_raw/compressedDepth
/cv_camera/image_raw/compressedDepth/parameter_descriptions
/cv_camera/image_raw/compressedDepth/parameter_updates
/cv_camera/image_raw/theora
/cv_camera/image_raw/theora/parameter_descriptions
/cv_camera/image_raw/theora/parameter_updates
/fiducial_images
/fiducial_images/compressed
/fiducial_images/compressed/parameter_descriptions
/fiducial_images/compressed/parameter_updates
/fiducial_images/compressedDepth
/fiducial_images/compressedDepth/parameter_descriptions
/fiducial_images/compressedDepth/parameter_updates
/fiducial_images/theora
/fiducial_images/theora/parameter_descriptions
/fiducial_images/theora/parameter_updates
/fiducial_map
/fiducial_pose
/fiducial_slam/camera_pose
/fiducial_transforms
/fiducial_vertices
/fiducials
/ignore_fiducials
/rosout
/rosout_agg
/tf
/tf_static

then rostopic echo the /fiducial_images, something is echoed out (big arrays of numbers). rqt_image_view shows /fiducial_image. The marker is even being detected, as can be seen here:

The issue comes now, as the problem is that the /fiducial_transforms is not working as it should. Even though the marker is being detected, and the vertices are being detected, the pose of the marker in relation to the camera is not coming out, as can be seen on the image too in the tf_tree

And yeah, we can take a Zoom call, but I can give you the steps I've taken so you can replicate this as it should be pretty simple:

1. Get USB camera and connect it to the PC or to the robot (at this point it does not really matter since it has been established that the problem is between the aruco_detect package and the camera). Install all necessary packages like cv_camera, compressed_image_transport, and so on.
2. Create a launch file that launches the cv_camera node and the aruco_detect node, for example the launch file in my previous post. Point the camera at the aruco code (very important that it is the same one as I'm using, as dictionary is set to 0 so it will only recognize that pattern, can be found here https://chev.me/arucogen/).
3. Open RQT and rqt_image_view, and select the /fiducial_images from the camera option. This should show that the aruco marker is being detected, the rqt_tf_tree can also be opened. Usually when the aruco code is detected there should be a tf frame from the camera to the fiducial, but it won't be there because that's what the issue is here.

After this the different topics can be echoed and checked, and what will become apparent is that the aruco_detect package does not manage to get the frame_id of the camera from the camera_info topic for some reason

Also: Right now this is being tested with only a USB camera, not the actual turtlebot/raspberry pi because I don't have access to it right now

[anfederman]

Ok so the issue is ROS doesn't know where the camera is in relation to the robot. This is usually fixed by linking the camera to the base_link or base_fpotprint of the robot. In RVIZ when you look at the topics being displayed you will see things flagged as red with warning messages about can't display because no transform exists.. blah blah.

This is fixed by adding a static transform in the launch, parameters typically in robot bringup or description.

If you download Ubiquity Robotics GitHub for Magni robot, you'll see where we can specify different positions for the camera so fiducial SLAM works. I believe the robot.yaml, bring up and description sections all have pieces of the needed code.

PS everything about robotics is hard. There is no learning curve for ROS. There is a learning cliff. Hold your nose and jump.

On Thu, Dec 9, 2021, 2:19 PM roaz1801 @.***> wrote:

Alright, let me give all the info I can: rostopic list:

/aruco_detect/compressed/parameter_descriptions /aruco_detect/compressed/parameter_updates /aruco_detect/parameter_descriptions /aruco_detect/parameter_updates /cv_camera/camera_info /cv_camera/image_raw /cv_camera/image_raw/compressed /cv_camera/image_raw/compressed/parameter_descriptions /cv_camera/image_raw/compressed/parameter_updates /cv_camera/image_raw/compressedDepth /cv_camera/image_raw/compressedDepth/parameter_descriptions /cv_camera/image_raw/compressedDepth/parameter_updates /cv_camera/image_raw/theora /cv_camera/image_raw/theora/parameter_descriptions /cv_camera/image_raw/theora/parameter_updates /fiducial_images /fiducial_images/compressed /fiducial_images/compressed/parameter_descriptions /fiducial_images/compressed/parameter_updates /fiducial_images/compressedDepth /fiducial_images/compressedDepth/parameter_descriptions /fiducial_images/compressedDepth/parameter_updates /fiducial_images/theora /fiducial_images/theora/parameter_descriptions /fiducial_images/theora/parameter_updates /fiducial_map /fiducial_pose /fiducial_slam/camera_pose /fiducial_transforms /fiducial_vertices /fiducials /ignore_fiducials /rosout /rosout_agg /tf /tf_static

then rostopic echo the /fiducial_images, something is echoed out (big arrays of numbers). rqt_image_view shows /fiducial_image. The marker is even being detected, as can be seen here: [image: image] https://user-images.githubusercontent.com/54395470/145484480-db3343bd-c8f2-4356-ba81-f6b621d3644b.png

The issue comes now, as the problem is that the /fiducial_transforms is not working as it should. Even though the marker is being detected, and the vertices are being detected, the pose of the marker in relation to the camera is not coming out, as can be seen on the image too in the tf_tree

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/UbiquityRobotics/fiducials/issues/263#issuecomment-990343151, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHZ372WF6LCWI3K7VDAO5TUQETQNANCNFSM5JWY2EMA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[roaz1801]

I did have this line in the first launch file: <node pkg="tf" type="static_transform_publisher" name="camera_frames_pub" args="0.05 0.0 0.1 0 0 0 tb3_0/base_link /camera 35"/> So I don't think it's a problem with the camera not knowing where it is in relation to the robot, I still think there's something that's making it so the aruco_detect does not manage to get the frame_id from the cv_camera/camera_info topic.

But anyway, I'll look more into it tomorrow. Thank you for your time!

[anfederman]

Fiducial SLAM works by assuming markers are fixed, and the camera is attached to the robot. For it to work, the position of the camera and how it is pointed, relative to the base_link of the robot is used to compute the motion of the robot based on transforms to odom, cmd_vel, and map topics. This further gets confused when we add scan topics from lasers and obstacle avoidance from sonar.

The entire reason Ubiquity Robotics developed fiducial SLAM was it was inexpensive and Lidar's 10 years ago were twice the cost of the robot. Now there are decent time of flight lidar packages for $130, and 3d cameras for similar prices, we are moving away from fiducial SLAM, and using cv cameras for informational navigation markers.

On Thu, Dec 9, 2021, 2:49 PM roaz1801 @.***> wrote:

I did have this line in the first launch file: <node pkg="tf" type="static_transform_publisher" name="camera_frames_pub" args="0.05 0.0 0.1 0 0 0 tb3_0/base_link /camera 35"/> So I don't think it's a problem with the camera not knowing where it is in relation to the robot, I still think there's something that's making it so the aruco_detect does not manage to get the frame_id from the cv_camera/camera_info topic.

But anyway, I'll look more into it tomorrow. Thank you for your time!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/UbiquityRobotics/fiducials/issues/263#issuecomment-990371710, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHZ37YZSHDVZB5ITJQI7UDUQEW6BANCNFSM5JWY2EMA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[roaz1801]

I had it working with an Xtion (3d camera, using the camera_rgb image (this is via USB)in both Kinetic and Noetic. I am not positive, but I think I had it work both on a laptop and a Jetson. I think you will need to have a calibrated camera however. … On Thu, Dec 9, 2021 at 12:03 PM roaz1801 @.***> wrote: Is there no way of making this work without a raspicam/ raspicam_node? Would like to use it it with a USB camera — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#263 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHZ377WD2Y3WSUSBYCFJIDUQEDRBANCNFSM5JWY2EMA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

This seems to be where the issue was. My camera was not calibrated and I managed to calibrate it today. After having done this, the /fiducial_transform works as it should. Thank you for mentioning that.