Having a Database of different objects with corresponding names?

[WayoOchoa]

Hi,

I want to ask if there is a way or if someone knows and extension of this package that will let you create a database with multiple images and multiple objects; that lets you give the respective name of the object being recognized. What I need in specific is to get the TF of the object respect to my robot with the respective ID/name of this object; and what I have understand so far is that the database gives the same name to all the images (e.g. "object_2" or "object_30"); and i need to be able to recognize if it is a "bottle" or a "can" for example.

If someone knows a way please let me know.

Thanks.

[matlabbe]

If you are using the TCP connection, the json sent contains the full path of the object file. If you store objects in sub directories, you could parse the path to get the directory, which would be the object kind:

~/objects/can/1.png
~/objects/can/2.png
~/objects/bottle/3.png
~/objects/bottle/4.png

With ROS, you would have to modify ObjectsStamped.msg with a string containing the file path:

Header header
std_msgs/Float32MultiArray objects 
string[] filePaths

In the code, you may fill the msg paths in this loop.

QMultiMap<int, QString>::const_iterator iterPaths=info.objDetectedFilePaths_.constBegin();
for(...)
{
   ...
   msgStamped.filePaths.push_back(iterPaths.value())
   ++iterPaths;
}

[WayoOchoa]

If you are using the TCP connection, the json sent contains the full path of the object file. If you store objects in sub directories, you could parse the path to get the directory, which would be the object kind:

~/objects/can/1.png
~/objects/can/2.png
~/objects/bottle/3.png
~/objects/bottle/4.png

With ROS, you would have to modify ObjectsStamped.msg with a string containing the file path:

Header header
std_msgs/Float32MultiArray objects 
string[] filePaths

In the code, you may fill the msg paths in this loop.

QMultiMap<int, QString>::const_iterator iterPaths=info.objDetectedFilePaths_.constBegin();
for(...)
{
   ...
   msgStamped.filePaths.push_back(iterPaths.value())
   ++iterPaths;
}

Hi @matlabbe , just as a question. In ROS is it ok just to modify this portion of the code or there is another one that it has to be modify? And for the filePaths variable added to the message where how do i give it to the node? as a rosparam? does this path is a list of directories or just the main directory that contains the objects sessions?

[matlabbe]

While trying to test an example, I finally updated the code with a new message topic (find_object_2d/DetectionInfo), which extends the (now old) objectsStamped message with more info. There is a field filePaths, which will contain a valid path if the objects were loaded by setting objects_path parameter from find_object_2d node.

cheers, Mathieu

[ican9595]

Hi @matlabbe,

I have a session I saved which contains different objects for instance a can and a bottle. I launch the find_object_3d file with the session path and it detects the objects correctly.

But as @WayoOchoa said, it detects them as "Object1" etc. How can I change it to "Can" or "Bottle" or any other custom name? I dont quit understand your solution with the DetectionInfo.msg and the filePaths.

Hope you can help me out. Thanks in Advance!

[matlabbe]

Hi,

If find-object loaded objects from a directory of images, they have a all a unique path. That path is published when the object is detected, so by parsing the filepath, it could be possible to know if it is a "Can" or "Bottle". To see "Can" instead of "Object1" in TF, you would have to modify the following line to use the last part of the filepath (which would be the filename) from the info object: https://github.com/introlab/find-object/blob/856d7f0d62c4fe5a7606a2ce6a0f5c9dd6fd80fa/src/ros/FindObjectROS.cpp#L118

[ican9595]

Hi @matlabbe,

thanks for the quick response. How exactly do I "parse the filepath"? I created a folder called "images" which contains "can.png" and "bottle.png". I set the "objects_path" to this folder. When I launch my file I get the preloaded images and detect them, but the output is still "Object1 is detected" instead of "Can is detected"

This is how my launch file looks: `

<!-- $roslaunch openni_launch openni.launch depth_registration:=true -->

<arg name="object_prefix" default="object"/>
<arg name="target_frame_id" default="/base_link"/>
<arg name="objects_path"  default="$(find find_object_2d)/images/"/>
<arg name="gui"           default="true"/>
<arg name="approx_sync"   default="true"/>
<arg name="pnp"           default="true"/>
<arg name="tf_example"    default="true"/>
<arg name="settings_path" default="~/.ros/find_object_2d.ini"/>

<arg name="rgb_topic"         default="/realsense/color/image_raw"/>
<arg name="depth_topic"       default="/realsense/depth/image_rect_raw"/>
<arg name="camera_info_topic" default="/realsense/depth/camera_info"/>

<node name="find_object_3d" pkg="find_object_2d" type="find_object_2d" output="screen">
    <param name="gui" value="$(arg gui)" type="bool"/>
    <param name="settings_path" value="$(arg settings_path)" type="str"/>
    <param name="subscribe_depth" value="true" type="bool"/>
    <!--param name="session_path" value="$(find find_object_2d)/sessions/session1.bin" type="str"/-->
    <param name="objects_path" value="$(arg objects_path)" type="str"/>
    <param name="object_prefix" value="$(arg object_prefix)" type="str"/>
    <param name="approx_sync" value="$(arg approx_sync)" type="bool"/>
    <param name="pnp" value="$(arg pnp)" type="bool"/>

    <remap from="rgb/image_rect_color" to="$(arg rgb_topic)"/>
    <remap from="depth_registered/image_raw" to="$(arg depth_topic)"/>
    <remap from="depth_registered/camera_info" to="$(arg camera_info_topic)"/>
</node>

<!-- Example of tf synchronisation with the objectsStamped message -->
<node if="$(arg tf_example)" name="tf_example" pkg="find_object_2d" type="tf_example" output="screen">
    <param name="target_frame_id" value="$(arg target_frame_id)" type="str"/>
    <param name="object_prefix" value="$(arg object_prefix)" type="str"/>
</node>

`

[ican9595]

I see now. So its only possible to get the name of the object through the filePath when I do rostopic echo /info:

filePaths: data:"/home/ican/my_catkin_ws/src/find-object/objects/bottle1.png" data:"/home/ican/my_catkin_ws/src/find-object/objects/can2.png"

My intention was, when I launch the find_object_3d.launch file and get the output from the tf_example like this:

[ INFO] [1621281876.600972427, 158.594000000]: object_12 [x,y,z] [x,y,z,w] in "/base_link" frame: [0.804474,0.201849,0.511531] [-0.356714,0.000539,-0.695519,0.623705]

to get the 6D-Pose (which works fine) and instead of "object_12" I wanted the actual name of the object like "bottle" or "can"

@matlabbe Is there a way to do this?

Thank you!

[matlabbe]

For https://github.com/introlab/find-object/blob/master/src/ros/tf_example_node.cpp, this is an example of usage. This example can be modified. It could subscribe to DetectionInfo message instead of objectsStampted to have the map between ObjectId and filepath, so you could print the file name instead.