Open ArghyaChatterjee opened 1 year ago
Hi @ArghyaChatterjee, do you by any chance have a file called CATKIN_IGNORE
in the sel_map/sel_map_segmentation/pytorch_encoding_wrapper/
folder? That will tell catkin not to build the pytorch_encoding_wrapper
package, which means you can't use the pytorch encoding networks.
If that is not the case, and if you are using catkin build
instead of catkin_make
it may just be failing to build that specific package. This popped up internally at some point and an issue should have been opened, but I may have to do an update to that wrapper package to use the latest version of PyTorch-Encoding since the current committed version has issues compiling on CUDA 12. If you run into this issue, can you try updating line 33 of sel_map/sel_map_segmentation/pytorch_encoding_wrapper/CMakeLists.txt
with the hash for the latest PyTorch-Encoding commit c959dab8312b637fcc7edce83607acb4b0f82645
and let me know if it works?
It should look like this after the change:
ExternalProject_Add(
pytorch_encoding
GIT_REPOSITORY https://github.com/zhanghang1989/PyTorch-Encoding.git
PREFIX ${ENCODING_BUILD_DIR}
BINARY_DIR ${ENCODING_BUILD_DIR}/src/pytorch_encoding
GIT_TAG c959dab8312b637fcc7edce83607acb4b0f82645
CONFIGURE_COMMAND ""
BUILD_COMMAND
${PYTHON} ${ENCODING_BUILD_DIR}/src/pytorch_encoding/setup.py build --build-lib ${ENCODING_BUILD_DIR}/build
INSTALL_COMMAND
${CMAKE_COMMAND} -E copy_directory ${ENCODING_BUILD_DIR}/build/encoding ${ENCODING_INSTALL_DIR}
TEST_COMMAND ""
)
Hey, thanks for responding. Your reply worked actually but for some reason, I can't visualize the textured mesh, rather I can see the wireframe mesh. How to visualize the textured mesh in RVIZ ?
Here is the terminal output, can you see there any problem :
arghya@arghya-Pulse-GL66-12UEK:~/sel_map_ws$ roslaunch sel_map spot_bag_sel.launch semseg_config:=Encoding_ResNet50_PContext_full.yaml terrain_properties:=pascal_context.yaml
... logging to /home/arghya/.ros/log/b6bd02f4-e330-11ed-84e6-39ea0baed006/roslaunch-arghya-Pulse-GL66-12UEK-115236.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.
started roslaunch server http://127.0.0.1:36645/
SUMMARY
========
PARAMETERS
* /robot_description: <?xml version="1....
* /rosdistro: noetic
* /rosversion: 1.16.0
* /sel_map/cameras_registered/realsense/camera_info: /zed2/zed_node/de...
* /sel_map/cameras_registered/realsense/depth_registered: /zed2/zed_node/de...
* /sel_map/cameras_registered/realsense/image_rectified: /zed2/zed_node/le...
* /sel_map/cameras_registered/realsense/pose:
* /sel_map/cameras_registered/realsense/pose_with_covariance:
* /sel_map/colorscale/ends/one: 0,0,0
* /sel_map/colorscale/ends/zero: 255,255,255
* /sel_map/colorscale/stops: [0.2, 0.8]
* /sel_map/colorscale/type: linear_ends
* /sel_map/colorscale/unknown: 120,120,120
* /sel_map/colorscale/values: ['252,197,192', '...
* /sel_map/enable_mat_display: True
* /sel_map/num_cameras: 1
* /sel_map/point_limit: 20
* /sel_map/publish_rate: 20.0
* /sel_map/queue_size: 2
* /sel_map/robot_base: base_link
* /sel_map/save_classes: True
* /sel_map/save_confidence: False
* /sel_map/save_interval: 0.0
* /sel_map/save_mesh_location:
* /sel_map/semseg/extra_args: []
* /sel_map/semseg/model: Encnet_ResNet50s_...
* /sel_map/semseg/num_labels: 59
* /sel_map/semseg/onehot_projection: False
* /sel_map/semseg/ongpu_projection: True
* /sel_map/semseg/package: pytorch_encoding_...
* /sel_map/sync_slop: 0.3
* /sel_map/terrain_properties: /home/arghya/sel_...
* /sel_map/update_policy: fifo
* /sel_map/world_base: odom
* /use_sim_time: True
NODES
/
rviz (rviz/rviz)
/sel_map/
sel_map (sel_map/main.py)
static_tf_linker (sel_map_utils/StaticTFLinker.py)
auto-starting new master
process[master]: started with pid [115284]
ROS_MASTER_URI=http://127.0.0.1:11311
setting /run_id to b6bd02f4-e330-11ed-84e6-39ea0baed006
process[rosout-1]: started with pid [115313]
started core service [/rosout]
process[rviz-2]: started with pid [115320]
process[sel_map/static_tf_linker-3]: started with pid [115321]
process[sel_map/sel_map-4]: started with pid [115322]
[ INFO] [1682403388.901918492]: rviz version 1.14.20
[ INFO] [1682403388.901956624]: compiled against Qt version 5.12.8
[ INFO] [1682403388.901963892]: compiled against OGRE version 1.9.0 (Ghadamon)
[ INFO] [1682403388.909752088]: Forcing OpenGl version 0.
[ INFO] [1682403389.016917513]: Stereo is NOT SUPPORTED
[ INFO] [1682403389.016950926]: OpenGL device: Mesa Intel(R) Graphics (ADL GT2)
[ INFO] [1682403389.016960059]: OpenGl version: 4.6 (GLSL 4.6) limited to GLSL 1.4 on Mesa system.
[INFO] [1682403389.068008, 0.000000]: No static frames to spin, exiting...
[ INFO] [1682403389.254020685]: Mesh Display: Update
[ERROR] [1682403389.254095687]: Mesh display: no visual available, can't draw mesh! (maybe no data has been received yet?)
[ INFO] [1682403389.254203133]: Mesh Display: Update
[ERROR] [1682403389.254218842]: Mesh display: no visual available, can't draw mesh! (maybe no data has been received yet?)
[ INFO] [1682403389.257986500]: No initial data available, waiting for callback to trigger ...
[ INFO] [1682403389.258034535]: Mesh Display: Update
[ERROR] [1682403389.258060906]: Mesh display: no visual available, can't draw mesh! (maybe no data has been received yet?)
[ INFO] [1682403389.258096922]: Mesh Display: Update
[ERROR] [1682403389.258119573]: Mesh display: no visual available, can't draw mesh! (maybe no data has been received yet?)
[ INFO] [1682403389.258174455]: Mesh Display: Update
[ERROR] [1682403389.258197386]: Mesh display: no visual available, can't draw mesh! (maybe no data has been received yet?)
[ INFO] [1682403389.259785813]: Mesh Display: Update
[ERROR] [1682403389.259811436]: Mesh display: no visual available, can't draw mesh! (maybe no data has been received yet?)
[sel_map/static_tf_linker-3] process has finished cleanly
log file: /home/arghya/.ros/log/b6bd02f4-e330-11ed-84e6-39ea0baed006/sel_map-static_tf_linker-3*.log
[INFO] [1682403396.412495, 0.000000]: [sel_map] Callbacks registered, awaiting...
[INFO] [1682403405.665022, 1682352581.268983]: [sel_map] Message received!
/home/arghya/sel_map_ws/src/sel_map/sel_map_segmentation/sel_map_segmentation/src/sel_map_segmentation/cameraSensor.py:57: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:189.)
self.depth = torch.from_numpy(depth).float().to(self.network.device, non_blocking=True)
[INFO] [1682403405.869540, 1682352581.480928]: [sel_map] Publishing...
publish time 0.0005059160030214116
[ INFO] [1682403405.913901298, 1682352581.522340328]: Creating MeshVisual 0_TexturedMesh_0_8400
[ INFO] [1682403405.914056310, 1682352581.522340328]: Resetting MeshVisual 0_TexturedMesh_0_8400
[ INFO] [1682403405.914433902, 1682352581.522340328]: Mesh Display: Update
[ INFO] [1682403405.915071337, 1682352581.522340328]: Failed vertex colors service call!
[ INFO] [1682403405.917473417, 1682352581.522340328]: Successful materials service call!
[ INFO] [1682403405.917850448, 1682352581.522340328]: Received 59 materials.
[ INFO] [1682403405.918102701, 1682352581.522340328]: Mesh Display: Update
[ INFO] [1682403405.918703852, 1682352581.522340328]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403405.918738101, 1682352581.522340328]: The cost layer "height" has been added.
[ INFO] [1682403405.919072650, 1682352581.522340328]: Mesh Display: Update
[ INFO] [1682403405.945529013, 1682352581.552623943]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403405.945660037, 1682352581.552623943]: The cost layer "height" has been updated.
[ INFO] [1682403405.946691458, 1682352581.552623943]: Mesh Display: Update
[ INFO] [1682403405.977731268, 1682352581.584151742]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403405.977872121, 1682352581.584151742]: The cost layer "height" has been updated.
[ INFO] [1682403405.978777874, 1682352581.584151742]: Mesh Display: Update
[ INFO] [1682403406.040830043, 1682352581.644978404]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.040957980, 1682352581.644978404]: The cost layer "height" has been updated.
[ INFO] [1682403406.041741198, 1682352581.644978404]: Mesh Display: Update
[INFO] [1682403406.084003, 1682352581.685774]: [sel_map] Message received!
[ INFO] [1682403406.104273293, 1682352581.706013236]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.104301180, 1682352581.706013236]: The cost layer "height" has been updated.
[ INFO] [1682403406.104445279, 1682352581.706013236]: Mesh Display: Update
[ INFO] [1682403406.136218855, 1682352581.746346247]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.136246950, 1682352581.746346247]: The cost layer "height" has been updated.
[ INFO] [1682403406.136372623, 1682352581.746346247]: Mesh Display: Update
[ INFO] [1682403406.200877076, 1682352581.796650820]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.200926118, 1682352581.796650820]: The cost layer "height" has been updated.
[ INFO] [1682403406.201172223, 1682352581.796650820]: Mesh Display: Update
[INFO] [1682403406.224942, 1682352581.836237]: [sel_map] Publishing...
publish time 0.0006569319994014222
[ INFO] [1682403406.265059982, 1682352581.866394725]: Creating MeshVisual 0_TexturedMesh_0_3943
[ INFO] [1682403406.265213178, 1682352581.876486756]: Destroying MeshVisual 0_TexturedMesh_0_8400
[ INFO] [1682403406.265237545, 1682352581.876486756]: Resetting MeshVisual 0_TexturedMesh_0_8400
[ INFO] [1682403406.265480257, 1682352581.876486756]: Resetting MeshVisual 0_TexturedMesh_0_3943
[ INFO] [1682403406.265705083, 1682352581.876486756]: Mesh Display: Update
[ INFO] [1682403406.266287765, 1682352581.876486756]: Failed vertex colors service call!
[ INFO] [1682403406.268367316, 1682352581.876486756]: Successful materials service call!
[ INFO] [1682403406.268447197, 1682352581.876486756]: Received 59 materials.
[ INFO] [1682403406.268717404, 1682352581.876486756]: Mesh Display: Update
[ INFO] [1682403406.268815979, 1682352581.876486756]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.268826485, 1682352581.876486756]: The cost layer "height" has been updated.
[ INFO] [1682403406.268987880, 1682352581.876486756]: Mesh Display: Update
[ INFO] [1682403406.296794302, 1682352581.906660226]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.296848634, 1682352581.906660226]: The cost layer "height" has been updated.
[ INFO] [1682403406.297231680, 1682352581.906660226]: Mesh Display: Update
[ INFO] [1682403406.328916885, 1682352581.937816544]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.329037064, 1682352581.937816544]: The cost layer "height" has been updated.
[ INFO] [1682403406.329756706, 1682352581.937816544]: Mesh Display: Update
[ INFO] [1682403406.392951497, 1682352582.000850590]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.393085215, 1682352582.000850590]: The cost layer "height" has been updated.
[ INFO] [1682403406.394064526, 1682352582.000850590]: Mesh Display: Update
[ INFO] [1682403406.425201919, 1682352582.031416812]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.425333707, 1682352582.031416812]: The cost layer "height" has been updated.
[ INFO] [1682403406.426163387, 1682352582.031416812]: Mesh Display: Update
[ INFO] [1682403406.488784324, 1682352582.092415479]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.488925047, 1682352582.092415479]: The cost layer "height" has been updated.
[ INFO] [1682403406.489909584, 1682352582.092415479]: Mesh Display: Update
[ INFO] [1682403406.553501217, 1682352582.163543695]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.553636634, 1682352582.163543695]: The cost layer "height" has been updated.
[ INFO] [1682403406.554464400, 1682352582.163543695]: Mesh Display: Update
[ INFO] [1682403406.585389937, 1682352582.194181320]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.585525465, 1682352582.194181320]: The cost layer "height" has been updated.
[ INFO] [1682403406.586365420, 1682352582.194181320]: Mesh Display: Update
[INFO] [1682403406.631062, 1682352582.234921]: [sel_map] Message received!
[ INFO] [1682403406.648793944, 1682352582.255100740]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.648912613, 1682352582.255100740]: The cost layer "height" has been updated.
[ INFO] [1682403406.649413841, 1682352582.255100740]: Mesh Display: Update
[ INFO] [1682403406.680978496, 1682352582.285275807]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.681001959, 1682352582.285275807]: The cost layer "height" has been updated.
[ INFO] [1682403406.681133702, 1682352582.285275807]: Mesh Display: Update
[ INFO] [1682403406.745768909, 1682352582.355671827]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.745813269, 1682352582.355671827]: The cost layer "height" has been updated.
[ INFO] [1682403406.746032820, 1682352582.355671827]: Mesh Display: Update
[INFO] [1682403406.774576, 1682352582.385969]: [sel_map] Publishing...
publish time 0.0005915120018471498
[ INFO] [1682403406.809105907, 1682352582.416136777]: Creating MeshVisual 0_TexturedMesh_0_7829
[ INFO] [1682403406.809186267, 1682352582.416136777]: Destroying MeshVisual 0_TexturedMesh_0_3943
[ INFO] [1682403406.809202993, 1682352582.416136777]: Resetting MeshVisual 0_TexturedMesh_0_3943
[ INFO] [1682403406.809321643, 1682352582.416136777]: Resetting MeshVisual 0_TexturedMesh_0_7829
[ INFO] [1682403406.809511142, 1682352582.416136777]: Mesh Display: Update
[ INFO] [1682403406.810061498, 1682352582.416136777]: Failed vertex colors service call!
[ INFO] [1682403406.811806095, 1682352582.416136777]: Successful materials service call!
[ INFO] [1682403406.811902995, 1682352582.416136777]: Received 59 materials.
[ INFO] [1682403406.812231613, 1682352582.416136777]: Mesh Display: Update
[ INFO] [1682403406.812345749, 1682352582.416136777]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.812363009, 1682352582.416136777]: The cost layer "height" has been updated.
[ INFO] [1682403406.812587422, 1682352582.416136777]: Mesh Display: Update
[ INFO] [1682403406.840885311, 1682352582.446557463]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.840914856, 1682352582.446557463]: The cost layer "height" has been updated.
[ INFO] [1682403406.841148386, 1682352582.446557463]: Mesh Display: Update
[ INFO] [1682403406.904726768, 1682352582.506962267]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.904870352, 1682352582.506962267]: The cost layer "height" has been updated.
[ INFO] [1682403406.905820319, 1682352582.506962267]: Mesh Display: Update
[ INFO] [1682403406.937023669, 1682352582.547597042]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403406.937168422, 1682352582.547597042]: The cost layer "height" has been updated.
[ INFO] [1682403406.938105918, 1682352582.547597042]: Mesh Display: Update
[ INFO] [1682403407.001530493, 1682352582.608647405]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403407.001663950, 1682352582.608647405]: The cost layer "height" has been updated.
[ INFO] [1682403407.002562501, 1682352582.608647405]: Mesh Display: Update
[ INFO] [1682403407.033322872, 1682352582.639213706]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403407.033451679, 1682352582.639213706]: The cost layer "height" has been updated.
[ INFO] [1682403407.034358830, 1682352582.639213706]: Mesh Display: Update
[ INFO] [1682403407.097400864, 1682352582.700007925]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403407.097603878, 1682352582.700007925]: The cost layer "height" has been updated.
[ INFO] [1682403407.099106946, 1682352582.700007925]: Mesh Display: Update
[ INFO] [1682403407.129624519, 1682352582.740334523]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403407.129764209, 1682352582.740334523]: The cost layer "height" has been updated.
[ INFO] [1682403407.130759099, 1682352582.740334523]: Mesh Display: Update
[INFO] [1682403407.131221, 1682352582.730261]: [sel_map] Message received!
[ INFO] [1682403407.192381224, 1682352582.801351732]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403407.192405172, 1682352582.801351732]: The cost layer "height" has been updated.
[ INFO] [1682403407.192537201, 1682352582.801351732]: Mesh Display: Update
[ INFO] [1682403407.225015642, 1682352582.831752484]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403407.225071510, 1682352582.831752484]: The cost layer "height" has been updated.
[ INFO] [1682403407.225275669, 1682352582.831752484]: Mesh Display: Update
[INFO] [1682403407.272464, 1682352582.883867]: [sel_map] Publishing...
publish time 0.0004796370012627449
[ INFO] [1682403407.289218461, 1682352582.893922496]: Creating MeshVisual 0_TexturedMesh_0_7982
[ INFO] [1682403407.289308945, 1682352582.893922496]: Destroying MeshVisual 0_TexturedMesh_0_7829
[ INFO] [1682403407.289330295, 1682352582.893922496]: Resetting MeshVisual 0_TexturedMesh_0_7829
[ INFO] [1682403407.289479357, 1682352582.893922496]: Resetting MeshVisual 0_TexturedMesh_0_7982
[ INFO] [1682403407.289692514, 1682352582.893922496]: Mesh Display: Update
[ INFO] [1682403407.290214757, 1682352582.893922496]: Failed vertex colors service call!
[ INFO] [1682403407.294254127, 1682352582.893922496]: Successful materials service call!
[ INFO] [1682403407.294371845, 1682352582.893922496]: Received 59 materials.
[ INFO] [1682403407.294785296, 1682352582.893922496]: Mesh Display: Update
[ INFO] [1682403407.294930935, 1682352582.893922496]: Cache vertex cost map 'height' for UUID
[ INFO] [1682403407.294959697, 1682352582.893922496]: The cost layer "height" has been updated.
[ INFO] [1682403407.295231455, 1682352582.893922496]: Mesh Display: Update
Here is the output:
Great to hear! I'm a bit busy now, but I'll make sure push a fix based on that small dependency issue for PyTorch-Encoding later.
In your Mesh display, can you make sure that you're using the get_materials
materials service call and visualizing with materials? It looks like it may be attempting to get the colors from a vertex colors service, which we don't use.
Hello, thanks for your work. I was trying out your work but found some problem. I tried to run the other launch file also, but had some issue. The one without any config or terrain property flag launches with only wireframe mesh visualizable but no semantic class and when I specifically mention the config and property flag, I get the following error: N.B: I tried to install
pip3 install pytorch-encoding
and it was installed successfully but still had the same issue. Didn't change the problem / issue at all.What should I do now? Thanks in advance.