Error in CSAIL_ResNet50ppm_ADE. yaml?

crankler commented 5 months ago

Sorry to bother you aggain, thanks again for your work here. When I try to use MIT Resnet, the error shows as follows:

[sel_map/static_tf_linker-5] process has finished cleanly
log file: /home/yc/.ros/log/26ff1b50-e363-11ee-9c22-3b683d376962/sel_map-static_tf_linker-5*.log
Loading weights for net_encoder
Loading weights for net_decoder
Traceback (most recent call last):
  File "/home/yc/catkin_map/catkin_michgen/src/sel_map/sel_map/scripts/main_kinect.py", line 243, in <module>
    sel_map_node(mesh_bounds, elem_size, threshold)
  File "/home/yc/catkin_map/catkin_michgen/src/sel_map/sel_map/scripts/main_kinect.py", line 201, in sel_map_node
    segmentation_network = CameraSensor()
  File "/home/yc/catkin_map/catkin_michgen/src/sel_map/sel_map_segmentation/sel_map_segmentation/src/sel_map_segmentation/cameraSensor.py", line 46, in __init__
    self.network = SemsegNetworkWrapper(model=model_name, args=args)
  File "/home/yc/catkin_map/catkin_michgen/src/sel_map/sel_map_segmentation/mit_semseg_wrapper/src/mit_semseg_wrapper/semsegNetwork.py", line 77, in __init__
    net_decoder = ModelBuilder.build_decoder(
  File "/home/yc/catkin_map/catkin_michgen/src/sel_map/sel_map_segmentation/mit_semseg_wrapper/src/mit_semseg/models/models.py", line 155, in build_decoder
    net_decoder.load_state_dict(
  File "/home/yc/.conda/envs/selmap/lib/python3.8/site-packages/torch/nn/modules/module.py", line 2152, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for PPMDeepsup:
    size mismatch for conv_last.1.weight: copying a param with shape torch.Size([150, 512, 1, 1]) from checkpoint, the shape in current model is torch.Size([512]).
    size mismatch for conv_last.1.bias: copying a param with shape torch.Size([150]) from checkpoint, the shape in current model is torch.Size([512]).
[sel_map/sel_map-6] process has died [pid 744608, exit code 1, cmd /home/yc/catkin_map/catkin_michgen/src/sel_map/sel_map/scripts/main_kinect.py 4 4 2.5 0.05 20 mesh:=/mesh mesh/costs:=/mesh/costs get_materials:=/get_materials __name:=sel_map __log:=/home/yc/.ros/log/26ff1b50-e363-11ee-9c22-3b683d376962/sel_map-sel_map-6.log].
log file: /home/yc/.ros/log/26ff1b50-e363-11ee-9c22-3b683d376962/sel_map-sel_map-6*.log

My params are as follows:

  <arg name="semseg_config"         default="CSAIL_ResNet50ppm_ADE_onehot.yaml"/>
  <arg name="terrain_properties"  default="csail_semseg_properties.yaml"/>

I tried my own bag and you provide spot bag, all shows such error. So, what I can do to fix such erro? Hope to receive your help. thank you.

BuildingAtom commented 5 months ago

Hi, I noticed in a different issue you commented on downloading the weights? Have you confirmed that the weights you are loading match the model selected from the CSAIL semseg wrapper package? Not all the models have the same architecture, and the error here suggests that it was attempting to load mismatching weights.

crankler commented 5 months ago

Yeah, I have tested and it works. Thank you for your kindly reply here. By the way, I try to combine SAMTrack or FastSAM with sel_map in these days. Should I only replace the self.runSemanticSegmentation() in getProjectedPointCloudWithLabels()? Is it true?

BuildingAtom commented 5 months ago

You can do that, but in keeping with the structure of the project, I’d recommend utilizing the segmentation _wrapper_template provided to interface with whatever semantic segmentation network you want.

You can find more information in the included README, but it’s pretty straightforward. Just copy and customize that package, and you can then write a configuration yaml like the others provided to load your network of choice and swap it out freely. https://github.com/roahmlab/sel_map/tree/main/sel_map_segmentation/segmentation_wrapper_template.

roahmlab / sel_map

Error in CSAIL_ResNet50ppm_ADE. yaml? #12