There are some issues while running the code

[xiaoxia0722]

Hello, I'm having some problems running Rein's code. Because this is my first time coming into contact with this direction, there may be errors in my operation. Please try to solve the following problems:

1. After the configuration is completed according to the readme.md, the error shows that checkpoints/dinov2_converted.pth does not exist after running the training command python tools/train.py configs/dinov2/rein_dinov2_mask2former_512x512_bs1x4.py. There are only dinov2_rein_and_head.pth and dinov2_vitl14_pretrain.pth in the checkpoint file. How to download dinov2_converted.pth?
2. I found that renaming dinov2_rein_and_head.pth to dinov2_converted.pth worked, but the results were not ideal.
3. Where do I download the backbone checkpoints for releases? I didn't find the corresponding file in dinov2's github. There are many models in it.
4. Is the 10k semantic segmentation dataset used in bdd100k the same data in bdd100k_seg.zip? The download from the official website is not available. If not, can you provide a download link to the dataset?

[w1oves]

• A1: You can generate dinov2_converted.pth by running the command: python tools/convert_models/convert_dinov2_large_512x512.py checkpoints/dinov2_vitl14_pretrain.pth.

• A2: This does not work.

• A3: You can obtain the checkpoint for DINOv2-large from https://dl.fbaipublicfiles.com/dinov2/dinov2_vitl14/dinov2_vitl14_pretrain.pth. More backbones can be found according to the model from https://github.com/facebookresearch/dinov2/tree/main.

• A4: Yes, we use the segmentation data from the 10K part of BDD100K. The official website of BDD100K seems to be down. You might be able to obtain these data from https://opendatalab.com/OpenDataLab/BDD100K, but I'm not certain.

[xiaoxia0722]

Do other backbone networks also use command python tools/convert_models/convert_dinov2_large_512x512.py checkpoint/backbone for conversion? I get error RuntimeError: shape '[1, 37, 37, 1024]' is invalid for input of size 525696 when I run the dinov2_vits14_pretrain.pth model transformation using command python tools/convert_models/convert_dinov2_large_512x512.py checkpoint/dinov2_vits14_pretrain.pth. And the same problem is encountered when transforming other pretain models. The download address is https://dl.fbaipublicfiles.com/dinov2/dinov2_vits14/dinov2_vits14_pretrain.pth

[xiaoxia0722]

When I used config file GTAV+ConvNeXt-L, GTAV+ResNet101 and GTAV+ResNet50 from tags, running it showed that model ReinsConvNeXt and ReinsResNetV1c didn't exist, and I couldn't find them in the code.

[w1oves]

Do other backbone networks also use command python tools/convert_models/convert_dinov2_large_512x512.py checkpoint/backbone for conversion? I get error RuntimeError: shape '[1, 37, 37, 1024]' is invalid for input of size 525696 when I run the dinov2_vits14_pretrain.pth model transformation using command python tools/convert_models/convert_dinov2_large_512x512.py checkpoint/dinov2_vits14_pretrain.pth. And the same problem is encountered when transforming other pretain models. The download address is https://dl.fbaipublicfiles.com/dinov2/dinov2_vits14/dinov2_vits14_pretrain.pth

As the name suggests, convert_dinov2_large_512x512.py is developed for dinov2_large. You can easily adapt it for different backbones by copying and making simple edits, mainly by changing the number of channels from 1024 to the corresponding number of channels in lines 15 to 29.

[w1oves]

When I used config file GTAV+ConvNeXt-L, GTAV+ResNet101 and GTAV+ResNet50 from tags, running it showed that model ReinsConvNeXt and ReinsResNetV1c didn't exist, and I couldn't find them in the code.

It has been updated! Thank you for point out!

[xiaoxia0722]

Thank you for your answer. The code has run successfully

[w1oves]

good lock to you