What is difference between FrozenBackboneEncoderDecoder and EncoderDecoder

[seabearlmx]

Hello, the Rein is trained on the frozen dinov2 backbones, but the Rein likely uses EncoderDecoder. so, what is the difference when I add the code model = dict(type="FrozenBackboneEncoderDecoder") to the file rein_dinov2_mask2former_512x512_bs1x4.py

[w1oves]

FrozenBackboneEncoderDecoder freezes the entire backbone (including the rein) and disables gradient backpropagation throughout the backbone. Here, Backbone operates under torch.no_grad(), ensuring a very fast execution due to reduced computational overhead.

During the training of Rein, only the pretrained backbone is frozen, allowing for the fine-tuning of Rein. This selective freezing is implemented through the train() function of the backbone coupled with the PEFTOptimizerConstructor, and it is distinct from the functionality of FrozenBackboneEncoderDecoder.

[w1oves]

Rein will not be fine-tuned if you add add the code model = dict(type="FrozenBackboneEncoderDecoder") to the file rein_dinov2_mask2former_512x512_bs1x4.py.