In the current state, SMIT. will save the whole model (encoder + projector + decoder) during the pretraining. As only the linear_projector is trained during pre-training, this unnecessarily consumes disk space.
During the pretraining stage, the Trainer should only saves the linear_projector in order to limit disk usage.
In the current state, SMIT. will save the whole model (encoder + projector + decoder) during the pretraining. As only the
linear_projectoris trained during pre-training, this unnecessarily consumes disk space.During the pretraining stage, the
Trainershould only saves thelinear_projectorin order to limit disk usage.