Open Mediumcore opened 11 months ago
Hi @Mediumcore ,
This would require distilling a new model, for this you may be able to follow these steps.
Disclaimer: I haven't tested these, so let me know if you run into issues.
Register a new model, and ensure that it outputs features of shape 256x64x64 for your desired input resolution. For example, for an input of size 512x512, your model must have an output stride of 8.
You can register the model similar to here:
You could try registering a new model with a different stride, and setting the student size to a lower resolution (ie: 512x512).
Next, you'll need to train the model on unlabeled images. Follow the training instructions in the README, but set the "student_size" parameter to the desired size (512).
Follow the evaluation instructions in the README to compare the accuracy for small / medium / large objects.
As a note: It's worth noting that distillation only applies to the image encoder. It's worth benchmarking the mask decoder to see if this is worth it, as the image encoding speed is approaching the decoding speed and may no longer be a performance bottleneck.
Hope this helps. If we end up releasing a lower resolution model I will update this thread, but we have no current plans at the moment. John
Understood,thank you very much for reply
Register a new model, and ensure that it outputs features of shape 256x64x64 for your desired input resolution. For example, for an input of size 512x512, your model must have an output stride of 8.
You can register the model similar to here:
You could try registering a new model with a different stride, and setting the student size to a lower resolution (ie: 512x512).
Hey there, could I just check which particular layer of convolution the output stride of 8 should be applied to? Also, where is the setting for the student size resolution? I am trying to get a model with an input size of 512x512.
image encoder image size is too big ,can reduce 1024 to 640/480 for acceleration?