The Hyper-parameters for training the DinoV2+BoQ?

[Frost24K]

Hi, Great work! Can you provide the concrete information in training BoQ with DinoV2 backbone? Such as the learning rate, scheduler options, train epochs, etc. Thank you!

[amaralibey]

Hi @Frost24K,

The model we shared has been trained on GSV-Cities with:

• backbone: dinov2_vitb14 (unfreeze last 2 blocks)
• batch size: 160x4
• image size: 280x280
• lr: 0.0002
• optimizer: AdamW
• warmup: 10 epochs (linear warmup)
• lr_schedul: x0.1 every 10 epochs
• total epochs: 40 (converges way before, between 20-30).
• data augmentation: RandAugment(num_ops=3)

The VRAM usage when employing batches of size 160x4 is ~31GB. However, I've got similar performance using batch size of 100x4, which requieres ~19GB of VRAM.

Best, Amar

[Frost24K]

Hi @amaralibey, Thank you for your prompt response !