Suggested ways to fine-tuning for Focal-Dino (Dino based) , thanks.

Hi Sir, thanks for your great work~

For the Focal-Dino (Dino based) releasing , is it possible to fine-tune Focal-Dino(with custom-dataset) with these latest checkpoints as backbone/pretrained as well ?

EX : o365_ckpt(pretrained on o365) as Backbone, and coco_ckpt(Fine-tune on coco) as pretrain

[12/16/2022] 💥 We are pleased to release our [FocalNet-Large-DINO checkpoint](https://github.com/FocalNet/FocalNet-DINO#model-zoos) pretrained on Object365 and finetuned on COCO, which help to get 63.5 mAP without tta on COCO minival! Check it out

Seems that some RuntimeError from 'loading state_dict for DINO' for our side so far (ignore as Dino ?), and any kindly suggestion/advice will be great help for us, thanks in advance.

Traceback (most recent call last):
  File "main.py", line 398, in <module>
    main(args)
  File "main.py", line 251, in main
    _load_output = model_without_ddp.load_state_dict(_tmp_st, strict=False)
  File "/home/jovyan/.conda/envs/detrex_env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1604, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for DINO:
        size mismatch for transformer.level_embed: copying a param with shape torch.Size([5, 256]) from checkpoint, the shape in current model is torch.Size([4, 256])...

microsoft / FocalNet

Suggested ways to fine-tuning for Focal-Dino (Dino based) , thanks. #18