Is there a plan to release the dit pretraining code publicly? I am very interested in it.

microsoft / unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

https://aka.ms/GeneralAI

MIT License

19.12k stars 2.44k forks source link

Is there a plan to release the dit pretraining code publicly? I am very interested in it. #1139

Open Masterchenyong opened 1 year ago

Masterchenyong commented 1 year ago

Describe Model I am using (UniLM, MiniLM, LayoutLM ...):

senthil-r-10 commented 10 months ago

DIT and BEIT encoders are the same, except the image tokenizer is fine-tuned for the specified dataset. I have tried it but the model is not converging. you can check the issue #1268

https://github.com/microsoft/unilm/issues/1268