search

huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
132.77k stars 26.46k forks source link

Adding TF Implementation of BEiT #18085

Open MadElf1337 opened 2 years ago

MadElf1337 commented 2 years ago

Feature request

Addition of TF implementation of BEiT

Motivation

I have always seen that there is a discrepancy in the availability of models for PyTorch and the models available in TensorFlow, and want to have models for usage in both backends.

Your contribution

I will add the implementation of BEiT in TF :)

cc - @gante

LysandreJik commented 2 years ago

cc @NielsRogge @amyeroberts

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

amyeroberts commented 2 years ago

Hi @MadElf1337 do you have any updates? Are you still planning on contributing this model?

MadElf1337 commented 2 years ago

Yep I’m still working on the model, had to keep it aside for a bit due to my uni exam schedule, but will start again the day my exams are over

Regarding the updates, I am done with the architecture, have to write the functions for specific purposes(like segmentation) and the tests

amyeroberts commented 2 years ago

Great - glad to hear you're still interested :)

As @NielsRogge pointed out, data2vec vision is an extension of BEiT. This means the porting should be a lot simpler! In our pytorch BEiT implementation, you can see this from the #Copied from statements. Ideally the TF implementation would reflect this and be the same as our pytorch implementation, however TF data2vec vision is already implemented. So, we need to move the data2vec code to beit, and then add the necessary #Copied from statement in data2vec. Does this make sense?

Could you open a draft PR for the model please so that the code is visible?

Good luck with the last of your exams!

MadElf1337 commented 2 years ago

Yes I’ll open a draft PR to show the code that’s been done till date

And thanks!