search

lightly-ai / lightly

A python library for self-supervised learning on images.
https://docs.lightly.ai/self-supervised-learning/
MIT License
3.1k stars 270 forks source link

Integrate Lightly with Hugging Face #1672

Open NielsRogge opened 1 day ago

NielsRogge commented 1 day ago

Hi,

Niels here from the community science team at Hugging Face 🤗 . I noticed your work as it was trending on paperswithcode, and saw you currently host all checkpoints on your own servers, e.g. https://lightly-ssl-checkpoints.s3.amazonaws.com/imagenet_resnet50_barlowtwins_2023-08-18_00-11-03/pretrain/version_0/checkpoints/epoch%3D99-step%3D500400.ckpt.

Did you know that hosting on HF is free, and would allow for better discoverability of your work?

If you're interested, leaving a guide here on how Lightly could be integrated with the hub: https://huggingface.co/docs/hub/en/models-adding-libraries. Basically, it comes down to automated tagging of models which get pushed to the hub (if a user uses lightly for training, we could integrate a push_to_hub method which then automatically tags the corresponding model in the model card with library_name: lightly).

We've done a similar thing in the past with libraries like Sentence Transformers, Timm, OpenCLIP, Spacy, and so on.

Let us know if you're interested, we're happy to assist the HF integration.

Kind regards,

Niels

guarin commented 1 day ago

Hi, thanks for opening the issue!

Do I understand correctly that the model code and checkpoint would be hosted on HF? We have checkpoints but the corresponding model code cannot be imported from the package as the models are benchmark implementations and not distributed with the package.

NielsRogge commented 1 day ago

Hi,

The model code would still live in your Github repository. e.g. OpenCLIP is an example: https://github.com/mlfoundations/open_clip. The code lives on Github, but they integrate with the hub here for instance.

We also do have a feature where code can live on the hub, but that's mainly for people who want to make their models compatible with the Transformers library (guide for that is here).

guarin commented 10 hours ago

Thanks for the info! We'll check internally if this is an option for us :)