search

google-research / inksight

Apache License 2.0
598 stars 30 forks source link

Plans to release the model weights? #8

Open Vaibhavs10 opened 3 weeks ago

Vaibhavs10 commented 3 weeks ago

Hey hey, I'm VB, I work in the open source team at Hugging Face.

Congrats on the release! I was wondering if you'd release your model checkpoints.

IMO it'd be of great use for the community and fantastic research artefact too.

In case you do decide to release it, I'd be happy to help make it happen.

Cheers, VB

amaksai commented 3 weeks ago

Hi VB,

Thanks! As you have probably seen, we've released the model in a form of tf.SavedModel and you can run the inference with it in the colab. The weights are baked inside the saved model. We could also release the raw checkpoint from the training, but AFAIK the PaLI training/inference code was never open-sourced so I am not sure if that will be very helpful (although I believe that there are open-source reproductions).

I don't think we will have capacity on our side to actually produce the inference code to run the checkpoints, but if you think that nevertheless releasing the checkpoints (and any additional info) could be helpful and perhaps somebody could make use of it, I think it is likely doable. LMK!

Cheers, Andrii

amaksai commented 3 weeks ago

And if you were not talking about the checkpoint / weights themselves, but rather the model being available through HF hub, we've made https://huggingface.co/Derendering/InkSight-Small-p public

amaksai commented 3 weeks ago

FYI, we have linked the model on HF in the main README. LMK your thoughts on raw checkpoints / weights.

robinchm commented 3 weeks ago

Any chance to release the better performing models small-i and large-i as well? The in-house datasets won't be released for sure, but the models trained on them may have a chance?

amaksai commented 3 weeks ago

Hi Robin, I will check internally and update here in case of positive reply, but to manage the expectations, I think it's quite unlikely

robinchm commented 3 weeks ago

Hi Robin, I will check internally and update here in case of positive reply, but to manage the expectations, I think it's quite unlikely

Yeah I understand chance is slim, thanks for making the effort.

Vaibhavs10 commented 2 weeks ago

Hi @amaksai - Thanks for uploading the model checkpoints and sorry for the delay in responding! Lovely that you have a uploaded the model checkpoints and linked them to the GitHub too.

re: raw model weights - IMO they'd only be useful with the infrence codebase, although it might make sense to release them with potential directions on how one can go about setting up the inference codebase! - this would be helpful for the curious ones. Let me know what you think!