search

kohjingyu / fromage

🧀 Code and models for the ICML 2023 paper "Grounding Language Models to Images for Multimodal Inputs and Outputs".
https://jykoh.com/fromage
Apache License 2.0
466 stars 34 forks source link

Can I use the embedding for training #35

Closed LiJichen0114 closed 2 months ago

LiJichen0114 commented 3 months ago

Hello, thank you for your work. I would like to ask if I can directly use cc3m_embeddings.pkl for model training? Or do I need to download the cc3m dataset? Looking forward to your reply.

kohjingyu commented 3 months ago

For training the model you would most likely have to use the CC3M dataset. The embeddings are mostly used as retrieval candidates during inference.

LiJichen0114 commented 2 months ago

Thank you!