search

YuxinWenRick / hard-prompts-made-easy

MIT License
580 stars 54 forks source link

Fluency loss #17

Closed sunwoo76 closed 1 year ago

sunwoo76 commented 1 year ago

Does this version of the code reflect fluency loss?

Thanks :)

YuxinWenRick commented 1 year ago

Hi, we don't have fluency loss on the prompt inversion trained on the CLIP model, because we need another autoregressive language model which shares the same tokenizer and word embeddings with CLIP to calculate the loss, but to our knowledge, such a model is not available online. Therefore we have to train an autoregressive language model from scratch, which is very expensive for us.

However, we do include fluency_weight in our lm prompt tuning code. Because we use gpt-2 model, we can get fluency constraint for free.