Open vishaal27 opened 1 year ago
did you Re-run the Implementation?
because I am trying to Re-implement the same Modeling design but instead of LLaMa I would like to use using LoRa
and I was think also formalizing Loss For multi-Task Learning such as MCO
Hi, is it possible to get the tokenwise log-likelihood scores of different outputs from the model?
The use-case would be something like: Given an interleaved image/text input and a list of output text candidates, we should be able to get a score for each output candidate and then return their ranked list, rather than generating the outputs directly. This would be close to how LLMs are evaluated on MCQ tasks. An example from the T0 paper Page 6 (https://arxiv.org/pdf/2110.08207.pdf):
Is it straightforward to do this with LaVIN? I assume since the LM is built with transformers there should be a possibility to use output score functions already implemented (haven't dug into this yet)?