Closed Taha-Bahadori closed 4 months ago
Hi @Taha-Bahadori, check out our tutorial on implementing a custom model output for BERT: https://trak.readthedocs.io/en/latest/bert.html. I haven't used tiny llama before, but my guess is, to start, you'll need to replace transformers.AutoModelForSequenceClassification
with transformers.AutoModelForCausalLM
. Hope this helps.
We intend to use Trak for attributing the outputs of a Q&A system to its data sources. We can see an example in qnli.py. Can you provide an example where the LLM is used for generating free-form text (generative setup)?
More specifically, consider
tinyllama
. How can we use Trak to attribute its response to questions to the data sources? How should we define theAbstractModelOutput
in the generative case?Even some guidelines would help too.