search

CStanKonrad / long_llama

LongLLaMA is a large language model capable of handling long contexts. It is based on OpenLLaMA and fine-tuned with the Focused Transformer (FoT) method.
Apache License 2.0
1.45k stars 85 forks source link

Comparison with other tuning methods #6

Closed FLLLIGHT closed 1 year ago

FLLLIGHT commented 1 year ago

Thanks for your interesting work! I've some questions about your work:

  1. In my opinion, TruthfulQA is just an ordinary dataset, and there is no difference between it and other datasets (like MedicalQA). So your work is just an interesting method to regress given datasets (by adjusting the distribution), or it can impore the general ability of the model to generate more "truthful" answer?

  2. In Table1, you compared your method with Supervised Finetuning and Few-shot prompting, Is there more comparison between your mothod and other tuning methods like LLaMA+LoRA? If possible, could you compare your method with LLaMA+LangChain, because in practice if we want LLaMA to generate more precise answer, we'd consider LLaMA+LangChain first, though I think this method is inelegant and don't like the idea of LLM with database.

FLLLIGHT commented 1 year ago

Sorrrry, I tended to raise this issue in another repo.