search

mayhugotong / GenTKG

This is the official impletation repository of NAACL findings paper, GenTKG: Generative Forecasting on Temporal Knowledge Graph with Large Language Models. (https://arxiv.org/abs/2310.07793).
32 stars 5 forks source link

which one pre-trained model checkpoint in fune-tuning? #7

Open zsxzs opened 1 month ago

zsxzs commented 1 month ago

Thank you great work! I have a few questiones about peft? I hope you can answer that. Thank you a lot!

  1. Which model fine-tuning is best to use? Is it a pre-trained model (llama2-7b) or a supervised fine-tuned (llama2-7b-chat) model?
  2. What is the difference between TheBloke/Llama-2-7B-fp16 and meta-llama/Llama-2-7b?
mayhugotong commented 1 month ago

Hi,

Thank you for your interest in our work.

Q1: We use llama2-7b instead of llama2-7b-chat. Q2: TheBloke/Llama-2-7B-fp16 was the reproduced llama-2 version by a third party when we started our topic. We fine-tuned TheBloke version due to limited access to llama-2 back then.

I hope this answers your questions.