search

kbressem / medAlpaca

LLM finetuned for medical question answering
GNU General Public License v3.0
474 stars 54 forks source link

Question about the medAlpaca init weight #43

Open XZhang97666 opened 1 year ago

XZhang97666 commented 1 year ago

Hi, I want to clarify the setting of medAlpaca. To my best of knowledge, the medAlpaca is initiate from Alpaca. However, have you ever try to use your dataset to directly ft Llama instead of Alpaca.

kbressem commented 12 months ago

Hi @XZhang97666. MedAlapca is trained in the style of Alpaca (same prompt, very similar code) hence the name. However, the model is based on LLaMA weights not on the Alpaca weights.