Is it ok to write the target_modules in full?

huggingface / autotrain-advanced

🤗 AutoTrain Advanced

https://huggingface.co/autotrain

Apache License 2.0

3.84k stars 472 forks source link

Is it ok to write the target_modules in full? #752

Open jackswl opened 1 month ago

jackswl commented 1 month ago

The configs gave

target_modules: all-linear

Is this equivalent to the code below?

target_modules: "q_proj,v_proj,o_proj,k_proj,gate_proj,down_proj,up_proj"

Will there be any differences between the either option? Is that the correct format for the second code? This is for SFT LLM. Thanks. @abhishekkrthakur

github-actions[bot] commented 3 days ago

This issue is stale because it has been open for 30 days with no activity.