search

Yiwei98 / TDG

24 stars 2 forks source link

data_path: str = "MATH_ChatGPT_8_t0.5" in finetune.py #3

Closed Git-L1 closed 8 months ago

Git-L1 commented 10 months ago

hello, Where can I find this dataset？ Looking forward to your reply. Thank you.

Yiwei98 commented 9 months ago

We are sorry that it actually refers to GPT3.5 turbo data. We have modified it in finetune.py. Thank you.

Git-L1 commented 9 months ago

@Yiwei98 Thank you, In addition, I have a small question that I would like to consult with you about. When running run_neg. sh, set load_in_8bit=True, and then set model=prepare_model-for-int8_training (model). During training, an error occurred, "You cannot perform fine-tuning on purely quantized models.Please attach trainable adapters on top of the quantized model to correctly perform fine-tuning. Please see: https://huggingface.co/docs/transformers/peft for more details" I am looking forward to your reply, thank you so much!

Yiwei98 commented 8 months ago

Is the problem related to the version of peft? May I ask if you are using the same version in requirements.txt?

Git-L1 commented 8 months ago

ok,I will try to use the same version,thank you so much!