[REQUEST] how to RLHF-fine-tune custom data for deepspeedChat?

[ucas010]

hi,dear any doc for the data dir and the model to load and infer ?

down is Chinese 大佬，微调的数据存放在哪啊？能改成自己的数据么？ 另外：最后微调好的模型在哪啊？怎么加载和推理啊？

[ucas010]

训练完step3后的模型保存在哪了啊？

[ucas010]

微调后的actor model python inference/chatbot.py --path output/actor-models/1.3b/

 Human: can you speak chinese ?
 Assistant:  I’m sorry, I don’t understand.  Can you explain what you mean?

这是什么情况啊？

[ucas010]

and the default model opt-1.3b

 python inference/chatbot.py --path /root/.cache/huggingface/hub/models--facebook--opt-1.3b/snapshots/8c7b10754972749675d22364c25c428b29face51/
Enter input (type 'quit' to exit, 'clear' to clean memory): can you speak chinese ?
/lib/python3.9/site-packages/transformers/generation/utils.py:1219: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation)
  warnings.warn(
------------------------------ Round 1 ------------------------------
 Human: can you speak chinese ?
 Assistant:                                                                                                                                 
Enter input (type 'quit' to exit, 'clear' to clean memory): what's your name ?
------------------------------ Round 2 ------------------------------
 Human: can you speak chinese ?
 Assistant:                                                                                                                                 

 Human: what's your name ?
 Assistant:                                                                                                                                 
Enter input (type 'quit' to exit, 'clear' to clean memory): can you speak chinese ?
------------------------------ Round 3 ------------------------------
 Human: can you speak chinese ?
 Assistant:                                                                                                                                 

 Human: what's your name ?
 Assistant:                                                                                                                                 

 Human: can you speak chinese ?
 Assistant:                                   

[zy-sunshine]

python3 chat.py --path output/actor-models/1.3b Human: can you speak chinese ? Assistant: I can speak some Chinese, but I don't know much. I can read a few words and phrases, but I don't know the words or phrases themselves.

这是我的step1模型，效果好像稍微好点。

[zy-sunshine]

step3训练完后我感觉应该是在output/step3-models/1.3b/actor