search

KwaiKEG / KwaiAgents

A generalized information-seeking agent system with Large Language Models (LLMs).
Other
1.1k stars 105 forks source link

Training with publish kwaiAgentInstruct dataset #14

Closed superobk closed 10 months ago

superobk commented 10 months ago

Hello, may we have the training/finetune code with kwaiAgentInstruct, specially finetu ne for Qwen. Thank you!

Beckham007 commented 10 months ago

+1

zzp-seeker commented 10 months ago

You can refer to the finetuning instructions available at https://github.com/QwenLM/Qwen/tree/main?tab=readme-ov-file#finetuning. For our full parameter finetuning, we utilize a DeepSpeed + Zero3 configuration. You can find the DeepSpeed configuration details here: https://github.com/QwenLM/Qwen/blob/main/finetune/ds_config_zero3.json. For more training specifics, refer to our paper. Please note that during training, along with the Agent-type data we provide, you should independently incorporate General-type data into the training set.