DeepSpeed zero3

[Abolfazl-kr]

Check before submitting issues

• [X] Make sure to pull the latest code, as some issues and bugs have been fixed.
• [X] I have read the Wiki and FAQ section AND searched for similar issues and did not find a similar problem or solution
• [X] Third-party plugin issues - e.g., llama.cpp, LangChain, text-generation-webui, we recommend checking the corresponding project for solutions

Type of Issue

Model training and fine-tuning

Base Model

Chinese-LLaMA-2 (7B/13B)

Operating System

Linux

Describe your issue in detail

what should we do to use deep speed stage3? what changes should apply on deep speed config and the codes?

Dependencies (must be provided for code-related issues)

# Please copy-and-paste your dependencies here.

Execution logs or screenshots

# Please copy-and-paste your logs here.