Open cxxz opened 1 year ago
@cxxz , can you try the least transformers and accelerate library? I cannot reproduce the error on my end. It does include the offload section in my test. thanks
Thank you for responding to my request. I have done pip install git+https://github.com/huggingface/transformers
and pip install git+https://github.com/huggingface/accelerate
, as confirmed by pip_freeze. However, upon rerunning run_autotune_llama_4A100.sh, the offload
section still failed to be transferred to the ds_config.json
files in all attempts. The complete log has been documented in the repository. Any hint on what settings might have gone wrong?
Describe the bug I tried to enable offloading in the zero2_auto.json file with the following lines,
It works fine for normal runs without the
--autotune
flag. However, once I usedeepspeed --autotune
, none of the automatically generated .json files have theoffload_optimizer
section, i.e. one sample json file generated by autotunerThis contradicts what is stated in the README: "Currently, the DeepSpeed Autotuner does not tune offloading behaviors but instead uses the values defined in the offload section of the DeepSpeed configuration file." [https://github.com/microsoft/DeepSpeed/tree/master/deepspeed/autotuning#offloading-and-nvme]
To Reproduce
Expected behavior All generated ds_config.json during the search should have the
offload
section.System info (please complete the following information):