Bump accelerate from 0.30.1 to 0.33.0

[dependabot[bot]]

Bumps accelerate from 0.30.1 to 0.33.0.

Release notes

Sourced from accelerate's releases.

v0.32.0: Profilers, new hooks, speedups, and more!

Core

• Utilize shard saving from the huggingface_hub rather than our own implementation (huggingface/accelerate#2795)
• Refactor logging to use logger in dispatch_model (huggingface/accelerate#2855)
• The Accelerator.step number is now restored when using save_state and load_state (huggingface/accelerate#2765)
• A new profiler has been added allowing users to collect performance metrics during model training and inference, including detailed analysis of execution time and memory consumption. These can then be generated in Chrome's tracing tool. Read more about it here (huggingface/accelerate#2883)
• Reduced import times for doing import accelerate and any other major core import by 68%, now should be only slightly longer than doing import torch (huggingface/accelerate#2845)
• Fixed a bug in get_backend and added a clear_device_cache utility (huggingface/accelerate#2857)

Distributed Data Parallelism

• Introduce DDP communication hooks to have more flexibility in how gradients are communicated across workers, overriding the standard allreduce. (huggingface/accelerate#2841)
• Make log_line_prefix_template optional the notebook_launcher (huggingface/accelerate#2888)

FSDP

• If the output directory doesn't exist when using accelerate merge-weights, one will be automatically created (huggingface/accelerate#2854)
• When merging weights, the default is now .safetensors (huggingface/accelerate#2853)

XPU

• Migrate to pytorch's native XPU backend on torch>=2.4 (huggingface/accelerate#2825)
• Add @require_triton test decorator and enable test_dynamo work on xpu (huggingface/accelerate#2878)
• Fixed load_state_dict not working on xpu and refine xpu safetensors version check (huggingface/accelerate#2879)

XLA

• Added support for XLA Dynamo backends for both training and inference (huggingface/accelerate#2892)

Examples

• Added a new multi-cpu SLURM example using accelerate launch (huggingface/accelerate#2902)

Full Changelog

• Use shard saving from huggingface_hub by @​SunMarc in huggingface/accelerate#2795
• doc: fix link by @​imba-tjd in huggingface/accelerate#2844
• Revert "Slight rename" by @​SunMarc in huggingface/accelerate#2850
• remove warning hook addede during dispatch_model by @​SunMarc in huggingface/accelerate#2843
• Remove underlines between badges by @​novialriptide in huggingface/accelerate#2851
• Auto create dir when merging FSDP weights by @​helloworld1 in huggingface/accelerate#2854
• Add DDP Communication Hooks by @​yhna940 in huggingface/accelerate#2841
• Refactor logging to use logger in dispatch_model by @​panjd123 in huggingface/accelerate#2855
• xpu: support xpu backend from stock pytorch (>=2.4) by @​dvrogozh in huggingface/accelerate#2825
• Drop torch re-imports in npu and mlu paths by @​dvrogozh in huggingface/accelerate#2856
• Default FSDP weights merge to safetensors by @​helloworld1 in huggingface/accelerate#2853
• [tests] fix bug in test_tracking.ClearMLTest by @​faaany in huggingface/accelerate#2863
• [tests] use torch_device instead of 0 for device check by @​faaany in huggingface/accelerate#2861
• [tests] skip bnb-related tests instead of failing on xpu by @​faaany in huggingface/accelerate#2860
• Potentially fix tests by @​muellerzr in huggingface/accelerate#2862
• [tests] enable XPU backend for test_zero3_integration by @​faaany in huggingface/accelerate#2864
• Support saving and loading of step while saving and loading state by @​bipinKrishnan in huggingface/accelerate#2765
• Add Profiler Support for Performance Analysis by @​yhna940 in huggingface/accelerate#2883
• Speed up imports and add a CI by @​muellerzr in huggingface/accelerate#2845
• Make log_line_prefix_template Optional in Elastic Launcher for Backward Compatibility by @​yhna940 in huggingface/accelerate#2888
• Add XLA Dynamo backends for training and inference by @​johnsutor in huggingface/accelerate#2892
• Added a MultiCPU SLURM example using Accelerate Launch and MPIRun by @​okhleif-IL in huggingface/accelerate#2902
• make more cuda-only tests device-agnostic by @​faaany in huggingface/accelerate#2876

... (truncated)

Commits

• 28a3b98 Release: v0.33.0
• 415eddf feat(ci): add pip caching in CI (#2952)
• 2308576 Properly handle Params4bit in set_module_tensor_to_device (#2934)
• a5a3e57 Add torch.float8_e4m3fn format dtype_byte_size (#2945)
• 0af1d8b delete CCL env var setting (#2927)
• d16d737 Improve test reliability for Accelerator.free_memory() (#2935)
• 7a5c231 Consider pynvml available when installed through the nvidia-ml-py distributio...
• 4f02bb7 Fix import test (#2931)
• 709fd1e Hotfix PyTorch Version Installation in CI Workflow for Minimum Version Matrix...
• f4f1260 Correct loading of models with shared tensors when using accelerator.load_sta...
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)