add use_all_gather for option

[SangbumChoi]

What does this PR do?

Fixes https://github.com/huggingface/accelerate/issues/3147 @muellerzr

I think testing might need but not sure where to add this corresponding test

Before submitting

• [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• [ ] Did you read the contributor guideline, Pull Request section?
• [ ] Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
• [ ] Did you make sure to update the documentation with your changes? Here are the documentation guidelines, and here are tips on formatting docstrings.
• [ ] Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

[SangbumChoi]

Also have additional question for LLM inference evaluation process. Does LLM evaluation process requires to have all_gather function to make all the prediction in all processes?

[muellerzr]

Thanks! Tests should be in src/accelerate/test_utils/scripts/test_ops.py

The all_gather is so that we can get the results across all processes. It's there as an example and for simplicity for the user. (But it is not generally required, no)

[SangbumChoi]

@muellerzr Hi I haved added test. However, to be more strict test I think we can add additional checking when the process is not in the state.is_main_process and check the return is empty list. Let me know how do you think about this.

[SangbumChoi]

@muellerzr Gentle Ping!

[muellerzr]

re; test:I think it's fine how it is right now