huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
134.9k stars 26.99k forks source link

Request more specific info from bug reporters when opening deepspeed issues #34145

Open Ben-Schneider-code opened 1 month ago

Ben-Schneider-code commented 1 month ago

Feature request

Hi!

I would like the bug reporters to be prompted (or have section to fill in the reports template) to provide ds_report info and zero3 config when opening a bug report related to deepspeed integration (maybe it could be more general). Anything to make sure these bits of info are more likely to included upfront would make some of these issues much more actionable.

Motivation

I've been looking at some deepspeed integration bugs lately (#28808,#29348,#31867), I noticed that often more deepspeed info has to be requested. I was wondering if some specific (and maybe BOLDED) guidelines about what info to provide would go a long way when opening bug reports. I think a reminder to include zero configs and ds_report might be helpful. I believe this is particularily a pitfall for stuff that is often parsed in (configs, etc).
Something like:

Reproduction

Please provide a code sample that reproduces the problem you ran into. It can be a Colab link or just a code snippet. If you have code snippets, error messages, stack traces please provide them here as well. Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code. If you are opening an issue related to one of the following please ensure the this info is included in your reproduction script: Deepspeed - zero3 config, ds_report output, Trainer - your trainer config file, etc.

@ArthurZucker @amyeroberts

ArthurZucker commented 3 weeks ago

Sounds good, would you like to open a PR to update the bug report here : https://github.com/huggingface/transformers/blob/e4c19d7a6c6e41ed1beb9290d610b9c92222058a/.github/ISSUE_TEMPLATE/bug-report.yml#L3 ?