pytorch / pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration
https://pytorch.org
Other
83.57k stars 22.54k forks source link

Use lintrunner-adapters for our adapters when possible #124857

Open ezyang opened 6 months ago

ezyang commented 6 months ago

🐛 Describe the bug

https://github.com/justinchuby/lintrunner-adapters seems nice. We should use it.

@justinchuby, do you happen to know which adapters are exactly compatible with PyTorch's and which ones we shouldn't use? Also, I'm a bit unsure about how to actually install lintrunner_adapters; you seem to suggest that you should manually pip install it before running lintrunner, but I'd like lintrunner init to automatically take care of this, ideally.

Versions

main

malfet commented 6 months ago

It is, when it is not. lintrunner_adapters kill ones ability to run pip3 install lintrunner; lintrunner init

justinchuby commented 6 months ago

Thanks! Compatible linters are

'RUFF', 'FLAKE8', 'MYPY', 'NOQA', all the grep linters, 'SHELLCHECK', 'UFMT', 'CLANGFORMAT', 'EXEC', 'NEWLINE'.

All of the above except for RUFF were ported from PyTorch. PyTorch's 'RUFF' was ported from lintrunner_adapters.

'MYPY' was updated over the PyTorch implementation to properly respect the ignore paths.

automatically take care of this

I think it would be possible to retain the init commands and add lintrunner_adapters as a required package there. This way it will be available at lint time. This also allows PyTorch to have full control over the init script. (lintrunner_adapters provides the init script only as a convenience, and it is used in projects like ONNX https://github.com/onnx/onnx/blob/5ecc0a9801b724bd71859ceacd1562cfed4c415c/.lintrunner.toml#L57-L65)

init_command = [
    'python3',
    'tools/linter/adapters/pip_init.py',
    '--dry-run={{DRYRUN}}',
    '--no-black-binary',
    ...
]