apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
36.59k stars 14.17k forks source link

Implement pre-commit plugins for VSCode/PyCharm/Intellij maybe #35253

Open potiuk opened 11 months ago

potiuk commented 11 months ago

Body

We are one of the biggest users of pre-commit framework and we have A LOT of pre-commit checks that not only tell you what you should fix but also often fix things for you. This is happening "early" (i.e. at commit time if you run pre-commit install. However, seems that it should be possible to run them even earlier in software development cycle - i.e. in the IDE as you modify files.

It shoudl be possible to build a plugin for IntelliJ(PyCharm) and VSCode that would register to the same files that .pre-commit-config.yml describe and run pre-commits in the background for those change - either fixing files as you save them or surfacing the errors it finds as IDE errors - with link to the sources and corrective actions even (possibly as long term option).

I think we have everything we need from the pre-commit side of things - also our pre-commits are optimized to be as fsat as possible especially a lot of emphasis is put on caching in some cases. So it should be entirely suitable to run those pre-commits as part of such plugins.

If someone has an experience with writing IntelliJ / VSCode plugins, that might be an interesting, good first task that we could even contribute to the general community - it should be generic. Airflow does not use anything "special" in pre-commit, everything we do there is "generically" supported by standard pre-commit (we just wrap the calls to pre-commit in breeze to allow for autcompletion and some shorthand arguments).

Also see discussion in https://github.com/apache/airflow/pull/35236#issuecomment-1784118677

Committer

bolkedebruin commented 11 months ago

Black uses a protocol for this which allows integration with any ide that supports this. So they integrate with intellij / vscode etc without too much effort.

https://black.readthedocs.io/en/stable/integrations/editors.html

Might be something to look at.

potiuk commented 11 months ago

Black uses a protocol for this which allows integration with any ide that supports this. So they integrate with intellij / vscode etc without too much effort.

https://black.readthedocs.io/en/stable/integrations/editors.html

Might be something to look at.

Yes. But if you ever interacted with pre-commit creator (I did, and some other maintainer did), he will likely not cooperate on that. You can try, but the experience of trying to contribute to pre-commit something as simple as auto-complete was, let's say strange. So I am afraid what's left is to do wrapper around pre-commit.

See https://github.com/pre-commit/pre-commit/issues/1119 if you are interested.

vagi8 commented 11 months ago

Hey @potiuk, I would love to do it. But I guess there is already an extension for vscode which does exactly what you require.

Could you elaborate if you need something different than what already exists or anything totally different ?

potiuk commented 11 months ago

Hey @potiuk, I would love to do it. But I guess there is already an extension for vscode which does exactly what you require.

If so:

vagi8 commented 11 months ago

Sure, I will go ahead and create a PR.

ad-m-ss commented 10 months ago

Based on project other than Apache Airflow, VSCode extensions work great with pre-commit for our internal project. I switched from Python Language Server (deprecated support for any linters) and Black plugin to pre-commit plugin to VSCode, and works smoothly.

Except documentation, Airflow may need to review if hooks types are successfully detected (see https://marketplace.visualstudio.com/items?itemName=elagil.pre-commit-helper#detection-of-hook-type ), and we may need extra remarks about it in pre-commit.config.yml to maintain future compatibility.

potiuk commented 10 months ago

Based on project other than Apache Airflow, VSCode extensions work great with pre-commit for our internal project. I switched from Python Language Server (deprecated support for any linters) and Black plugin to pre-commit plugin to VSCode, and works smoothly.

Except documentation, Airflow may need to review if hooks types are successfully detected (see https://marketplace.visualstudio.com/items?itemName=elagil.pre-commit-helper#detection-of-hook-type ), and we may need extra remarks about it in pre-commit.config.yml to maintain future compatibility.

Looks good. This is very much what I have imagined as such extension could do - looks like indeed someone who uses vscode might want to make PR to adjust some of the description etc. to get the hooks properly detected. Would be nice to see something like that for IntelliJ.

khushi-98 commented 9 months ago

I would like to work on this error.