search

matangover / mypy-vscode

VS Code extension that provides type checking and analysis for Python code using mypy.
https://marketplace.visualstudio.com/items?itemName=matangover.mypy
Other
96 stars 17 forks source link

Feature request: Support running a single dmypy process for the entire workspace #95

Open yajo opened 3 months ago

yajo commented 3 months ago

I have a metarepo with 38 sub-repos. All those are wired on VSCode using a multi-root workspace.

Now when I open that workspace, this extension will start 38 dmypy processes. This makes my computer sluggish, at least for a noticeable time until the processes "relax".

Would it be possible to run a single dmypy process per workspace (not one per repo)?

Thanks!

paul-michalik commented 3 months ago

Yes. This makes the extension unusable for multi-root workspaces with non-trivial number of projects.

matangover commented 1 month ago

Sorry for the delay in responding. You can do this by setting mypy.enabled to false in the workspace, and then setting it to true only for a single workspace folder. And set the parameters (especially mypy.targets) as you see fit. Does this work for you? You could also enable/disable the daemon for each workspace folder individually this way.

yajo commented 1 month ago

It might work, but honestly, I wouldn't consider that a fix.

matangover commented 1 month ago

That's a bit rude. I wasn't saying it's a fix. I offered you a workaround for your use case. I've renamed this issue to clarify that it is a feature request

yajo commented 1 month ago

Sorry I didn't mean to be rude! As a workaround it might serve for some. However this still needs a fix.

LaurensBosscher commented 1 month ago

I've looked into this and unless you're running a single .venv for the entire mono-repo this might not be so trivial.

What I've tried is the following:

  1. Start the Mypy Daemon in the mono-repo root
  2. Run "check" with the path to a specific project

This somewhat works but you'll run into the following issues:

  1. If a dependency is not in the .venv from where you're running the Mypy Daemon it seems to be unable to find that dependency
  2. If you have project specific configuration in a pyproject.toml it doesn't seem to be applied

I've only done a quick experiment so these limitations might be due to a project specific configuration.

You can however greatly reduce the startup time of the Mypy Daemon by doing the following:

  1. Create a script that recursively goes into all your projects in the mono-repo and runs "poetry run mypy --cache-fine-grained --check . " in each project to create a Mypy cache.
  2. Configure the Mypy extension to run with "--use-fine-grained-cache": image

You'll need to run the script specified at 1 every once in a while (I run it daily), to update the cold boot cache.

When rebooting VScode this reduces the time to start the Mypy daemons (we seem to have ~20) to about 60 seconds. Memory usage is still a concern, but this can be solved relatively cheaply by giving your developers more memory.

If this information is useful for others (I can imagine that this is a concern for all large Python projects), I'm happy to create a quick PR with instructions for the readme.

matangover commented 1 month ago

This is a great tip for those with large codebases! Thank you. I will definitely approve such a PR that provides a concise explanation in the readme, also pointing to the relevant docs

matangover commented 1 month ago

If a dependency is not in the .venv from where you're running the Mypy Daemon it seems to be unable to find that dependency

Yes, if you have more than one venv, you can't run a single daemon for all of them. However, if your monorepo uses a single (or several) venvs, you can configure any number of daemons for those venvs (using mypy.enabled as explained above), specifying mypy.targets as needed.