Use dmypy instead of mypy

fgblomqvist commented 4 years ago

First time contributor checklist

[x] I have read how to contribute to this project
[x] I have read the code of conduct to this project

Contributor checklist

[x] I am using the provided codeStyleConfig.xml
[x] I have tested my contribution on these versions of PyCharm/IDEA:
- IntelliJ IDEA 2019.3.1
[x] My contribution is fully baked and ready to be merged as is
[ ] I ensure that all the open issues my contribution fixes are mentioned in the commit message of my first commit using the Fixes #1234 syntax

Description

This PR intends to either switch this plugin over to use dmypy completely, or at least provide an option to do so. Using dmypy is much faster and efficient than using mypy, it will allow sub-second scans compared to the 5-15 seconds it could take for mypy to do the same task.

There is still some work left to do. E.g. deciding whether we should just always use dmypy or whether it should be configurable (and how that should work). A fix/solution is also needed to compensate for the fact that dmypy does not support --follow-imports silent. I'm proposing that the function that parses the mypy output can filter out any errors produced by --follow-imports error and that we default to that option instead.

Lastly, it probably just needs a tad more testing (e.g. in PyCharm, and perhaps some other projects).

Nonetheless, the code in this PR serves as a proof-of-concept for now, and I intend to follow through with getting it into a merge-able state as long as some decisions on the above are made.

Type of Changes

	Type
✓	:sparkles: New feature

Related Issue

Closes #57 Closes #43

leinardi commented 4 years ago

Unfortunately I can't find the time to work on this right now, I'll try to do something the next weekend.

fgblomqvist commented 4 years ago

No worries, don't expect you to finish this PR. I'm thinking the best approach is to completely switch the plugin to use dmypy since that makes it simpler for the users and also makes the most sense in pretty much every single way. However, I did do some more research on the follow imports stuff and realized that follow imports error is not the way to go. Ideally we still want normal, which is not supported by the daemon yet. There does seem to be a workaround tho: https://github.com/python/mypy/issues/5870#issuecomment-544339138 Utilizing that workaround might just have to become a dependency I suppose. Idk.

Nonetheless, feel free to provide your thoughts when you find the time. I might test the above out next week.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had activity in the last 60 days.

Bunkerbewohner commented 4 years ago

Update: by now dmypy supports the default option normal for follow-imports. So hypothetically it can be used instead of mypy. However, dmypy still is sort of experimental: Its output might vary slightly (in my case it complained about missing types where mypy wasn't complaining even when using the same config file). Also it sometimes can get stuck, e.g. when switching branches while the daemon is running. Therefore the use of dmypy should probably be an option rather than the default.

Bunkerbewohner commented 4 years ago

P.S. However I'd still be very keen on getting this dmypy option since for regular use cases the performance gain is immense. In one project I'm working on mypy runtimes decreased from 70+ seconds to 200ms for incremental changes.

leinardi commented 4 years ago

@Bunkerbewohner thanks for the update.

fgblomqvist commented 4 years ago

Thanks for the update @Bunkerbewohner. I'm atm not currently actively developing in Python, but I am willing to try to put some more effort into this to get it merged. I made the changes you suggested, happy that they added the "normal" option. However, I'm in a new environment and I can't get this plugin to build anymore.

This is the error I'm getting:

/.../mypy-pycharm/src/main/java/com/leinardi/pycharm/mypy/toolwindow/TogglableTreeNode.java:49: error: incompatible types: inference variable T has incompatible bounds
        return Collections.unmodifiableList(children);
                                           ^
    equality constraints: TogglableTreeNode
    lower bounds: TreeNode
  where T is a type-variable:
    T extends Object declared in method <T>unmodifiableList(List<? extends T>)

I'm using Java 9 which I'm not sure is right (I'm no java dev). Maybe @leinardi can shed some light on this. Once I (or someone else) can get this to build I can verify that the changes I made work.

fgblomqvist commented 4 years ago

I rebased on top of the latest master.

leinardi commented 4 years ago

For me your branch builds fine, but I'm using Java 8:

$ java -version
openjdk version "1.8.0_252"
OpenJDK Runtime Environment (build 1.8.0_252-8u252-b09-1ubuntu1-b09)
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)

However, when I use the plugin built from this branch, it seems to be completely broken for me and I keep getting this error inside the Event log:

19.51   Unexpected Exception Caught
                The scan failed due to an exception: Error while running Mypy: mypy: can't read file 'run': No such file or directory
                com.leinardi.pycharm.mypy.exception.MypyToolException: Error while running Mypy: mypy: can't read file 'run': No such file or directory
                at com.leinardi.pycharm.mypy.mpapi.MypyRunner.runMypy(MypyRunner.java:307)
                at com.leinardi.pycharm.mypy.mpapi.MypyRunner.scan(MypyRunner.java:261)
                at com.leinardi.pycharm.mypy.checker.ScanFiles.scan(ScanFiles.java:109)
                at com.leinardi.pycharm.mypy.checker.ScanFiles.checkFiles(ScanFiles.java:100)
                at com.leinardi.pycharm.mypy.checker.ScanFiles.call(ScanFiles.java:74)
                at com.leinardi.pycharm.mypy.MypyInspection.inspectFile(MypyInspection.java:88)
                at com.leinardi.pycharm.mypy.MypyInspection.lambda$checkFile$0(MypyInspection.java:65)
                at com.intellij.openapi.application.impl.ApplicationImpl$1.call(ApplicationImpl.ja... (show balloon)

This does not happen using the latest master so it seems to be related to this branch.

To reproduce the issue you can checkout the tag v0.7.10 of pyxel, import the project in PyCharm and then open the app.py (https://github.com/kitao/pyxel/blob/v0.7.10/pyxel/app.py):

This is how my environment looks:

Bunkerbewohner commented 4 years ago

@fgblomqvist thanks a lot!

@leinardi @fgblomqvist the problem is that dmypy has its own executable. The equivalent of "mypy --config-file mypy.ini file.py" using the daemon looks like this: "dmypy run -- --config-file mypy.ini file.py". I commented where it would need to be adjusted. I wonder if it's ok to simply take the path of mypy and prepend the "d" to the filename. If that file exists daemon mode is supported and should work, otherwise not.

fgblomqvist commented 4 years ago

@Bunkerbewohner ah that's new (they didn't use to have a separate executable, I'm pretty sure). Easy to fix. And yeah I think that sounds good enough.

@leinardi Hmm okay let me try with Java 8 then.

fgblomqvist commented 4 years ago

I was able to compile with Java 8 :tada:

I pushed changes that does a very simple (and stupid) path-replacement to use dmypy instead of mypy is daemon mode is enabled. It gets the job done I think. However, when I run it on a local Python repo I got the daemon crashes with a KeyError sys. Googled a bit and it sounds like it might be a dmypy bug. Lmk if it works for you guys.

Regardless, the behavior can probably be improved a bit.

leinardi commented 4 years ago

I tried the latest branch and I found the following issues:

The "Use Daemon" option is not persisted: if I check it and press OK, if I re-open the settings is again unchecked:
If I run the plugin on the gwe module the plugin finds 279 errors instead using this branch but 33 if I build master. Probably some changes in this branch broke the mypy.ini recognition?

This branch: master:

To reproduce you can run the plugin on this module: https://gitlab.com/leinardi/gwe/-/blob/master/gwe/__main__.py

fgblomqvist commented 4 years ago

Fixed the persistence problem. I sometimes now get around 50 errors in that project and sometimes no problems at all. Can't figure out how to get debug logging in IDEA to work so don't really know how to debug this further. Regardless, the problem you're seeing with 200+ errors is not something I can replicate.

Bunkerbewohner commented 4 years ago

Thanks @fgblomqvist! I have tested the current version on a project of my own and still have some issues. I get the following error when starting "Check Current File" using the plugin:

2020-07-25 14:45:56,754 [1525935]   INFO - .pycharm.mypy.mpapi.MypyRunner - Command Line string: /Users/mathias/Library/Java/JavaVirtualMachines/adopt-openj9-1.8.0_262/Contents/Home /Users/mathias/.pyenv/versions/3.8.1/bin/mypy -V 
2020-07-25 14:45:56,755 [1525936]   WARN - .pycharm.mypy.mpapi.MypyRunner - Error while checking Mypy path 
com.intellij.execution.process.ProcessNotCreatedException: Cannot run program "/Users/mathias/Library/Java/JavaVirtualMachines/adopt-openj9-1.8.0_262/Contents/Home": error=13, Permission denied
    at com.intellij.execution.configurations.GeneralCommandLine.createProcess(GeneralCommandLine.java:418)
    at com.leinardi.pycharm.mypy.mpapi.MypyRunner.isMypyPathValid(MypyRunner.java:96)
    at com.leinardi.pycharm.mypy.mpapi.MypyRunner.checkMypyAvailable(MypyRunner.java:179)
    at com.leinardi.pycharm.mypy.mpapi.MypyRunner.checkMypyAvailable(MypyRunner.java:152)
    at com.leinardi.pycharm.mypy.MypyInspection.inspectFile(MypyInspection.java:76)
    at com.leinardi.pycharm.mypy.MypyInspection.lambda$checkFile$0(MypyInspection.java:65)
    at com.intellij.openapi.application.impl.ApplicationImpl$1.call(ApplicationImpl.java:255)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.io.IOException: Cannot run program "/Users/mathias/Library/Java/JavaVirtualMachines/adopt-openj9-1.8.0_262/Contents/Home": error=13, Permission denied
    at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
    at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
    at com.intellij.execution.configurations.GeneralCommandLine.startProcess(GeneralCommandLine.java:449)
    at com.intellij.execution.configurations.GeneralCommandLine.createProcess(GeneralCommandLine.java:414)
    ... 10 more
Caused by: java.io.IOException: error=13, Permission denied
    at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
    at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:340)
    at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:271)
    at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1107)
    ... 13 more

However, if I manually start the daemon on the command line the plugin works as expected and errors are checked super quickly. Also I don't get any unexpected mypy errors listed in my project which is quite sizeable (a clean mypy run takes more than 70s).

My settings look like this:

The "Test" button successfully finds the mypy executable. Obviously the path in the error log above doesn't make much sense. I will investigate further and see if I can get this to run properly on my machine (macOS).

Bunkerbewohner commented 4 years ago

P.S. I changed the error logging a little and now the error I see is this:

The scan failed due to an exception: Error while running Mypy command /Users/mathias/.virtualenvs/kialo/bin/dmypy run -- --show-column-numbers --config-file /Users/mathias/kialo/backend/mypy.ini /Users/mathias/kialo/backend/kialo/entities/base_unit_test.py:
            [Errno 61] Connection refused

So the way that dmypy is invoked is correct including the path to the config file. No idea why launching the process doesn't work though. I mean in the earlier error it said "java.io.IOException: error=13, Permission denied" but why...

fgblomqvist commented 4 years ago

I had similar odd java errors when using adopt. Switched to standard openjdk which worked a lot better. So yeah beyond the weird java error which I don't think has to do with the plug-in, it seems like the only thing left to do is to disable the daemon by default. I can do that this coming week. Thanks for reporting back and lmk if you solve the java issue!

fgblomqvist commented 4 years ago

Got around to changing it, lmk if anything else should be done. When you have time @leinardi, please give it another go to see if you still see weird behavior.

leinardi commented 4 years ago

Hi @fgblomqvist, sorry for the late reply, it was a busy week at work.

I tested your branch again and the daemon checkbox works correctly. But now, when I enable it, mypy doesn't find any violation at all. The Event log shows no errors.

I made a short video to show the issue (unfortunately to be able to attach it I had to zip it): 2020-08-08 10-58-04.zip

fgblomqvist commented 4 years ago

I see. Yeah I had similar results sometimes. Idk if it's because the daemon is finicky or if there is something wrong with how I run it (but I don't think so) 🤔 . Appreciate the response at least.

DetachHead commented 3 years ago

i believe i've fixed some of these issues https://github.com/fgblomqvist/mypy-pycharm/pull/1

DetachHead commented 3 years ago

some further outstanding issues i've identified

[ ] the daemon will sometimes report false positive when run on individual files that don't appear when running on the whole project (ie. with dmypy run -- mainn.py vs dmypy run -- .
[ ] the daemon will crash if there are lots of files in the project and you attempt to scan the whole project

looking at the mypy vscode plugin which uses the daemon and seems to be much more reliable, by default it only ever runs on the whole project and not individual files: this isn't optimal though as it would need to run on ignored files and filter out those results, but i think it may result in more stability when using the daemon.

jesse-viola commented 2 years ago

Any updates on this PR? Would be a great feature.

nateschickler-puzzle commented 2 years ago

I updated the code to scan the whole project as per the above comment on my own branch, rebased, and tried running it on a project but am now running into this mypy bug: https://github.com/python/mypy/issues/10709 Just wanted to report in case others are using ignore_missing_imports = True in their projects, dmypy may not run until that issue is resolved. I was not successful with the workaround mentioned by some commenters in the linked issue :(

isuro commented 1 year ago

Hey @fgblomqvist the issue linked by @nateschickler-puzzle seems to have been resolved. Any chance of getting this going again?

fgblomqvist commented 1 year ago

Hi @isuro, unfortunately I don't really have the time or motivation to continue this at the moment (haven't worked with Python in the last couple of years at least), but I welcome any attempts to pick it up! 🙂

I would start with @nateschickler-puzzle's work here: https://github.com/nateschickler-puzzle/mypy-pycharm And then try to rebase it again and then see if it runs now that the bug is fixed. Projects like this tend to require quite a bit of testing/tinkering so it's easier to work on it if you actually use it daily.

leinardi / mypy-pycharm