search

facebook / pyre-check

Performant type-checking for python.
https://pyre-check.org/
MIT License
6.81k stars 434 forks source link

Checking a single line of Python takes 5s because the whole typeshed is parsed #592

Open not-my-profile opened 2 years ago

not-my-profile commented 2 years ago

Pyre Feature Request

$ mkdir foo
$ echo "print('hello world')" > foo/hello.py
$ time pyre --source-directory foo check
ƛ No binary specified, looking for `pyre.bin` in PATH
ƛ Could not determine the number of Pyre workers from configuration. Auto-set the value to 1.
ƛ No typeshed specified, looking for it...
ƛ Found: `/home/martin/.local/lib/pyre_check/typeshed`
ƛ Writing arguments into /tmp/pyre_arguments_yv2clp4p.json...
ƛ  Initializing shared memory [heap_size=8589934592, dep_table_pow=27, hash_table_pow=26]
ƛ  Building module tracker...
ƛ  Building type environment...
ƛ  Parsing 1934 stubs and sources...
ƛ  Parsing 1934 stubs and sources... [0.5s]
ƛ  Parsing 1934 stubs and sources... [0.6s]
ƛ  Parsing 1934 stubs and sources... [0.7s]
ƛ  Parsing 1934 stubs and sources... [0.8s]
ƛ  Parsing 1934 stubs and sources... [0.9s]
ƛ  Parsing 1934 stubs and sources... [1.0s]
ƛ  Parsing 1934 stubs and sources... [1.1s]
ƛ  Parsing 1934 stubs and sources... [1.2s]
ƛ  Parsing 1934 stubs and sources... [1.3s]
ƛ  Parsing 1934 stubs and sources... [1.4s]
ƛ  Parsing 1934 stubs and sources... [1.5s]
ƛ  Parsing 1934 stubs and sources... [1.6s]
ƛ  Parsing 1934 stubs and sources... [1.7s]
ƛ  Parsing 1934 stubs and sources... [1.8s]
ƛ  Parsing 1934 stubs and sources... [1.9s]
ƛ  Parsing 1934 stubs and sources... [2.0s]
ƛ  Parsing 1934 stubs and sources... [2.1s]
ƛ  Parsing 1934 stubs and sources... [2.2s]
ƛ  Parsing 1934 stubs and sources... [2.3s]
ƛ  Parsing 1934 stubs and sources... [2.4s]
ƛ  Parsing 1934 stubs and sources... [2.5s]
ƛ  Parsing 1934 stubs and sources... [2.6s]
ƛ  Parsing 1934 stubs and sources... [2.7s]
ƛ  Parsing 1934 stubs and sources... [2.8s]
ƛ  Parsing 1934 stubs and sources... [2.9s]
ƛ  Parsing 1934 stubs and sources... [3.0s]
ƛ  Parsing 1934 stubs and sources... [3.1s]
ƛ  Parsing 1934 stubs and sources... [3.2s]
ƛ  Parsing 1934 stubs and sources... [3.3s]
ƛ  Parsing 1934 stubs and sources... [3.4s]
ƛ  Parsing 1934 stubs and sources... [3.5s]
ƛ  Parsing 1934 stubs and sources... [3.6s]
ƛ  Parsing 1934 stubs and sources... [3.7s]
ƛ  Parsing 1934 stubs and sources... [3.8s]
ƛ  Checking 1 functions...
ƛ  Processed 1 of 1 functions
ƛ  Postprocessing 1934 sources...
ƛ  Postprocessed 242 of 1934 sources
ƛ  Postprocessed 484 of 1934 sources
ƛ  Postprocessed 726 of 1934 sources
ƛ  Postprocessed 968 of 1934 sources
ƛ  Postprocessed 1210 of 1934 sources
ƛ  Postprocessed 1452 of 1934 sources
ƛ  Postprocessed 1694 of 1934 sources
ƛ  Postprocessed 1934 of 1934 sources
ƛ No type errors found
pyre --source-directory foo check 4.74s user 0.53s system 99% cpu 5.297 total

It would be nice if pyre could parse the type stubs on demand instead of upfront to reduce the startup time.

grievejia commented 2 years ago

Thanks for the report!

Lazy parsing is something that we want to implement, but it requires some nontrivial amount of work so it probably will not happen soon. The tentative plan we have now is to implement some short-term mitigations (e.g. supporting PEP 561 out-of-the-box so we could drop the third-party typeshed bundled with Pyre) first. Fully addressing the issue would probably happen later this half or early next half.