search

hauntsaninja / mypy_primer

Run mypy and pyright over millions of lines of code
MIT License
55 stars 29 forks source link

Should `mypy_primer` default to `--strict` and `--install-types` ? #64

Closed Avasam closed 1 year ago

Avasam commented 1 year ago

I was adding a project with a ton of types-* stubs when I thought: why not just auto-install all necessary stubs? Wouldn't that remove some configuration burden?

I was also thinking, what if the primer ran with --strict? It would result in more errors for projects that didn't turn it on. Which doesn't really matter when we're interested in the diff. And it would be beneficial to catch more of the changes' effects (whether they're negative, or positive: allowing libraries to turn on strict checks that would otherwise fail). At least for those configured with expected_success=False

hauntsaninja commented 1 year ago

Mainly because --install-types is super duper slow and I dislike it :-P Also worth noting that in typeshed CI, we'll still type check against typeshed stubs even if we don't specify types-xyz in the pip_cmd, so it's not super critical if one gets missed. We will get slightly less accurate typing in mypy CI, but my sense is that if you've got the main dependencies covered it's not too bad for finding regressions in mypy. Would definitely be useful to have a mode to update configuration if we got something like https://github.com/python/mypy/issues/10600 though!

Running with --strict is an interesting idea. In mypy CI we definitely do not want to run with --strict — we want to run mypy similar to how the projects themselves run it — but it's a more compelling idea for typeshed CI. I'm genuinely unsure how helpful it would be... you get some amount of value from --check-untyped-defs, but for the most part everything unannotated is still Any, so feels like if it catches something it'd probably just catch the same kind of shape errors that stubtest can catch. That said, mypy_primer already supports a --additional-flags argument, so this is a thing typeshed could just try today! Maybe make an issue on typeshed and see what people think?

Avasam commented 1 year ago

--install-types is super duper slow

In the context of mypy_primer, that is good enough reason for me, it needs all the help it can get to speedup ^^

hauntsaninja commented 1 year ago

Akuli brought this up today as well in https://github.com/hauntsaninja/mypy_primer/issues/66 ... Sounds like we should try it in typeshed CI!

Avasam commented 1 year ago

Unless you see value in leaving this open, I think you can close in favor of the other issue. Plus "mypy_primer already supports a --additional-flags argument" it's something we'd change in typeshed.

hauntsaninja commented 1 year ago

Yup, agreed that using --additional-flags to pass --strict in typeshed CI is the way to go.