Document that we can't use new stdlib symbols until added to type-checkers

Avasam commented 6 months ago

https://github.com/python/typeshed/pull/11890#discussion_r1597530831 An unwritten convention (because we've been bitten by it a few times in the past), is to not use newly added symbols from the stdlib stubs until all supported type-checkers (pyright, mypy, pytype??, realistically we're always waiting on mypy release because pyright releases so often) updated their vendored version of typeshed.

This is a suggestion to add this convention to the contributing guideline in a more official way, and to teach contributors about this unobvious gotcha.

Relates to #5835

srittau commented 6 months ago

One of the ideas behind #5952 was to actually test third-party stubs using the vendored stdlib of the respective type checkers. Maybe we should resurrect that idea?

AlexWaygood commented 6 months ago

One of the ideas behind #5952 was to actually test third-party stubs using the vendored stdlib of the respective type checkers. Maybe we should resurrect that idea?

I don't think we should only test our third-party stubs against type checkers' vendored stdlib stubs, because it would mean we'd be blind to the impact changes we make to our stdlib stubs would have on our third-party stubs. That would:

mean we'd lose an important signal for when a change might be really disruptive for the broader community
possibly make upgrading our type checker pins much harder, because we might suddenly and unexpectedly have to deal with a significant number new of new errors in our third-party stubs that arise from changes we made to our stdlib stubs several months earlier

python / typeshed

Document that we can't use new stdlib symbols until added to type-checkers #11903