Ensure `*/extern/__init__.py` is in sync with `*/_vendor/vendored.txt`

[abravalheri]

Sometimes I have the feeling that the vendored packages, _vendor/vendored.txt and extern/__init__.py drift apart. This is an attempt to minimise that.

Please feel free to close if you don't think this is relevant.

• Note: I haven't run tox -e vendor in this PR.

Summary of changes

• Added a small cog script that attempts to derive the *.extern.names from the */_vendor/vendored.txt file
• Added support function in tools/vendored.py
• Modified tox -e vendor to automatically run cog
• Added tox -e check-extern.
• Added more_itertools to pkg_resources/_vendor/vendored.txt? (maybe it was missing?)

Closes

Pull Request Checklist

• [ ] Changes have tests
• [ ] News fragment added in newsfragments/. (See [documentation][PR docs] for details)

[PR docs]: https://setuptools.pypa.io/en/latest/development/developer-guide.html#making-a-pull-request

[Avasam]

I've been thinking we may be able to simplify it by merely adding _vendor to sys.path (and eliminating extern altogether)

@jaraco Eliminating extern in favor of importing from _vendor (or making it so import vendored_module imports _vendor.vendored_module) would also solve type imports issues (like #4242) for type-checkers.

[abravalheri]

I think we can go with this one for now and any improvements can follow up. I also opened other issues for the comments we left.

I've been thinking we may be able to simplify it by merely adding _vendor to sys.path (and eliminating extern altogether), but I haven't experimented with that and I'm unsure what dragons might lurk in that approach.

Yeah, that would be good. The obvious drawback is the dependency clash if other packages installed in the same environment require conflicting versions.

A different approach (as referred by @Avasam) would be to eliminate the MetaPathFinder (and therefore the extern), but still explicitly import from setuptools._vendor. That would simply a little while avoiding the version clash. We still would have to keep the patching for absolute imports of all modules installed inside of _vendor.