Reconsider stdlib-list usage

[miketheman]

The author of stdlib-list has archived the repo on Jan 26, 2023, and mentioned they will no longer be maintaining the library. https://github.com/jackmaney/python-stdlib-list#this-repo-is-now-archived-i-no-longer-have-the-spoons-to-maintain-this-in-my-spare-time

No names for Python 3.10, 3.11, or the upcoming 3.12 are included.

We have sys.stdlib_module_names from 3.10 onwards - https://docs.python.org/3/library/sys.html#sys.stdlib_module_names

There's a couple of courses of action we could/should take:

• ask to adopt stdlib-list (or fork and publish under a new name? 👎 )
• leave it alone as a point-in-time artifact, and update our code to also use sys.stdlib_module_names
• something else?

[dstufft]

The reason we don't use sys.stdlid_module_names is that only tells you the stdlib module names in the version of Python you're currently running under. We were using stdlib-list to get the stdlib module names for every version of Python, independently of what version we're running on.

I think that leaves out using sys.stdlib_module_names?

We could probably adopt/fork stdlib-list I guess? Or drop the dependency and make the list ourselves internally inside of Warehouse.

[di]

+1 to attempting to adopt it. I think sys. stdlib_module_names would only tell us the stdlib names under the version we're running, right?

[miketheman]

The reason we don't use sys.stdlid_module_names ...

Gonna guess the reason we don't use it is because it didn't exist when we wrote it in #2409 😉

I think that leaves out using sys.stdlib_module_names?

Likely, unless they change its behavior.

I think sys. stdlib_module_names would only tell us the stdlib names under the version we're running, right?

Yes, that's correct. Part of me wonders if that's more suited for adding to stdlib-list running under newer Python versions to generate the values in some CI.

Another thought is to make isort a runtime dependency, and use the values it has: https://github.com/PyCQA/isort/tree/main/isort/stdlibs

>>> from isort.stdlibs.all import stdlib
>>> stdlib
{'cfmfile', 'sqlite3', 'asyncio', 'rexec', 'Queue', 'fl', 'SUNAUDIODEV', 'poplib', 'quopri', 'symbol', 'numbers', 'abc', 'array', 'Tix', 'datetime', 'commands', 'importlib', 'fileinput', 'types', 'getopt', 'linecache', 'tarfile', 'collections', 'FL', 'copy_reg', 'macpath', 'unittest', 'socket', 'sched', 'concurrent', 'errno', 'ic', 'builtins', 'faulthandler', 're', 'plistlib', 'binascii', 'encodings', 'tabnanny', 'symtable', 'sre_compile', 'timeit', 'dumbdbm', 'zipimport', 'UserString', 'secrets', 'imaplib', 'imp', 'ctypes', 'wsgiref', 'aetools', 'imageop', 'urllib2', 'Bastion', 'compileall', 'sre', 'lzma', 'urllib', 'email', 'traceback', 'codecs', 'gettext', 'inspect', '_ast', 'idlelib', 'formatter', 'pathlib', 'distutils', 'htmlentitydefs', 'ssl', 'xdrlib', 'aifc', 'mutex', 'codeop', 'uuid', 'whichdb', 'venv', 'popen2', 'gl', 'tracemalloc', 'xmlrpc', 'gensuitemodule', 'al', 'atexit', 'code', 'winsound', 'dl', 'logging', 'Carbon', 'Cookie', 'contextvars', 'imgfile', 'mimetools', 'compiler', 'bz2', '_thread', 'applesingle', 'anydbm', 'zipapp', 'icopen', 'termios', 'tkinter', 'select', 'dummy_threading', 'future_builtins', 'hotshot', 'ScrolledText', 'ast', 'sunau', 'UserDict', 'flp', 'jpeg', 'stringprep', 'threading', 'multifile', '__builtin__', 'bdb', 'cmath', 'zlib', 'heapq', 'posixpath', 'io', 'findertools', 'exceptions', 'ensurepip', 'pipes', 'md5', 'pickletools', '_winreg', 'BaseHTTPServer', 'buildtools', 'ossaudiodev', 'typing', 'site', 'optparse', 'turtledemo', 'DEVICE', 'decimal', 'hashlib', 'subprocess', 'sunaudiodev', 'reprlib', 'thread', 'pprint', 'mhlib', 'configparser', 'operator', 'binhex', 'chunk', 'nntplib', 'mimify', 'shlex', 'DocXMLRPCServer', 'posix', 'SimpleHTTPServer', 'Nav', 'msvcrt', 'random', 'modulefinder', 'AL', 'ntpath', 'calendar', 'cPickle', 'mailcap', 'CGIHTTPServer', 'fnmatch', 'msilib', 'cookielib', 'httplib', 'difflib', 'ConfigParser', 'MimeWriter', 'syslog', 'macerrors', 'html', 'uu', 'telnetlib', 'multiprocessing', 'warnings', 'contextlib', 'marshal', 'shutil', 'videoreader', 'sha', 'signal', 'asynchat', 'htmllib', 'profile', 'lib2to3', 'winreg', 'getpass', 'sgmllib', 'PixMapWrapper', 'colorsys', 'smtplib', 'nis', 'mimetypes', 'sndhdr', 'statvfs', 'fcntl', 'grp', 'gdbm', 'textwrap', 'UserList', 'xml', 'cStringIO', 'ipaddress', 'sysconfig', 'gc', 'FrameWork', 'copy', 'Tkinter', 'bisect', 'dummy_thread', 'dataclasses', 'mailbox', 'doctest', 'GL', 'functools', 'new', 'filecmp', 'cgi', 'parser', 'csv', 'rlcompleter', 'http', 'cd', 'ftplib', 'SimpleXMLRPCServer', 'zoneinfo', 'string', 'locale', 'json', 'posixfile', 'queue', 'bsddb', 'dircache', 'cmd', 'audioop', 'pydoc', 'W', 'MiniAEFrame', 'spwd', 'aepack', 'curses', 'zipfile', 'runpy', 'copyreg', 'os', 'smtpd', 'MacOS', 'py_compile', 'shelve', 'math', 'pickle', 'wave', 'argparse', 'macresource', 'pstats', 'xmlrpclib', 'sre_parse', 'urlparse', 'tomllib', 'mmap', 'turtle', 'graphlib', 'autoGIL', 'rfc822', 'weakref', 'sre_constants', 'trace', 'enum', 'gzip', 'resource', 'cgitb', 'unicodedata', 'dbm', 'tokenize', 'crypt', 'pkgutil', 'glob', 'tempfile', 'itertools', 'SocketServer', 'stat', 'cProfile', 'readline', 'fm', 'platform', 'token', 'StringIO', 'fpformat', 'imputil', 'dbhash', 'time', 'tty', 'ColorPicker', 'dis', 'webbrowser', 'selectors', 'hmac', 'EasyDialogs', 'asyncore', 'sets', 'ttk', 'struct', 'pdb', '_dummy_thread', 'robotparser', 'macostools', 'aetypes', 'statistics', 'pyclbr', 'socketserver', 'base64', 'keyword', 'test', 'HTMLParser', 'netrc', 'fpectl', 'fractions', 'imghdr', 'pwd', 'sys', 'user', 'pty'}

[woodruffw]

Yes, that's correct. Part of me wonders if that's more suited for adding to stdlib-list running under newer Python versions to generate the values in some CI.

That's how I'd do it 🙂 -- if you need an extra person to help maintain it, I've done a very similar thing with CPython's stable ABI information: https://github.com/woodruffw/abi3info

[dstufft]

All I'm hearing is @woodruffw is offering to take ownership of stdlib-list

[di]

@woodruffw, wanna file an issue there proposing we transfer this to https://github.com/pypi and take over maintainership?

[woodruffw]

All I'm hearing is @woodruffw is offering to take ownership of stdlib-list

wanna file an issue there proposing we transfer this to https://github.com/pypi and take over maintainership?

Sure, doing now!

[woodruffw]

Ah, spoke too soon: the repo is archived, so I can't open an issue.

I'll try contacting the maintainer directly.

Edit: Reached out.

[woodruffw]

I've cut a 0.9.0 release of stdlib-list that includes lists up to and including 3.11, so I think this can be considered resolved! We should probably update to that version.

[miketheman]

Thanks @woodruffw !!