Tracking issue for cleaning up python lib formulae

[cho-m]

A dedicated tracker for any tasks to handle based on decision in #157500.

For Homebrew/core:

• [ ] Triage python library formulae:
  • [x] Remove python library formulae that do not meet our acceptable requirements and have low number of on-request installs
  • [ ] Deprecate python library formulae that do not meet our acceptable requirements and have high number of on-request installs (or have been around for a while in Homebrew/core). We can do a 3mo deprecation + 3mo disable period.
  • [ ] Convert/revert Python application formulae that are installed to prefix to instead use virtualenvs
  • [ ] Document anything remaining as approved exceptions

Extras for Homebrew/brew:

• [ ] Improve audit checks, particularly for --new formulae, e.g. https://github.com/Homebrew/brew/pull/16663
• [ ] Improve brew doctor and --ignore-dependencies "recommendation" for removed formulae to avoid users blindly running commands
• [ ] Optional: Maybe perform automatic "migration" to new rebuild/revision for removed formulae (which would also need to handle tab of recursive dependents)

---

Quick table based on 90-day install-on-request analytics data for formulae that may expose libs in HOMEBREW_PREFIX/lib. Also added whether the formula provides any files in bin and non-pure python libs.

Formula	90-day installs	90-day rank	# direct usage	Comment	bin/* cmd?	Ext libs
pycparser	38,795	136	4	dependency of cryptography	❎	❎
numpy	38,584	138	44	keep, https://github.com/Homebrew/brew/pull/16662	✅︎	✅︎
cffi	33,997	154	7	dependency of cryptography	❎	✅︎
mercurial	16,247	273	4	keep	✅︎	✅︎
pillow	14,684	293	21	keep	❎	✅︎
pyqt@5	10,765	352	5	keep	✅︎	✅︎
pyqt	10,336	363	3	keep	✅︎	✅︎
python-setuptools	7,381	463	87		❎	❎
sip	6,932	477	2		✅︎	❎
cryptography	6,285	511	69	keep, https://github.com/Homebrew/brew/pull/16662	❎	✅︎
scipy	6,248	512	4	keep, https://github.com/Homebrew/brew/pull/16662	❎	✅︎
six	2,887	806	4		❎	❎
python-matplotlib	2,823	820	0		❎	✅︎
python-tabulate	1,771	1083	0	keep #175440	✅︎	❎
certifi	1,707	1109	164	keep, https://github.com/Homebrew/brew/pull/16662	❎	❎
python-lxml	1,648	1135	0	deprecate, #171823	❎	✅︎
python-packaging	1,547	1189	1	deprecate #176511	❎	❎
pyyaml	1,319	1293	3	deprecate #176591	❎	✅︎
python-markdown	1,293	1313	0	keep #175442, decide on rename	✅︎	❎
python-build	1,564	1176	1	reverted to venv, #165713 Decide if rename?	✅︎	❎
pymol	964	1577	0		✅︎	✅︎
pygit2	904	1630	0		❎	✅︎
pymupdf	824	1717	1		✅︎	✅︎
python-argcomplete	813	1731	0		✅︎	❎
lit	678	1908	1		✅︎	❎
wxpython	639	1962	0		✅︎	✅︎
python-ply	559	2101	0	deprecate, #171825	❎	❎
python-chardet	494	2260	0	keep #175446	✅︎	❎
pyqt-builder	214	3530	3		✅︎	❎
ly	58	6025	0	keep bin, decide on lib.	✅︎	❎

Table for completed tasks (i.e. merged PRs), excluding removals:

Formula	90-day installs	90-day rank	# direct usage	Decision/PR	bin/* cmd?	Ext libs
docutils	32,666	162	10	reverted to venv, #168183	✅︎	❎
python-cryptography	24,100	205	0	renamed to cryptography	n/a	n/a
python-requests	4,118	650	0	deprecated, #166056	❎	❎
python-certifi	2,262	928	0	renamed to certifi	n/a	n/a
python-pytz	1,104	1447	0	deprecated, #168071	❎	❎
python-typing-extensions	1,033	1499	1	deprecated, #168114	❎	❎
python-urllib3	877	1667	1	deprecated, #166056	❎	❎
asciidoc	696	1886	25	reverted to venv, #165317	✅︎	❎
python-dateutil	644	1959	0	deprecated, #168071	❎	❎
python-psutil	603	2019	0	deprecated, #168071	❎	✅︎
python-trove-classifiers	507	2224	0	deprecated, #168181	❎	❎
python-idna	441	2385	1	deprecated, #166056	❎	❎
python-charset-normalizer	218	3503	1	deprecated, #166056	❎	❎

Collapsed table for removed formulae

| Formula | 90-day
installs | 90-day
rank | # direct
usage | Decision/PR | bin/*
cmd? | Ext
libs | | -- | -: | -: | -: | -- | -- | -- | | python-markupsafe | 1,742 | 1093 | 0 | removed, #165568 | n/a | n/a | | python-platformdirs | 447 | 2369 | 0 | removed, #163603 | n/a | n/a | | python-boto3 | 398 | 2511 | 0 | removed, #163229 | n/a | n/a | | python-pyparsing | 389 | 2541 | 0 | removed, #168201 | ❎ | ❎ | | python-jinja | 366 | 2631 | 0 | removed, #165495 | n/a | n/a | | python-hatchling | 339 | 2749 | 0 | removed, #166358 | n/a | n/a | | python-distlib | 247 | 3277 | 0 | removed, #163570 | | meson-python | 235 | 3363 | 0 | removed, #166060 | ❎ | ❎ | | python-setuptools-scm | 179 | 3879 | 0 | removed, #168073 | ❎ | ❎ | | python-flit-core | 174 | 3942 | 0 | removed, #168201 | ❎ | ❎ | | python-click | 171 | 3983 | 0 | removed, #165496 | | python-toml | 170 | 4000 | 0 | removed, #165493 | | python-sympy | 169 | 4010 | 0 | removed, #165494 | | python-mako | 169 | 4016 | 0 | removed, #165568 | | python-pluggy | 149 | 4239 | 0 | removed, #166358 | | python-pycurl | 148 | 4247 | 0 | removed, #164368 | | python-mutagen | 116 | 4715 | 0 | removed, #164365 | | python-pathspec | 114 | 4742 | 0 | removed, #166358 | | python-brotli | 107 | 4869 | 0 | removed, #165112 | | python-attrs | 102 | 4937 | 0 | removed, #163625 | | python-networkx | 95 | 5069 | 0 | removed, #166185 | | python-botocore | 84 | 5306 | 0 | removed, #163229 | | python-magic | 81 | 5374 | 0 | removed, #164158 | | python-filelock | 78 | 5446 | 0 | removed, #164940 | | python-openapi3 | 71 | 5638 | 0 | removed, #163442 | | python-cli-helpers | 70 | 5666 | 0 | removed, #163436 | | python-colorama | 69 | 5694 | 0 | removed, #163437 | | python-distro | 67 | 5750 | 0 | removed, #164160 | | python-cycler | 64 | 5839 | 0 | removed, #166060 | ❎ | ❎ | | python-abseil | 62 | 5908 | 0 | removed, #163343 | | python-regex | 61 | 5918 | 0 | removed, #164146 | | python-docopt | 55 | 6100 | 0 | removed, #164149 | | python-pbr | 55 | 6134 | 0 | removed, #164154 | | python-configargparse | 52 | 6242 | 0 | removed, #163438 | | python-hatch-vcs | 52 | 6243 | 0 | removed, #164144 | | python-json5 | 52 | 6244 | 0 | removed, #163642 | | python-xlsxwriter | 51 | 6288 | 0 | removed, #163646 | | python-kiwisolver | 49 | 6382 | 0 | removed, #166060 | ❎ | ✅︎ | | python-requests-oauthlib | 48 | 6421 | 0 | removed, #163590 | | python-hatch-fancy-pypi-readme | 47 | 6462 | 0 | removed, #164085 | | python-prompt-toolkit | 47 | 6463 | 0 | removed, #163576 | | python-termcolor | 46 | 6511 | 0 | removed, #163378 | | python-rich | 45 | 6557 | 0 | removed, #163623 | | python-asn1crypto | 43 | 6655 | 0 | removed, #163363 | | python-oauthlib | 43 | 6656 | 0 | removed, #163712 | | python-cachetools | 41 | 6745 | 0 | removed, #163425 | | python-pyproject-hooks | 39 | 6835 | 0 | removed, #166183 | | python-websocket-client | 39 | 6836 | 0 | removed, #163645 | | python-dicttoxml | 32 | 7219 | 0 | removed, #163567 | | python-mpmath | 32 | 7220 | 0 | removed, #165494 | | python-markdown-it-py | 30 | 7362 | 0 | removed, #164086 | | python-anytree | 29 | 7431 | 0 | removed, #163424 | | python-wcwidth | 28 | 7489 | 0 | removed, #163592 | | python-msgpack | 27 | 7555 | 0 | removed, #164148 | | python-jmespath | 13 | 9027 | 0 | removed, #163229 | | python-configobj | 7 | 10540 | 0 | removed, #163565 | | python-mdurl | 7 | 10541 | 0 | removed, #164086 | | python-s3transfer | 6 | 10927 | 0 | removed, #163229 |

[carlocab]

Should probably keep pygit2 -- it links with libgit2, and vendoring it into its dependents leads to really long build times (due to revision bumps) when libgit2 is updated.

[LecrisUT]

I've opened an issue to point out a potential problems with the build process, specifically affecting python projects which have setuptools-scm as a dependency