search

google / benchmark

A microbenchmark support library
Apache License 2.0
8.89k stars 1.61k forks source link

Latest wheel delivery problems #1840

Open nicholasjng opened 3 weeks ago

nicholasjng commented 3 weeks ago

Good morning. I saw the latest wheel errors when I was browsing the v1.9.0 job logs after you mentioned that trusted publishing had been set up.

This is just to let you know that I'm currently working on a fix. Logs look good (although only applied on CPython 3.10-3.12): https://github.com/nicholasjng/benchmark/actions/runs/10448608191/job/28929367744

I'd welcome your input on a few things.

1) Should we move to newer Pythons and drop e.g. Python 3.8? I don't know about user statistics inside or outside of Google, but EOL is coming up in October (https://devguide.python.org/versions/#supported-versions), and a good number of established projects have moved to later Python versions (e.g. NumPy, SciPy require Python 3.10+). This would also cut down build times, see the above job (restricting to 3.10+ puts wheel builds in the order of <7 minutes, disregarding Linux aarch64). 2) I'll revert to trusted publishing in the job config if you give me the green light, since you mentioned that it's set up now in the issue I raised. 3) Should we drop setuptools_scm again? The proposition (zero-config versioning from git) is great, but in practice we've had so many problems with wheel delivery that it has turned out unfavorably for us (the last 3 versions are missing on PyPI).

The main problem is that without hacks, it's impossible to publish a version to PyPI after the fact if the release wheel job fails, since all subsequent versions are marked as dirty until the next tag. This is technically how it's supposed to be, but the result has been missing wheels. Publishing e.g. from main is its own topic, but better than not publishing at all imo.

Thanks! (I'm also working on new bindings for the memory and profiler managers, and I picked up the flag enum issue in nanobind, which would fix the Counter::Flags issue from before)

dmah42 commented 3 weeks ago

i saw the PR before i saw this.

  1. sgtm as long as it's EOL
  2. great!
  3. i defer to your judgement.