coreweave / ml-containers

MIT License
19 stars 3 forks source link

fix(torch-extras): Build `torch-extras` off of the most recent `torch` images #45

Closed Eta0 closed 10 months ago

Eta0 commented 10 months ago

torch-extras / torch mismatch

This change fixes a bug that was causing torch-extras to sometimes build off of outdated torch images in our CI pipeline.

The bug was because ghcr.io's implementation of the /v2/<package>/tags/list registry endpoint, which we use to determine the latest torch image to build from, automatically switches to paginated results unless otherwise requested once the result set is large enough, instead of only switching when requested. Since the results are ordered chronologically (oldest to newest), this means the last result—which would normally be the newest torch image—was not actually the newest torch image.

A more stable fix for this in the future would be to properly parse the paginated results, but since it accepts n=100000 (asking for a page size of 100000) without any warnings, this seems to work fine too.