flashlight / text

Text utilities, including beam search decoding, tokenizing, and more, built for use in Flashlight.
MIT License
64 stars 14 forks source link

Fix source distribution build and upgrade CI baselines #83

Closed jacobkahn closed 6 months ago

jacobkahn commented 6 months ago

Summary

Source distributions never had the right version file -- they'd always fail. bindings/python/.../version.py is always written out when building an sdist, so try to use it if it's available.

Add a CI baseline to try using the sdist/install with KenLM

Also move to Jammy for baseline Linux images since most Focal ones are being deprecate and upgrade GHA components depending on old Node versions. v4 of these actions breaks artifact storage with dup names/doesn't merge automatically; fix this. Contents of dist post-upload by wheel builder/sdist becomes

-rw-r--r-- 1 runner docker   60364 Feb 28 18:47 flashlight-text-0.0.6.dev336.tar.gz
-rw-r--r-- 1 runner docker 10395[8](https://github.com/flashlight/text/actions/runs/8085500728/job/22093301914#step:3:9)5 Feb 28 18:47 flashlight_text-0.0.6.dev336-cp312-cp312-macosx_10_[9](https://github.com/flashlight/text/actions/runs/8085500728/job/22093301914#step:3:10)_x86_64.whl
-rw-r--r-- 1 runner docker 1304785 Feb 28 18:47 flashlight_text-0.0.6.dev336-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
-rw-r--r-- 1 runner docker 1683721 Feb 28 18:47 flashlight_text-0.0.6.dev336-cp312-cp312-musllinux_1_1_x86_64.whl
-rw-r--r-- 1 runner docker  496932 Feb 28 18:47 flashlight_text-0.0.6.dev336-cp312-cp312-win_amd64.whl

Test Plan (required)

CI + local test

pipx run build --sdist
# move tarball out of git repo where commands to fetch hashes failed
cd ${SOURCE_DIST_DIR} # after untarring
USE_KENLM=0 pip install -v .

Checklist

facebook-github-bot commented 6 months ago

@jacobkahn has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 6 months ago

@jacobkahn merged this pull request in flashlight/text@bbe9e3c201f5c9c3f3c0d553f0ea73af5e0a5209.