search

Confirm-Solutions / confirmasaurus

3 stars 0 forks source link

Track poetry issues with JAX dependency #168

Closed tbenthompson closed 1 year ago

tbenthompson commented 1 year ago

Issue I posted that led me to the issues below: https://github.com/python-poetry/poetry/issues/6956

Issue discussing the core of the problem: https://github.com/python-poetry/poetry/issues/5506

PR with a proposed fix referenced in 5506 above: https://github.com/python-poetry/poetry/pull/6969

tbenthompson commented 1 year ago

There's also the possibility of this being solved on the JAX side: https://github.com/google/jax/issues/5410

tbenthompson commented 1 year ago

Also can be solved by just referencing package wheels URLs directly like in: https://github.com/xeb/dalle-mini-tools/pull/3/files

tbenthompson commented 1 year ago

Referencing package wheel URLs directly is sad because we would have to list out every version like:

jaxlib = [
    { platform = 'darwin', python="3.8", url = "https://storage.googleapis.com/jax-releases/mac/jaxlib-0.4.4-cp38-cp38-macosx_11_0_arm64.whl" },
    { platform = 'darwin', python="3.9", url = "https://storage.googleapis.com/jax-releases/mac/jaxlib-0.4.4-cp39-cp39-macosx_11_0_arm64.whl" },
    { platform = 'darwin', python="3.10", url = "https://storage.googleapis.com/jax-releases/mac/jaxlib-0.4.4-cp310-cp310-macosx_11_0_arm64.whl" },
    { platform = 'darwin', python="3.11", url = "https://storage.googleapis.com/jax-releases/mac/jaxlib-0.4.4-cp311-cp311-macosx_11_0_arm64.whl" },
    { platform = 'linux', python="3.8", url = "https://storage.googleapis.com/jax-releases/cuda11/jaxlib-0.4.4+cuda11.cudnn86-cp38-cp38-manylinux2014_x86_64.whl" },
    { platform = 'linux', python="3.9", url = "https://storage.googleapis.com/jax-releases/cuda11/jaxlib-0.4.4+cuda11.cudnn86-cp39-cp39-manylinux2014_x86_64.whl" },
    { platform = 'linux', python="3.10", url = "https://storage.googleapis.com/jax-releases/cuda11/jaxlib-0.4.4+cuda11.cudnn86-cp310-cp310-manylinux2014_x86_64.whl" },
    { platform = 'linux', python="3.11", url = "https://storage.googleapis.com/jax-releases/cuda11/jaxlib-0.4.4+cuda11.cudnn86-cp311-cp311-manylinux2014_x86_64.whl" },
]

This is doubly sad:

  1. Kinda annoying
  2. poetry lock goes really slow because it re-downloads all these packages every time!
tbenthompson commented 1 year ago

Closing this. Not worth ongoing attention and mostly closed by https://github.com/Confirm-Solutions/confirmasaurus/pull/297