microsoft / hummingbird

Hummingbird compiles trained ML models into tensor computation for faster inference.
MIT License
3.32k stars 274 forks source link

TVM MacOS pipeline intermittent failures #715

Closed ksaur closed 1 year ago

ksaur commented 1 year ago

Recently we've been seeing failures like this and this:

Processing ./dist/tvm-0.10.0-cp38-cp38-macosx_11_0_x86_64.whl
ERROR: Cannot install tvm 0.10.0 (from /Users/runner/work/hummingbird/tvm/python/dist/tvm-0.10.0-cp38-cp38-macosx_10_15_x86_64.whl) and tvm 0.10.0 (from /Users/runner/work/hummingbird/tvm/python/dist/tvm-0.10.0-cp38-cp38-macosx_11_0_x86_64.whl) because these package versions have conflicting dependencies.
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

The conflict is caused by:
    The user requested tvm 0.10.0 (from /Users/runner/work/hummingbird/tvm/python/dist/tvm-0.10.0-cp38-cp38-macosx_10_15_x86_64.whl)
    The user requested tvm 0.10.0 (from /Users/runner/work/hummingbird/tvm/python/dist/tvm-0.10.0-cp38-cp38-macosx_11_0_x86_64.whl)

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

This seems to happen every other time, todo: investigate. cc: @mshr-h , have you seen this other places?

mshr-h commented 1 year ago

No, I haven't seen any similar message.

mshr-h commented 1 year ago

@ksaur Can you try to remove the GHA cache and run the pipeline again?

I ran again the pipeline and found that there are multiple whl package in python/dist. The 10_15 whl was created in the previous pipeline execution and restored from the cache. Recent pipeline executions create 11_0 whl, then try to install them at the same time which caused the dependency conflict.

-rw-r--r-- 1 runner staff 42055067 Feb 24 22:03 tvm-0.10.0-cp38-cp38-macosx_10_15_x86_64.whl -rw-r--r-- 1 runner staff 42055470 Jun 20 02:39 tvm-0.10.0-cp38-cp38-macosx_11_0_x86_64.whl

ksaur commented 1 year ago

Done! Crossing fingers!