apache / tvm

Open deep learning compiler stack for cpu, gpu and specialized accelerators
https://tvm.apache.org/
Apache License 2.0
11.61k stars 3.44k forks source link

[CI Problem] Nightly docker re-build validations use different commits for CMake build and testing #9741

Open manupak opened 2 years ago

manupak commented 2 years ago

Where ?

https://ci.tlcpack.ai/job/docker-images-ci/job/daily-docker-image-validate/45/

calls another Job that shows the issue:

https://ci.tlcpack.ai/blue/organizations/jenkins/docker-images-ci%2Fdocker-image-run-tests/detail/docker-image-run-tests/45/pipeline/269

What ?

The above job uses different commits on main for building tvm and testing tvm as follows :

The above seems to be built using :

Checking out Revision d1dafbd6d050ddcd673f567c5ff720a6b6419cfe (refs/remotes/origin/main)
git rev-parse refs/remotes/origin/main^{commit} # timeout=10
git rev-parse refs/remotes/origin/origin/main^{commit} # timeout=10
Commit message: "[CUTLASS] More robust support for pattern matching and alignment (#9698)"

and tested using

 > git --version # timeout=10
 > git fetch --tags --force --progress -- https://github.com/apache/tvm +refs/heads/:refs/remotes/origin/ # timeout=10
Checking out Revision b4d595c518d6e1b0f67a683072be7d3e237c0acc (refs/remotes/origin/main)
Commit message: "Add CMake summary (#9696)"

So the following is a snapshot of the tree:

commit b4d595c518d6e1b0f67a683072be7d3e237c0acc
Author: driazati <9407960+driazati@users.noreply.github.com>
Date:   Mon Dec 13 21:45:02 2021 -0800

 Add CMake summary (#9696)

commit 69cae0b104fe2070fcff8710d562b7e22046cbd9
Author: masahi <masahi129@gmail.com>
Date:   Tue Dec 14 13:53:47 2021 +0900

  [Relay, BYOC] Make constant binding in PartitionGraph optional (#9721)

commit d1dafbd6d050ddcd673f567c5ff720a6b6419cfe
Author: masahi <masahi129@gmail.com>
Date:   Tue Dec 14 12:58:23 2021 +0900

[CUTLASS] More robust support for pattern matching and alignment (#9698)

Therefore this causes the validate job to fail because the testing uses new commit that were not built with.

manupak commented 2 years ago

cc : @leandron @areusch @tqchen @mbs-octoml @grant-arm