search

viash-io / viash

script + metadata = standalone component
https://viash.io
GNU General Public License v3.0
39 stars 2 forks source link

[BUG] organization from package config is not applied to the docker container name when running tests #766

Closed DriesSchaumont closed 3 months ago

DriesSchaumont commented 3 months ago

What happened?

When setting organization: my_org in the package config and running tests, the organization is not added to the docker image name.

Steps to reproduce

docker rmi -f docker.io/openpipelines-bio/dummycomponent/myscript:test docker.io/dummycomponent/myscript:test
if [[ -d /tmp/test_bug ]]; then
  rm -r /tmp/test_bug
fi
rm -rf /tmp/test_bug
mkdir -p /tmp/test_bug
cd /tmp/test_bug
cat > _viash.yaml << 'HERE'
viash_version: 0.9.0-RC7

source: src
target: target
organization: "openpipelines-bio"
HERE

mkdir -p src/dummycomponent

cat > src/dummycomponent/config.vsh.yaml << 'HERE'

name: myscript
namespace: "dummycomponent"
test_resources:
  - type: python_script
    path: test_script.py
    text: |
      raise ValueError("TEST FAIL")

engines:
  - type: docker
    image: 'python:latest'

runners:
  - type: executable
HERE
viash test ./src/dummycomponent/config.vsh.yaml
Running tests in temporary directory: '/tmp/viash_test_myscript_8289795475484293803'
====================================================================
+/tmp/viash_test_myscript_8289795475484293803/build_engine_environment/myscript ---verbosity 6 ---setup cachedbuild ---engine docker
[notice] Building container 'openpipelines-bio/dummycomponent/myscript:test' with Dockerfile
[info] docker build -t 'openpipelines-bio/dummycomponent/myscript:test'  '/tmp/viash_test_myscript_8289795475484293803/build_engine_environment' -f '/tmp/viash_test_myscript_8289795475484293803/build_engine_environment/tmp/dockerbuild-myscript-wObuwQ/Dockerfile'
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 292B done
#1 DONE 0.0s

#2 [internal] load metadata for docker.io/library/python:latest
#2 DONE 0.8s

#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [1/1] FROM docker.io/library/python:latest@sha256:e3d5b6f95ce66923b5e48a06ee5755abb097de96a8617c3f2f7d431d48e63d35
#4 resolve docker.io/library/python:latest@sha256:e3d5b6f95ce66923b5e48a06ee5755abb097de96a8617c3f2f7d431d48e63d35 0.0s done
#4 sha256:e3d5b6f95ce66923b5e48a06ee5755abb097de96a8617c3f2f7d431d48e63d35 9.72kB / 9.72kB done
#4 sha256:0218518c77bef087f9111b988785f0cef566c072474ee5324c7bb57faf07d93d 7.05kB / 7.05kB done
#4 sha256:903681d87777d28dc56866a07a2774c3fd5bf65fd734b24c9d0ecd9a13c9f636 0B / 49.55MB 0.1s
#4 sha256:6ed93aa58a52c9abc1ee472f1ac74b73d3adcccc2c30744498fd5f14f3f5d22c 0B / 64.14MB 0.1s
#4 sha256:2daa22a098caf2780fc451078e9c855da767d5612253171f6281bf320166376c 2.52kB / 2.52kB done
#4 sha256:3cbbe86a28c2f6b3c3e0e8c6dcfba369e1ea656cf8daf69be789e0fe2105982b 0B / 24.05MB 0.1s
#4 sha256:903681d87777d28dc56866a07a2774c3fd5bf65fd734b24c9d0ecd9a13c9f636 13.63MB / 49.55MB 0.3s
#4 sha256:903681d87777d28dc56866a07a2774c3fd5bf65fd734b24c9d0ecd9a13c9f636 18.87MB / 49.55MB 0.4s
#4 sha256:3cbbe86a28c2f6b3c3e0e8c6dcfba369e1ea656cf8daf69be789e0fe2105982b 6.29MB / 24.05MB 0.4s
#4 sha256:903681d87777d28dc56866a07a2774c3fd5bf65fd734b24c9d0ecd9a13c9f636 24.12MB / 49.55MB 0.5s
#4 sha256:3cbbe86a28c2f6b3c3e0e8c6dcfba369e1ea656cf8daf69be789e0fe2105982b 10.49MB / 24.05MB 0.5s
#4 sha256:903681d87777d28dc56866a07a2774c3fd5bf65fd734b24c9d0ecd9a13c9f636 30.41MB / 49.55MB 0.7s
#4 sha256:6ed93aa58a52c9abc1ee472f1ac74b73d3adcccc2c30744498fd5f14f3f5d22c 10.49MB / 64.14MB 0.7s
#4 sha256:3cbbe86a28c2f6b3c3e0e8c6dcfba369e1ea656cf8daf69be789e0fe2105982b 15.73MB / 24.05MB 0.7s
#4 sha256:903681d87777d28dc56866a07a2774c3fd5bf65fd734b24c9d0ecd9a13c9f636 37.75MB / 49.55MB 0.9s
#4 sha256:6ed93aa58a52c9abc1ee472f1ac74b73d3adcccc2c30744498fd5f14f3f5d22c 22.02MB / 64.14MB 0.9s
#4 sha256:3cbbe86a28c2f6b3c3e0e8c6dcfba369e1ea656cf8daf69be789e0fe2105982b 19.92MB / 24.05MB 0.9s
#4 sha256:903681d87777d28dc56866a07a2774c3fd5bf65fd734b24c9d0ecd9a13c9f636 47.19MB / 49.55MB 1.1s
#4 sha256:6ed93aa58a52c9abc1ee472f1ac74b73d3adcccc2c30744498fd5f14f3f5d22c 34.60MB / 64.14MB 1.1s
#4 sha256:3cbbe86a28c2f6b3c3e0e8c6dcfba369e1ea656cf8daf69be789e0fe2105982b 24.05MB / 24.05MB 1.1s done
#4 sha256:787c78da43830be6d988d34c7ee091f98d828516ce5478ca10a4933d655191bf 0B / 211.24MB 1.1s
#4 sha256:903681d87777d28dc56866a07a2774c3fd5bf65fd734b24c9d0ecd9a13c9f636 49.55MB / 49.55MB 1.1s done
#4 sha256:6ed93aa58a52c9abc1ee472f1ac74b73d3adcccc2c30744498fd5f14f3f5d22c 40.89MB / 64.14MB 1.2s
#4 extracting sha256:903681d87777d28dc56866a07a2774c3fd5bf65fd734b24c9d0ecd9a13c9f636
#4 sha256:239e5f5bf9b4aa2c2b1dea741d793f4120e031b92f8553040ad2a1f63ae3a195 0B / 6.16MB 1.2s
#4 sha256:6ed93aa58a52c9abc1ee472f1ac74b73d3adcccc2c30744498fd5f14f3f5d22c 45.09MB / 64.14MB 1.3s
#4 sha256:6ed93aa58a52c9abc1ee472f1ac74b73d3adcccc2c30744498fd5f14f3f5d22c 49.28MB / 64.14MB 1.4s
#4 sha256:239e5f5bf9b4aa2c2b1dea741d793f4120e031b92f8553040ad2a1f63ae3a195 1.05MB / 6.16MB 1.4s
#4 sha256:6ed93aa58a52c9abc1ee472f1ac74b73d3adcccc2c30744498fd5f14f3f5d22c 54.53MB / 64.14MB 1.5s
#4 sha256:787c78da43830be6d988d34c7ee091f98d828516ce5478ca10a4933d655191bf 14.68MB / 211.24MB 1.5s
#4 sha256:239e5f5bf9b4aa2c2b1dea741d793f4120e031b92f8553040ad2a1f63ae3a195 3.15MB / 6.16MB 1.5s
#4 sha256:6ed93aa58a52c9abc1ee472f1ac74b73d3adcccc2c30744498fd5f14f3f5d22c 58.72MB / 64.14MB 1.6s
#4 sha256:239e5f5bf9b4aa2c2b1dea741d793f4120e031b92f8553040ad2a1f63ae3a195 5.58MB / 6.16MB 1.6s
#4 sha256:6ed93aa58a52c9abc1ee472f1ac74b73d3adcccc2c30744498fd5f14f3f5d22c 64.14MB / 64.14MB 1.7s done
#4 sha256:787c78da43830be6d988d34c7ee091f98d828516ce5478ca10a4933d655191bf 31.46MB / 211.24MB 1.8s
#4 sha256:239e5f5bf9b4aa2c2b1dea741d793f4120e031b92f8553040ad2a1f63ae3a195 6.16MB / 6.16MB 1.6s done
#4 sha256:ed3e311c15d33d38a3dc936aefadad81e4827f6afd4807956f3d185d3e1e70fe 2.10MB / 22.44MB 1.8s
#4 sha256:f445a5851dabc97591ba4389a58fda4938fcc19955d92ebdf55951ea111fbe1f 0B / 232B 1.8s
#4 sha256:ed3e311c15d33d38a3dc936aefadad81e4827f6afd4807956f3d185d3e1e70fe 8.39MB / 22.44MB 1.9s
#4 sha256:f445a5851dabc97591ba4389a58fda4938fcc19955d92ebdf55951ea111fbe1f 232B / 232B 1.8s done
#4 sha256:815b2c9ae66da3c01de8f54110cebf074229f3ae1c9df06dcc94f4859d4ccd47 0B / 3.91MB 1.9s
#4 sha256:787c78da43830be6d988d34c7ee091f98d828516ce5478ca10a4933d655191bf 42.99MB / 211.24MB 2.0s
#4 extracting sha256:903681d87777d28dc56866a07a2774c3fd5bf65fd734b24c9d0ecd9a13c9f636 0.8s done
#4 sha256:ed3e311c15d33d38a3dc936aefadad81e4827f6afd4807956f3d185d3e1e70fe 13.63MB / 22.44MB 2.0s
#4 extracting sha256:3cbbe86a28c2f6b3c3e0e8c6dcfba369e1ea656cf8daf69be789e0fe2105982b
#4 sha256:ed3e311c15d33d38a3dc936aefadad81e4827f6afd4807956f3d185d3e1e70fe 16.78MB / 22.44MB 2.1s
#4 sha256:815b2c9ae66da3c01de8f54110cebf074229f3ae1c9df06dcc94f4859d4ccd47 1.05MB / 3.91MB 2.1s
#4 sha256:787c78da43830be6d988d34c7ee091f98d828516ce5478ca10a4933d655191bf 54.53MB / 211.24MB 2.2s
#4 sha256:ed3e311c15d33d38a3dc936aefadad81e4827f6afd4807956f3d185d3e1e70fe 20.97MB / 22.44MB 2.2s
#4 sha256:815b2c9ae66da3c01de8f54110cebf074229f3ae1c9df06dcc94f4859d4ccd47 3.91MB / 3.91MB 2.2s
#4 sha256:787c78da43830be6d988d34c7ee091f98d828516ce5478ca10a4933d655191bf 72.35MB / 211.24MB 2.4s
#4 sha256:ed3e311c15d33d38a3dc936aefadad81e4827f6afd4807956f3d185d3e1e70fe 22.44MB / 22.44MB 2.2s done
#4 sha256:815b2c9ae66da3c01de8f54110cebf074229f3ae1c9df06dcc94f4859d4ccd47 3.91MB / 3.91MB 2.2s done
#4 extracting sha256:3cbbe86a28c2f6b3c3e0e8c6dcfba369e1ea656cf8daf69be789e0fe2105982b 0.2s done
#4 extracting sha256:6ed93aa58a52c9abc1ee472f1ac74b73d3adcccc2c30744498fd5f14f3f5d22c
#4 sha256:787c78da43830be6d988d34c7ee091f98d828516ce5478ca10a4933d655191bf 89.13MB / 211.24MB 2.6s
#4 sha256:787c78da43830be6d988d34c7ee091f98d828516ce5478ca10a4933d655191bf 107.18MB / 211.24MB 2.8s
#4 sha256:787c78da43830be6d988d34c7ee091f98d828516ce5478ca10a4933d655191bf 126.88MB / 211.24MB 3.0s
#4 sha256:787c78da43830be6d988d34c7ee091f98d828516ce5478ca10a4933d655191bf 158.33MB / 211.24MB 3.3s
#4 sha256:787c78da43830be6d988d34c7ee091f98d828516ce5478ca10a4933d655191bf 180.36MB / 211.24MB 3.5s
#4 extracting sha256:6ed93aa58a52c9abc1ee472f1ac74b73d3adcccc2c30744498fd5f14f3f5d22c 1.0s done
#4 sha256:787c78da43830be6d988d34c7ee091f98d828516ce5478ca10a4933d655191bf 205.52MB / 211.24MB 3.9s
#4 sha256:787c78da43830be6d988d34c7ee091f98d828516ce5478ca10a4933d655191bf 211.24MB / 211.24MB 4.0s done
#4 extracting sha256:787c78da43830be6d988d34c7ee091f98d828516ce5478ca10a4933d655191bf 0.1s
#4 extracting sha256:787c78da43830be6d988d34c7ee091f98d828516ce5478ca10a4933d655191bf 2.7s done
#4 extracting sha256:239e5f5bf9b4aa2c2b1dea741d793f4120e031b92f8553040ad2a1f63ae3a195 0.1s
#4 extracting sha256:239e5f5bf9b4aa2c2b1dea741d793f4120e031b92f8553040ad2a1f63ae3a195 0.1s done
#4 extracting sha256:ed3e311c15d33d38a3dc936aefadad81e4827f6afd4807956f3d185d3e1e70fe 0.1s
#4 extracting sha256:ed3e311c15d33d38a3dc936aefadad81e4827f6afd4807956f3d185d3e1e70fe 0.3s done
#4 extracting sha256:f445a5851dabc97591ba4389a58fda4938fcc19955d92ebdf55951ea111fbe1f done
#4 extracting sha256:815b2c9ae66da3c01de8f54110cebf074229f3ae1c9df06dcc94f4859d4ccd47
#4 extracting sha256:815b2c9ae66da3c01de8f54110cebf074229f3ae1c9df06dcc94f4859d4ccd47 0.1s done
#4 DONE 7.7s

#5 exporting to image
#5 exporting layers done
#5 writing image sha256:788213f800cfd9aa558d3e8a4cf1da71fa3eb3720acf837625acfd5e0633374f done
#5 naming to docker.io/openpipelines-bio/dummycomponent/myscript:test done
#5 DONE 0.1s
====================================================================
+/tmp/viash_test_myscript_8289795475484293803/test_test_script/test_executable
[notice] Checking if Docker image is available at 'dummycomponent/myscript:test'
[warning] Could not pull from 'dummycomponent/myscript:test'. Docker image doesn't exist or is not accessible.
[notice] Building container 'dummycomponent/myscript:test' with Dockerfile
Traceback (most recent call last):
  File "/viash_automount/tmp/viash_test_myscript_8289795475484293803/test_test_script/tmp/viash-run-myscript-3JkBSM.py", line 31, in <module>
    raise ValueError("TEST FAIL")
ValueError: TEST FAIL
====================================================================
ERROR! Only 0 out of 1 test scripts succeeded!
Unexpected error occurred! If you think this is a bug, please post
create an issue at https://github.com/viash-io/viash/issues containing
a reproducible example and the stack trace below.

viash - 0.9.0-RC7
Stacktrace:
java.lang.RuntimeException: Only 0 out of 1 test scripts succeeded!
    at io.viash.ViashTest$.apply(ViashTest.scala:137)
    at io.viash.Main$.mainCLI(Main.scala:286)
    at io.viash.Main$.mainCLIOrVersioned(Main.scala:124)
    at io.viash.Main$.main(Main.scala:59)
    at io.viash.Main.main(Main.scala)

The second time the test is executed, this bug goed unnoticed! (because the container exists)

Running tests in temporary directory: '/tmp/viash_test_myscript_1338827784990535058'
====================================================================
+/tmp/viash_test_myscript_1338827784990535058/build_engine_environment/myscript ---verbosity 6 ---setup cachedbuild ---engine docker
[notice] Building container 'openpipelines-bio/dummycomponent/myscript:test' with Dockerfile
[info] docker build -t 'openpipelines-bio/dummycomponent/myscript:test'  '/tmp/viash_test_myscript_1338827784990535058/build_engine_environment' -f '/tmp/viash_test_myscript_1338827784990535058/build_engine_environment/tmp/dockerbuild-myscript-SpXGxL/Dockerfile'
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 292B done
#1 DONE 0.0s

#2 [internal] load metadata for docker.io/library/python:latest
#2 DONE 0.3s

#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [1/1] FROM docker.io/library/python:latest@sha256:e3d5b6f95ce66923b5e48a06ee5755abb097de96a8617c3f2f7d431d48e63d35
#4 CACHED

#5 exporting to image
#5 exporting layers done
#5 writing image sha256:c66db11f178aef4cc8f618e7805fcc430556a08bc44a0031ac0ae646e69cc147 done
#5 naming to docker.io/openpipelines-bio/dummycomponent/myscript:test done
#5 DONE 0.0s
====================================================================
+/tmp/viash_test_myscript_1338827784990535058/test_test_script/test_executable
Traceback (most recent call last):
  File "/viash_automount/tmp/viash_test_myscript_1338827784990535058/test_test_script/tmp/viash-run-myscript-eSgftK.py", line 31, in <module>
    raise ValueError("TEST FAIL")
ValueError: TEST FAIL
====================================================================
ERROR! Only 0 out of 1 test scripts succeeded!
Unexpected error occurred! If you think this is a bug, please post
create an issue at https://github.com/viash-io/viash/issues containing
a reproducible example and the stack trace below.

viash - 0.9.0-RC7
Stacktrace:
java.lang.RuntimeException: Only 0 out of 1 test scripts succeeded!
    at io.viash.ViashTest$.apply(ViashTest.scala:137)
    at io.viash.Main$.mainCLI(Main.scala:286)
    at io.viash.Main$.mainCLIOrVersioned(Main.scala:124)
    at io.viash.Main$.main(Main.scala:59)
    at io.viash.Main.main(Main.scala)

Expected behavior

Notice that the first part of the logs says: 

+/tmp/viash_test_myscript_8289795475484293803/build_engine_environment/myscript ---verbosity 6 ---setup cachedbuild ---engine docker
Building container 'openpipelines-bio/dummycomponent/myscript:test' with Dockerfile

This is already odd, I think this should be --setup build (but it might be expected behavior) However, a bit later in the logs:

+/tmp/viash_test_myscript_18019117146254022055/test_test_script/test_executable
[notice] Checking if Docker image is available at 'dummycomponent/myscript:test'

So the test_executable script is looking at the wrong container name because the organization is missing. This is a problem when running the tests for a second time: the build step is executed for the wrong image ID and the test_executable uses the image from a previous test of the component.

Relevant log output

No response

Version

- Ubuntu 22.04
- openjdk 21.0.4 2024-07-16
- Viash 0.9.0-RC7

Possible solution

No response

Confirmation

Additional context

No response

rcannood commented 3 months ago

I've been having this issue for a while without really understanding why.

Thanks finding out what the root cause is!

As a workaround, I've been running docker system prune -a whenever I had this issue.

@Grifs would you be willing to look into where the project config info isn't being passed to the right place?

Grifs commented 3 months ago

Fixed in Viash 0.9.0-RC8