search

Sage-Bionetworks / SynapseWorkflowOrchestrator

Apache License 2.0
3 stars 2 forks source link

Unable to specify locally built docker images with WORKFLOW_ENGINE_DOCKER_IMAGE #12

Open thomasyu888 opened 4 years ago

thomasyu888 commented 4 years ago

The Orchestrator will always attempt to pull an image from dockerhub. Here I built the python3 version of toil and name the docker image as test. This is the error I receive:

workflow-orchestrator_1  | [org.sagebionetworks.WorkflowOrchestrator.main()] INFO org.sagebionetworks.WorkflowManagerDocker - toil cmd: [toil-cwl-runner, --defaultMemory, 100M, --retryCount, 0, --defaultDisk, 1000000, --workDir, /var/lib/docker/volumes/workflow_orchestrator_shared/_data/1310bf13-2a7d-4909-8f49-ef0c5346e904, --noLinkImports, /var/lib/docker/volumes/workflow_orchestrator_shared/_data/1310bf13-2a7d-4909-8f49-ef0c5346e904/ChallengeWorkflowTemplates-develop/docker_agent_workflow.cwl, /var/lib/docker/volumes/workflow_orchestrator_shared/_data/1310bf13-2a7d-4909-8f49-ef0c5346e904/TMP17726918510012474563.yaml]
workflow-orchestrator_1  | [dockerjava-jaxrs-async-0] ERROR com.github.dockerjava.api.async.ResultCallbackTemplate - Error during callback
workflow-orchestrator_1  | com.github.dockerjava.api.exception.NotFoundException: pull access denied for test, repository does not exist or may require 'docker login'

Similarly if I try to name the tag sagebionetworks/synapse-workflow-orchestrator-toil:2.0 I get:

workflow-orchestrator_1  | [org.sagebionetworks.WorkflowOrchestrator.main()] INFO org.sagebionetworks.WorkflowManagerDocker - toil cmd: [toil-cwl-runner, --defaultMemory, 100M, --retryCount, 0, --defaultDisk, 1000000, --workDir, /var/lib/docker/volumes/workflow_orchestrator_shared/_data/446e7ef0-fe04-42e5-9c56-9e9675146584, --noLinkImports, /var/lib/docker/volumes/workflow_orchestrator_shared/_data/446e7ef0-fe04-42e5-9c56-9e9675146584/ChallengeWorkflowTemplates-develop/docker_agent_workflow.cwl, /var/lib/docker/volumes/workflow_orchestrator_shared/_data/446e7ef0-fe04-42e5-9c56-9e9675146584/TMP8817161377415703692.yaml]
workflow-orchestrator_1  | [dockerjava-jaxrs-async-0] ERROR com.github.dockerjava.api.async.ResultCallbackTemplate - Error during callback
workflow-orchestrator_1  | com.github.dockerjava.api.exception.NotFoundException: manifest for sagebionetworks/synapse-workflow-orchestrator-toil:2.0 not found

Here is my docker images

[tyu@ip-10-5-16-123 SynapseWorkflowOrchestrator]$ docker images
REPOSITORY                                           TAG                 IMAGE ID            CREATED             SIZE
test                                                 latest              dc4d3bf606d4        8 minutes ago       2.02GB
sagebionetworks/synapse-workflow-orchestrator-toil   2.0                 dc4d3bf606d4        8 minutes ago       2.02GB
node                                                 slim                071d01d48a48        3 days ago          178MB
sagebionetworks/synapse-workflow-orchestrator        1.0                 c70fff3513bd        12 days ago         971MB
sagebionetworks/synapse-workflow-orchestrator-toil   1.0                 5e2a363367c8        12 days ago         1.99GB
sagebionetworks/challengeutils                       develop             3cb4c2ebc363        2 weeks ago         1.1GB
python                                               3.6                 a2e9f0fba405        3 weeks ago         913MB
python                                               3.7                 023b89039ba4        3 weeks ago         918MB
sagebionetworks/synapse-workflow-orchestrator-toil   latest              85858597c42a        4 weeks ago         2.01GB
sagebionetworks/synapsepythonclient                  v1.9.2              9c11a899814b        7 months ago        234MB
thomasyu888 commented 4 years ago

The current workaround is to push the test toil image to my own dockerhub repository.