Azure / batch-shipyard

Simplify HPC and Batch workloads on Azure
MIT License
277 stars 121 forks source link

Docker error when batch-shipyard attempts to run job: chdir to cwd(\"\") set in config.json failed: permission denied #245

Closed sarahhp closed 6 years ago

sarahhp commented 6 years ago

Problem Description

I have run across a docker error when batch shipyard attempts to run my job and I just can’t figure out what’s going on. That is the pool creation succeeds but the job execution fails. Error text is: docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "chdir to cwd (\"/mnt/batch/tasks/workitems/snlacedryrun/job-1/task-00000/wd\") set in config.json failed: permission denied": unknown.

Batch Shipyard Version

3.6.0

Steps to Reproduce

I'm deploying from an azure VM (Ubuntu) using the following commands from the directory containing the configuration files: $SHIPYARD/shipyard pool add $SHIPYARD/shipyard jobs add -v --tail stderr.txt Also mounting three azure file shares to the docker container.

Expected Results

job created

Actual Results

docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "chdir to cwd (\"/mnt/batch/tasks/workitems/snlacedryrun/job-1/task-00000/wd\") set in config.json failed: permission denied": unknown.

Redacted Configuration

config.yml: image jobs.yml: image

Additional Logs

stdout to VM upon running shipyard jobs add: image

Additonal Comments

Not sure if this issue is stemming from my configuration setup, settings being passed to docker via shipyard, or is an underlying docker issue, sorry for that! Any guidance appreciated.

alfpark commented 6 years ago

Can you retry your task without the user_identity section?

sarahhp commented 6 years ago

@alfpark yes thankyou that seems to fix the issue

alfpark commented 6 years ago

Great, one other minor thing with your config: you don't need to specify additional_registries under global_resources. That is only required if you are using Docker images which aren't specified to preload and have a registry login associated with them.

sarahhp commented 6 years ago

Ah I see. Thanks!