Caching docker directory to speed up image loading

[fenech]

Hi, I have a Concourse build task that looks like

#!/bin/bash

source /docker-lib.sh
start_docker

docker load -i image1.tar
docker load -i image2.tar
docker load -i image3.tar

docker-compose up

The problem is that some of the images are quite large and this makes the task slow to run.

I thought that it might be possible to cache /scratch/docker in my build configuration:

caches: [{path: /scratch/docker}]

But it seems that this doesn't have any effect - docker images returns an empty list before I run docker load.

How can I cache the images loaded in previous builds to speed up my build task?

[dsawa]

@fenech Have you managed to find some kind of workaround ? I have similar problem now.

[fenech]

@dsawa No, I couldn't find a way to persist the state between tasks runs using this image.

In the end I decided to change my approach: instead of using this task image, I SSH into a VM with Docker running there and pull the images rather than load them, so that layer caching can speed things up.

[georggrab]

Note that the concourse documention on caches states:

The path to a directory to be cached.

Paths are relative to the working directory of the task. Absolute paths are not respected.

I usually set up caches in concourse like this:

script in the task amounting to (.sbt is just an example directory):

rm -rf ~/.sbt
ln -fs sbt-cache ~/.sbt

cache configuration in the task declaration

caches:
- path: sbt-cache

Perhaps you could confirm if that approach works for you (applied to the directory /scratch/docker)?