cirruslabs / gitlab-tart-executor

GitLab Runner executor to run jobs in Tart VMs
MIT License
60 stars 5 forks source link

Allow specifying custom builds_dir and cache_dir #26

Closed jlsalmon closed 1 year ago

jlsalmon commented 1 year ago

Hello,

I would like to be able to explicitly configure the builds_dir and cache_dir in config.toml to point to host-mounted directories, so I can persist my sources and caches across builds.

Currently the builds_dir and cache_dir options are ignored by the Tart executor, which overrides them to /private/tmp (unless TART_EXECUTOR_HOST_DIR is supplied, in which case a fixed-name hostdir directory is mounted from a generated tempdir, which is wiped at the end of the build).

This is something that's already possible with the built-in docker executor, for example:

[[runners]]
  executor = "docker+machine"
  builds_dir = "/builds"
  cache_dir = "/cache"
  [runners.custom_build_dir]
    enabled = true
  [runners.docker]
    volumes = ["/data/cache:/cache", "/data/builds:/builds"]

I would expect to be able to do the same with the tart executor:

[[runners]]
  executor = "custom"
  builds_dir = "/Volumes/My Shared Files/builds"
  cache_dir = "/Volumes/My Shared Files/cache"
  [runners.custom_build_dir]
    enabled = true
  [runners.custom]
    config_exec = "gitlab-tart-executor"
    config_args = ["config"]
    prepare_exec = "gitlab-tart-executor"
    prepare_args = ["prepare", "--dir", "builds:/some/host/path/builds", "--dir", "cache:/some/host/path/cache"]
    run_exec = "gitlab-tart-executor"
    run_args = ["run"]
    cleanup_exec = "gitlab-tart-executor"
    cleanup_args = ["cleanup"]

Support for arbitrary host mounts was already added in #24, so I think it should only be a short hop to support this.

edigaryev commented 1 year ago

Unfortunately, GitLab Runner does not pass these fields as variables to custom executors.

We have a --cache-dir command-line argument for config stage for working around this, and I've added --builds-dir command-line argument in https://github.com/cirruslabs/gitlab-tart-executor/pull/27.

jlsalmon commented 1 year ago

Excellent, thanks @edigaryev for the swift and effective response, this should do the trick 👍