Add git commit hash to ENV

[trungpham]

Is it possible to add a new GitCommit environment variable? So the Dockerfile can read that environment variable at build time.

[naartjie]

That would be awesome. Right now I have a bit of a nasty hack for this. I install git in the docker container first, and then:

# version info
RUN \
  VERSION=$(git rev-parse --short HEAD) && \
  DATE=$(date +%Y-%m-%dT%H:%M:%S) && \
  if ! [[ -z "`git status -s`" ]]; then VERSION="!! DIRTY ${VERSION}"; fi && \
  sed -i "s/@@__VERSION__@@/${VERSION}/g;s/@@__BUILT__@@/${DATE}/g" ./src/main/resources/VERSION.edn

Not amazing in terms of verbosity, but it works. GitCommit as an env variable would be a winner though.

[pchico83]

We inject the SOURCE_COMMIT env variable in the builder container. In order to use it in your docker file, you will need to add it as a build arg and define a build hook to pass this variable as a build arg:

docker build -t $IMAGE_NAME --build-arg SOURCE_COMMIT=$SOURCE_COMMIT .

Hooks are documented here https://docs.docker.com/docker-cloud/builds/automated-build/#/use-custom-build-phase-hooks. Although the build hook is not documented, it replaces the build command that we execute by default.

[naartjie]

Thanks @pchico83, so this is part of Docker Cloud only? Is there a way if I am using just a standard docker hub automated build from a github repo? Or should we just migrate to using Docker Cloud instead?

[pchico83]

This is also available in Docker Hub although it is not documented. I would recommend to switch to Docker Cloud, which has more features than Hub. Some new Docker Cloud features include:

• Run tests for pull requests.
• Integration with github/bitbucket commit statutes.
• Slack notifications
• Ability to cancel and retry a running build.

[naartjie]

Epic, thanks for confirming that @pchico83, will make a switch at some stage.

[ahaasler]

Thanks @pchico83, but I'm having problems trying to use build args on Docker Hub. I used some of the variables described on https://docs.docker.com/docker-cloud/builds/advanced/ (not docker hub, but you said it's also available there) and I can't get them working on my container.

You can see what I did on ahaasler/docker-jira@a6e8018854b9b454aa1a999a0cc1e3ca142dbe53. Am I using it wrong?

If I migrate these repos to Docker Cloud what happens to the page in the Hub? Does it still refresh the readme and all repo info on GitHub pushes? Does it still get marked as Automated build?

[pchico83]

@ahaasler You need to define a build hook to pass the variables into the Dockerfile arguments:

#!/bin/bash 
docker build -t $IMAGE_NAME --build-arg SOURCE_COMMIT=$SOURCE_COMMIT--build-arg DOCKER_TAG=$DOCKER_TAG .

The route of this file is hooks/build at the same path as your Dockerfile.

[ahaasler]

@pchico83 thanks, but $SOURCE_COMMIT doesn't contain anything. I'm echoing the value in the hook and it's empty. Hook: https://github.com/ahaasler/docker-jira/blob/e9f599e81e89c92d323f09eb6f4d7cafad70414d/hooks/build Log: https://hub.docker.com/r/ahaasler/jira/builds/bfwgkvgjy343pcnehxbkm6h/

$IMAGE_NAME and $DOCKER_TAG are working as expected.

[pchico83]

SOURCE_COMMIT is empty for manual builds. I have created an internal issue to fix it, in the meantime you can get it by executinbg:

git rev-parse HEAD

as part of your build hook.

[ahaasler]

@pchico83 thanks for the suggestion, now it's working as it should, but I'm getting the same error on random builds since I set up the build hook. You can see that some of the last builds (https://hub.docker.com/r/ahaasler/jira/builds/) are failing in the COPY launch.sh /launch instruction.

[pchico83]

@ahaasler We are working on a fix for and it will fixed in the next days. I will let you know when the fix is available.

[pchico83]

@ahaasler the fix took longer than expected, but the COPY instruction should be working now.

[ahaasler]

@pchico83 thanks!

[lilith]

SOURCE_COMMIT is still missing. Still gotta use git rev-parse HEAD

#!/bin/bash

echo "SOURCE_COMMIT: $SOURCE_COMMIT"

if [[ -z "$SOURCE_COMMIT" ]]; then
    export SOURCE_COMMIT="${SOURCE_COMMIT:-$(git rev-parse HEAD)}"
    echo "Updating SOURCE_COMMIT from git rev-parse HEAD"
    echo "SOURCE_COMMIT: $SOURCE_COMMIT"
fi

echo "DOCKER_TAG: $DOCKER_TAG"

docker build -t "$IMAGE_NAME" --build-arg "SOURCE_COMMIT=$SOURCE_COMMIT" --build-arg "DOCKER_TAG=$DOCKER_TAG" .

[andreif]

What is the status of this issue?

In my Dockerfile I have:

...
ARG SOURCE_COMMIT
ENV SOURCE_COMMIT $SOURCE_COMMIT
RUN echo "SOURCE_COMMIT = $SOURCE_COMMIT"
...

Docker Cloud build log is:

...
Step 21/25 : ARG SOURCE_COMMIT
---> Running in 68f212bbd977
---> 66b30dfb0318
Removing intermediate container 68f212bbd977
Step 22/25 : ENV SOURCE_COMMIT $SOURCE_COMMIT
---> Running in 1434006761a7
---> cfda94d58f00
Removing intermediate container 1434006761a7
Step 23/25 : RUN echo "SOURCE_COMMIT = $SOURCE_COMMIT"
---> Running in 5b6f00b2dc9b
SOURCE_COMMIT =
---> 1555addde746
Removing intermediate container 5b6f00b2dc9b
...

If I build locally, then it works:

$ docker-compose build --no-cache --force-rm --pull \
        --build-arg SOURCE_COMMIT=$(git rev-parse HEAD) api

...
Step 21/25 : ARG SOURCE_COMMIT
 ---> Running in ed1e3850db37
Removing intermediate container ed1e3850db37
 ---> 4f658cc5b3d0
Step 22/25 : ENV SOURCE_COMMIT $SOURCE_COMMIT
 ---> Running in c6909a2fda0e
Removing intermediate container c6909a2fda0e
 ---> 79071f63927d
Step 23/25 : RUN echo "SOURCE_COMMIT = $SOURCE_COMMIT"
 ---> Running in dc255dae1e5b
SOURCE_COMMIT = 0e0f20b5dc9a38a6bf867931e23d2c6d0dde0f85
Removing intermediate container dc255dae1e5b
 ---> ce2c51c2e93d
...

[andreif]

@pchico83 Could you confirm that the build args are still provided?

[andreif]

I have noticed that I missed to update docker-compose.test.yml according to the docs. After updating it, I still don't see $SOURCE_COMMIT set before or after using arg.

version: "3.3"

services:
  sut:
    build: .
    entrypoint:
      - wait-for-services.sh
      - postgres
      - neo4j
      - make
      - --makefile
      - ../Makefile
    command: test
    depends_on:
      - postgres
      - redis
      - neo4j
    env_file:
      - ./docker/envs/common/api.env
      - ./docker/envs/test/api.env
    environment:
      - SOURCE_COMMIT
...

...
RUN echo "SOURCE_COMMIT = $SOURCE_COMMIT"
ARG SOURCE_COMMIT
ENV SOURCE_COMMIT $SOURCE_COMMIT
RUN echo "SOURCE_COMMIT = $SOURCE_COMMIT"
...

...
Step 21/26 : RUN echo "SOURCE_COMMIT = $SOURCE_COMMIT"
---> Running in 8378310f89d4
SOURCE_COMMIT =
---> 2b7986207f06
Removing intermediate container 8378310f89d4
Step 22/26 : ARG SOURCE_COMMIT
---> Running in 9d2b49189a23
---> f9145e1482e8
Removing intermediate container 9d2b49189a23
Step 23/26 : ENV SOURCE_COMMIT $SOURCE_COMMIT
---> Running in aa2b63b84261
---> 0265c713a82f
Removing intermediate container aa2b63b84261
Step 24/26 : RUN echo "SOURCE_COMMIT = $SOURCE_COMMIT"
---> Running in 747d7fb8e2cb
SOURCE_COMMIT =
---> 11603c2797e5
Removing intermediate container 747d7fb8e2cb
...

[andreif]

I added two variables to config:

...
    environment:
      - SOURCE_BRANCH
      - SOURCE_COMMIT
...

and while they are not available during build step (when it runs Dockerfile?), I can see them during testing now:

...
.PHONY: test
test:
    @echo "SOURCE_BRANCH = $(SOURCE_BRANCH)"
    @echo "SOURCE_COMMIT = $(SOURCE_COMMIT)"
    @python3 manage.py test \
...

...
SOURCE_BRANCH = cloud
SOURCE_COMMIT = 2d2764edf305453aa92ef6b994dda41c99b50bd8
...

[andreif]

Alright, so I have some progress now and found out how to set build arg for sut (testing) step, which is kinda useless since I already have $SOURCE_COMMIT variable available in the test container, but anyway it's a progress. I got it by specifying build arg in sut config for build:

services:
  sut:
    build:
      context: .
      args:
        - SOURCE_COMMIT=${SOURCE_COMMIT}
...
    environment:
      - SOURCE_COMMIT

...
Step 21/26 : RUN echo "SOURCE_COMMIT = $SOURCE_COMMIT"
---> Running in a003bbeaa6c5
SOURCE_COMMIT =
---> 0683ab252cba
Removing intermediate container a003bbeaa6c5
Step 22/26 : ARG SOURCE_COMMIT
---> Running in 889c50ba8fde
---> 319583da11a0
Removing intermediate container 889c50ba8fde
Step 23/26 : ENV SOURCE_COMMIT $SOURCE_COMMIT
---> Running in ee0e2a482983
---> d06693498783
Removing intermediate container ee0e2a482983
Step 24/26 : RUN echo "SOURCE_COMMIT = $SOURCE_COMMIT"
---> Running in fbb9627fce36
SOURCE_COMMIT = b85e6f81d3a1f567534080035ebcf245a2eef347
---> cc2e2356d0d4
Removing intermediate container fbb9627fce36
...

So there must be a similar way to provide it for build step which is executed before sut.

[andreif]

Good news, my eyes see now and I got it to work with the build hook as originally proposed by @pchico83. Will there be an easier way to provide build args similar to how one can do it for sut?

[tleyden]

@pchico83 regarding https://github.com/docker/hub-feedback/issues/600#issuecomment-231009831, I tried adding the following to my Dockerfile that's being built via an automated build on dockerhub:

ARG SOURCE_COMMIT
ARG SOURCE_BRANCH

RUN echo "SOURCE_COMMIT: $SOURCE_COMMIT"
RUN echo "source branch: $SOURCE_BRANCH"

However the output is empty for both:

Removing intermediate container a5d6cecb74e5
Step 4/12 : ARG SOURCE_COMMIT
 ---> Running in b10eef64d7e2
 ---> dc81847407cd
Removing intermediate container b10eef64d7e2
Step 5/12 : ARG SOURCE_BRANCH
 ---> Running in 350d5e334cf4
 ---> 5ed34c63613e
Removing intermediate container 350d5e334cf4
Step 6/12 : RUN echo "SOURCE_COMMIT: $SOURCE_COMMIT"
 ---> Running in 93cb1c19f1eb
SOURCE_COMMIT: 
 ---> 2bf7a7227cb7
Removing intermediate container 93cb1c19f1eb
Step 7/12 : RUN echo "source branch: $SOURCE_BRANCH"
 ---> Running in 7f948ef79dae
source branch: 
 ---> 7dfc73ea5a8d
Removing intermediate container 7f948ef79dae

[jri-sp]

I can confirm that SOURCE_COMMIT varaible doesn't work in Docker Hub automated build.

[dotdoom]

For Docker Hub (Docker Cloud) automated build, one solution is to pull .git directory into the container, temporarily, to fetch the necessary information. Here's a working snippet:

# Save Git commit hash of this build into /docker_repo_version.
COPY .git /tmp/repo/.git
RUN apt-get install git && \
    git -C /tmp/repo rev-parse HEAD > /docker_repo_version && \
    rm -rf /tmp/repo

[ptsneves]

Small hack When you need to get the git hash of the current HEAD but do not want to install git into the build environment, this has some likelihood of working:

RUN export hash=$(cat .git/$(cat .git/HEAD | cut -d' ' -f2)) && echo $hash

[lilith]

Although the build hook hack worked for a few years, it spontaneously stopped working about 2 years ago with the following unhelpful error message - does anyone have any idea what this could be caused by?

Your branch is up-to-date with 'origin/master'.
Executing build hook...
Unexpected error
Encountered error: [Errno 2] No such file or directory
Traceback (most recent call last):
File "/stage/builder/runner.py", line 290, in _run
self.build()
File "/stage/builder/runner.py", line 214, in build
self._build()
File "/stage/builder/runner.py", line 196, in _build
if not hooks.run('build'):
File "/stage/builder/hooks.py", line 21, in run
utils.execute_command(hook_path, '{} hook failed!'.format(name))
File "/stage/builder/utils.py", line 43, in execute_command
bufsize=1)
File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

My dockerfile:

FROM imazen/imageflow_base_os

MAINTAINER Lilith River

ARG SOURCE_COMMIT
ARG DOCKER_TAG

RUN if [ -z "${SOURCE_COMMIT}" ]; then echo "SOURCE_COMMIT not set - should be $(git rev-parse HEAD). Exiting." && exit 1; else echo "SOURCE_COMMIT=${SOURCE_COMMIT}"; fi

RUN mkdir nightly && cd nightly && wget -nv -O ifs.tar.gz https://s3-us-west-1.amazonaws.com/imageflow-nightlies/commits/${SOURCE_COMMIT}/linux64_glibc227.tar.gz \
    && tar xvzf ifs.tar.gz && mv ./imageflow_tool ../ && cd .. && rm -rf nightly

ENTRYPOINT ["/home/imageflow/imageflow_tool"]
CMD ["help"]

My build hook in hooks/build

#!/bin/bash

echo "SOURCE_COMMIT: $SOURCE_COMMIT"

if [[ -z "$SOURCE_COMMIT" ]]; then
    export SOURCE_COMMIT="${SOURCE_COMMIT:-$(git rev-parse HEAD)}"
    echo "Updating SOURCE_COMMIT from git rev-parse HEAD"
    echo "SOURCE_COMMIT: $SOURCE_COMMIT"
fi

echo "DOCKER_TAG: $DOCKER_TAG"

docker build -t "$IMAGE_NAME" --build-arg "SOURCE_COMMIT=$SOURCE_COMMIT" --build-arg "DOCKER_TAG=$DOCKER_TAG" .

[github-actions[bot]]

We are clearing up our old issues and your ticket has been open for one year with no activity. Remove stale label or comment or this will be closed in 15 days.

[PeterDaveHello]

The lack of activity depends on Docker Hub staffs ...