Env variable $GOGS_HOME doesn't exist in docker container

[tarfeef102]

I'm running off of the docker image provided by the gogs project here.

I've tried running those commands specified in the README of this repo in the locations I thought might be appropriate:

• /app/gogs
• /app/gogs/data
• /data/gogs

but none made any impact. Can you please advise what exactly the "root directory" of gogs is (at least in this image), and perhaps update the README to help those who run in Docker off of the provided image (which i'd have to imagine is most people). Thanks.

[Kos-M]

Well this is how looks like my gogs root dir (without docker)

try to run it with presistence data with a volume like this If still not working , drop another comment here to fire up a docker container to investigate further .. Hey , thanks for your feedback.

[tarfeef102]

Here is what the dir structure of the image looks like:

/app/gogs # ls -la
total 51196
drwxr-xr-x    1 root     root          4096 Oct 12 03:06 .
drwxr-xr-x    1 root     root          4096 Aug 22 10:05 ..
lrwxrwxrwx    1 root     root            15 Oct 12 03:06 data -> /data/gogs/data
drwxr-xr-x    1 root     root          4096 Oct  7 15:19 docker
-rwxr-xr-x    1 root     root      52395608 Oct  7 15:18 gogs
lrwxrwxrwx    1 root     root            14 Oct 12 03:06 log -> /data/gogs/log
/app/gogs # ls -la /app/
total 20
drwxr-xr-x    1 root     root          4096 Aug 22 10:05 .
drwxr-xr-x    1 root     root          4096 Oct 11 08:09 ..
drwxr-xr-x    1 root     root          4096 Oct 12 03:06 gogs

It seems that the equivalent to what you have shown there is /app/gogs, since that's where data/ log/ and gogs are located. The documentation provided by gogs tells you to persist /data, which does NOT contain /app/gogs, but only some sub-directories. I'll try to create a custom/ folder which is symlinked into /data so it persists properly, and get back to you on this (although I swear I tried /app/gogs already and it did survive a container restart...)

Do you know of a way to restart the application from within the container, just so I can test the functionality w/o having to deal with persistence for the time being? Dealing with smaller issues 1 at a time is usually better, after all

[Kos-M]

i have to say there , that custom folder in latest version of gogs , didn't exist by default. So maybe that why you didn't see it there.

Do you know of a way to restart the application from within the container

No , i'm afraid. only in docker run level.

Dealing with smaller issues 1 at a time is usually better, after all

yeah , multitasking dont work well in troubleshooting lol

[tarfeef102]

1. Updating the docs to help users identify what the GOGS_HOME directory is when they're in a situation like mine where that env var doesn't exist, and the custom directory isn't present to clearly indicate where this stuff should go would be very helpful, then.
2. Well I'll have to try the aforementioned strategy, though since I still swear I got that data to persist before, my hopes aren't high.
3. Same as above.

I'll post a comment when I get the results from that experiment

[Kos-M]

sure i ll add a section for docker setup , when we find out correct settings. :)

[tarfeef101]

(It's me again, but on another acct). Still failing. Here's the Dockerfile and compose I used:

[tarfeef101@tarfeef101-pc gogs_dark]$ cat Dockerfile 
FROM gogs/gogs:0.12.3
RUN mkdir -p /app/gogs/custom/templates/inject/ && \
    mkdir -p /app/gogs/custom/public/css/ && \
    git clone https://github.com/Kos-M/GogsThemes.git /app/gogs/GogsThemes && \
    echo '<link rel="stylesheet" href="/css/themes/velvet.css">' >> /app/gogs/custom/templates/inject/head.tmpl && \
    mv /app/gogs/GogsThemes/themes/ /app/gogs/custom/public/css/ && \
    cp -r /app/gogs/GogsThemes/img/ /app/gogs/custom/public/ && \
    rm -r /app/gogs/GogsThemes
[tarfeef101@tarfeef101-pc gogs_dark]$ cat docker-compose.yaml 
version: "3"
services:
  gogs:
    build: .
    image: tarfeef101/gogs_dark:0.12.3
    container_name: gogs
    restart: on-failure
    ports:
      - '3000:3000'
      - '10022:22'
    volumes:
      - 'test_gogs_data:/data'
      - 'test_gogs_custom:/app/gogs/custom'
volumes:
  test_gogs_data:
  test_gogs_custom:

I started the container, didn't have dark mode (but this was on the setup page which you may not have bothered with, and also it had never been restarted yet, so that's okay). Clicked through the menus, and ran docker-compose down and docker-compose up, but still no dark mode after that. Exec-ed into the container at that point, and confirmed the files WERE present, they just didn't do anything. Also did the same test but instead of docker-compose down and docker-compose up, ran docker-compose restart (which won't destroy the container). Same end result.

Can you confirm that this even works with the 0.12.3 version of gogs? I feel like this should work...

[Kos-M]

after a while , i make it work.. docker-compose.yml

version: "3"
services:
  mysql:
    image: mariadb
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: 1234
      MYSQL_DATABASE: gogs
    expose:
      - 3306
    networks:
      - core
    volumes:
      - db_data:/var/lib/mysql  
  gogs:
    build: .
    image: tarfeef101/gogs_dark:0.12.3
    container_name: gogs
    restart: on-failure
    networks:
      - core
    links:
     - mysql
    depends_on:
    - mysql     
    ports:
      - '3000:3000'
      - '10022:22'

volumes:
  db_data:

networks:
  core:
    external: false

Dockerfile

FROM gogs/gogs:0.12.3

COPY docker-entrypoint.sh /usr/local/bin/
RUN ln -s /usr/local/bin/docker-entrypoint.sh /  
ENTRYPOINT ["docker-entrypoint.sh"]

docker-entrypoint.sh

#!/bin/bash

mkdir -p /data/gogs/templates/inject/ && \
    mkdir -p /data/gogs/public/css/ && \
    git clone https://github.com/Kos-M/GogsThemes.git /data/gogs/GogsThemes && \
    echo '<link rel="stylesheet" href="/css/themes/dark_theme.css">' >> /data/gogs/templates/inject/head.tmpl && \
    mv /data/gogs/GogsThemes/themes/ /data/gogs/public/css/ && \
    cp -r /data/gogs/GogsThemes/img/ /data/gogs/public/ && \
    rm -r /data/gogs/GogsThemes && \
    chown -hR git:git  /data/gogs/templates/* && \
    chown -hR git:git  /data/gogs/public/* 

 create_socat_links() {
    # Bind linked docker container to localhost socket using socat
    USED_PORT="3000:22"
    while read -r NAME ADDR PORT; do
        if test -z "$NAME$ADDR$PORT"; then
            continue
        elif echo $USED_PORT | grep -E "(^|:)$PORT($|:)" > /dev/null; then
            echo "init:socat  | Can't bind linked container ${NAME} to localhost, port ${PORT} already in use" 1>&2
        else
            SERV_FOLDER=/app/gogs/docker/s6/SOCAT_${NAME}_${PORT}
            mkdir -p "${SERV_FOLDER}"
            CMD="socat -ls TCP4-LISTEN:${PORT},fork,reuseaddr TCP4:${ADDR}:${PORT}"
            # shellcheck disable=SC2039
            echo -e "#!/bin/sh\nexec $CMD" > "${SERV_FOLDER}"/run
            chmod +x "${SERV_FOLDER}"/run
            USED_PORT="${USED_PORT}:${PORT}"
            echo "init:socat  | Linked container ${NAME} will be binded to localhost on port ${PORT}" 1>&2
        fi
    done << EOT
    $(env | sed -En 's|(.*)_PORT_([0-9]+)_TCP=tcp://(.*):([0-9]+)|\1 \3 \4|p')
EOT
}

cleanup() {
    # Cleanup SOCAT services and s6 event folder
    # On start and on shutdown in case container has been killed
    rm -rf "$(find /app/gogs/docker/s6/ -name 'event')"
    rm -rf /app/gogs/docker/s6/SOCAT_*
}

create_volume_subfolder() {
    # Modify the owner of /data dir, make $USER(git) user have permission to create sub-dir in /data.
    chown -R "$USER:$USER" /data

    # Create VOLUME subfolder
    for f in /data/gogs/data /data/gogs/conf /data/gogs/log /data/git /data/ssh; do
        if ! test -d $f; then
            gosu "$USER" mkdir -p $f
        fi
    done
}

setids() {
    export USER=git
    PUID=${PUID:-1000}
    PGID=${PGID:-1000}
    groupmod -o -g "$PGID" $USER
    usermod -o -u "$PUID" $USER
}

setids
cleanup
create_volume_subfolder

LINK=$(echo "$SOCAT_LINK" | tr '[:upper:]' '[:lower:]')
if [ "$LINK" = "false" ] || [ "$LINK" = "0" ]; then
    echo "init:socat  | Will not try to create socat links as requested" 1>&2
else
    create_socat_links
fi

CROND=$(echo "$RUN_CROND" | tr '[:upper:]' '[:lower:]')
if [ "$CROND" = "true" ] || [ "$CROND" = "1" ]; then
    echo "init:crond  | Cron Daemon (crond) will be run as requested by s6" 1>&2
    rm -f /app/gogs/docker/s6/crond/down
    /bin/sh /app/gogs/docker/runtime/backup-init.sh "${PUID}"
else
    # Tell s6 not to run the crond service
    touch /app/gogs/docker/s6/crond/down
fi

# Exec CMD or S6 by default if nothing present
if [ $# -gt 0 ];then
    exec "$@"
else
    exec /bin/s6-svscan /app/gogs/docker/s6/
fi

gogs will start its server when theme is installed.

in web installation select mysql > and as host : mysql:3306 , password 1234

[tarfeef101]

I appreciate the effort, but that's wayyyy too much.

I guess what this really boils down to is the single important change: you run the download of the dark-theme related files before the app itself starts, and you chown them to the git user (even though other files aren't owned by them). Is that the part that's actually necessary?

Edit: actually, no, that's not even true. When I run things in my original edited dockerfile, that also happens prior to the entrypoint script running. so that should not be an issue. is it literally just permissions?

Edit 2: I see the dirs are also different (e.g. no "custom" in the path). Is that the actual important change?

[Kos-M]

entry-point script is just copy from gogs repo . i just added there theme installation logic. im sure there is more straight way to do it , but im still learning docker.

chown commands here , don't be crusial , but i faced some issues (before) not reading correct from web server if some files arent owned by git run level user.

/data/gogs dir act like custom dir. all folders inside custom must be inside /data/gogs https://github.com/gogs/gogs/commit/f41f5785bfa2b1c6beffc5765efef7328f859658

if you find something more easy and more fast please post it here, to adapt it. Thanks

[Kos-M]

actually this need more work. not persist ALL data and tries to install theme every time you start - restart containers. needs change..

[tarfeef101]

I tried the following, but the data is not even there when you exec in and inspect the filesystem:

[tarfeef101@tarfeef101-pc gogs_dark]$ docker-compose up -d
Creating network "gogs_dark_default" with the default driver
Creating volume "gogs_dark_test_gogs_data" with default driver
Creating gogs ... done
[tarfeef101@tarfeef101-pc gogs_dark]$ docker exec -it gogs /bin/sh
/app/gogs # ls /data/gogs
conf  data  log
/app/gogs # exit
[tarfeef101@tarfeef101-pc gogs_dark]$ cat Dockerfile 
FROM gogs/gogs:0.12.3
RUN mkdir -p /data/gogs/templates/inject/ && \
    mkdir -p /data/gogs/public/css/ && \
    git clone https://github.com/Kos-M/GogsThemes.git /data/gogs/GogsThemes && \
    echo '<link rel="stylesheet" href="/css/themes/dark_theme.css">' >> /data/gogs/templates/inject/head.tmpl && \
    mv /data/gogs/GogsThemes/themes/ /data/gogs/public/css/ && \
    cp -r /data/gogs/GogsThemes/img/ /data/gogs/public/ && \
    rm -r /data/gogs/GogsThemes && \
    chown -hR git:git /data/gogs/templates/ && \
    chown -hR git:git /data/gogs/public/
[tarfeef101@tarfeef101-pc gogs_dark]$ cat docker-compose.yaml 
version: "3"
services:
  gogs:
    build: .
    image: tarfeef101/gogs_dark:0.12.3
    container_name: gogs
    restart: on-failure
    ports:
      - '3000:3000'
      - '10022:22'
    volumes:
      - 'test_gogs_data:/data'
volumes:
  test_gogs_data:

[tarfeef101]

Something is deleting the files, and I think that's the issue here

[Kos-M]

try to delete that volume , before start compose. docker volume rm test_gogs_data

[tarfeef101]

Yeah, sadly, that's not the problem. I have been pruning my volumes every try (and even just ran it without the volume at all, but the files still were not there)

[Kos-M]

i see , in my setup above , did you tried ? theme there at least is installed. just we want to trigger install only in build time. not in start...

[tarfeef101]

No, I didn't run yours because it changed wayyy more than I actually need. I don't want an external DB, the extra networking in the compose file is unnecessary (you can remove that, those are basically just the defaults compose will do for you), and it shouldn't matter if you run the download in a RUN or at the beginning of the entrypoint script (but doing it in the former results in a much cleaner setup)

Edit: doing the download in a RUN in an extended dockerfile rather than modifying the entrypoint is exactly what will make it only run at build time, not when the container starts.

[Kos-M]

doing the download in a RUN in an extended dockerfile rather than modifying the entrypoint is exactly what will make it only run at build time, not when the container starts.

that makes sense indeed . im suspecting something ovverides file changes in /data/gogs volume or app setup maybe.. will keep trying to solve this..

[tarfeef101]

Here's a fun bit of information:

Building gogs
Step 1/3 : FROM gogs/gogs:0.12.3
 ---> 4075c4a42f12
Step 2/3 : RUN mkdir -p /data/gogs/templates/inject/ &&     mkdir -p /data/gogs/public/css/ &&     git clone https://github.com/Kos-M/GogsThemes.git /data/gogs/GogsThemes &&     echo '<link rel="stylesheet" href="/css/themes/dark_theme.css">' >> /data/gogs/templates/inject/head.tmpl &&     mv /data/gogs/GogsThemes/themes/ /data/gogs/public/css/ &&     cp -r /data/gogs/GogsThemes/img/ /data/gogs/public/ &&     rm -r /data/gogs/GogsThemes &&     chown -hR git:git /data/gogs/templates/ &&     chown -hR git:git /data/gogs/public/
 ---> Using cache
 ---> e847cf4009c7
Step 3/3 : RUN ls -la /data/gogs
 ---> Running in 8bae714b364a
ls: /data/gogs: No such file or directory
ERROR: Service 'gogs' failed to build : The command '/bin/sh -c ls -la /data/gogs' returned a non-zero code: 1

The commands we've been running don't even work properly, /data/gogs can't be ls-ed

[tarfeef101]

and this is just from within the Dockerfile, the container doesn't even exist yet, that's just the image. so the entrypoint hasn't run (and therefore had a chance to mess with things)

[Kos-M]

To keep your data out of Docker container, we do a volume (/var/gogs -> /data) here, and you can change it based on your situation.

from gogs repo docker. i check in my host (ubuntu) i dont even have that folder under var dir. you?

[tarfeef101]

Well docker will create the dir on the host if it doesn't exist. Personally, I wouldn't put stuff in /var, so not sure why they decided that was a good place.

I tried using a local bind-mount, a named volume, and no volume, all with the same result.

However, the fact that just this:

FROM gogs/gogs:0.12.3
RUN mkdir -p /data/gogs/templates/inject/ && \
    mkdir -p /data/gogs/public/css/ && \
    git clone https://github.com/Kos-M/GogsThemes.git /data/gogs/GogsThemes && \
    echo '<link rel="stylesheet" href="/css/themes/dark_theme.css">' >> /data/gogs/templates/inject/head.tmpl && \
    mv /data/gogs/GogsThemes/themes/ /data/gogs/public/css/ && \
    cp -r /data/gogs/GogsThemes/img/ /data/gogs/public/ && \
    rm -r /data/gogs/GogsThemes && \
    chown -hR git:git /data/gogs/templates/ && \
    chown -hR git:git /data/gogs/public/
RUN ls -la /data/gogs

returns an error when you build it is a problem that precedes runtime errors

[tarfeef101]

So if we try to ls /data instead of /data/gogs, we get this very interesting result:

Building gogs
Step 1/3 : FROM gogs/gogs:0.12.3
 ---> 4075c4a42f12
Step 2/3 : RUN mkdir -p /data/gogs/templates/inject/ &&     mkdir -p /data/gogs/public/css/ &&     git clone https://github.com/Kos-M/GogsThemes.git /data/gogs/GogsThemes &&     echo '<link rel="stylesheet" href="/css/themes/dark_theme.css">' >> /data/gogs/templates/inject/head.tmpl &&     mv /data/gogs/GogsThemes/themes/ /data/gogs/public/css/ &&     cp -r /data/gogs/GogsThemes/img/ /data/gogs/public/ &&     rm -r /data/gogs/GogsThemes &&     chown -hR git:git /data/gogs/templates/ &&     chown -hR git:git /data/gogs/public/
 ---> Using cache
 ---> e847cf4009c7
Step 3/3 : RUN ls -la /data
 ---> Running in 44abc0e69eb3
total 8
drwxr-xr-x    2 root     root          4096 Oct 12 22:23 .
drwxr-xr-x    1 root     root          4096 Oct 12 23:11 ..
Removing intermediate container 44abc0e69eb3
 ---> 78f845666247

Successfully built 78f845666247
Successfully tagged tarfeef101/gogs_dark:0.12.3

[tarfeef101]

I found out why this is happening: in their infinite wisdom, gogs decided to do this: https://github.com/gogs/gogs/blob/release/0.12/Dockerfile#L41

as can be seen in the docker documentation, "Changing the volume from within the Dockerfile: If any build steps change the data within the volume after it has been declared, those changes will be discarded."

[tarfeef101]

so I have a very bad solution: If you force yourself to use a bind-mount, you can run this bootstrap script before bringing the container up, and it works:

[tarfeef101@tarfeef101-pc gogs_dark]$ docker-compose up -d
Creating network "gogs_dark_default" with the default driver
Creating gogs ... done
[tarfeef101@tarfeef101-pc gogs_dark]$ docker exec -it gogs /bin/sh
/app/gogs # ls /data/gogs/
conf       data       log        public     templates
/app/gogs # exit
[tarfeef101@tarfeef101-pc gogs_dark]$ cat bootstrap.sh 
mkdir -p ./data/gogs/templates/inject/ && \
    mkdir -p ./data/gogs/public/css/ && \
    git clone https://github.com/Kos-M/GogsThemes.git ./data/gogs/GogsThemes && \
    echo '<link rel="stylesheet" href="/css/themes/dark_theme.css">' >> ./data/gogs/templates/inject/head.tmpl && \
    mv ./data/gogs/GogsThemes/themes/ ./data/gogs/public/css/ && \
    cp -r ./data/gogs/GogsThemes/img/ ./data/gogs/public/ && \
    rm -rf ./data/gogs/GogsThemes
[tarfeef101@tarfeef101-pc gogs_dark]$ cat docker-compose.yaml 
version: "3"
services:
  gogs:
    build: .
    image: gogs/gogs:0.12.3
    container_name: gogs
    restart: on-failure
    ports:
      - '3000:3000'
      - '10022:22'
    volumes:
      - './data:/data'

However, this forces you to use a bind-mount, and to run that script prior to bringing the image up. Sadly, you'd have to rewrite the Dockerfile and rebuild it, removing the VOLUME declaration to avoid this issue. Perhaps we can expand this project to include building images based on theirs with themes pre-baked in, and without the volume declarations to make deployment easier for people? I'd be glad to help with making that a reality

[tarfeef101]

In fact, since I have an existing deployment with a named volume, I couldn't do this without a significant manual investment in trying to backup user data and changing over to a bind mount, so that seems like an effort I'll have to undergo anyways. It would make sense for it to live here rather than in a personal repo, since it is more or less tied to the goal of deploying themed gogs, so I think we should go for that.

[tarfeef101]

for now, I made this fork and added this commit to it (got a prototype working on my local): https://github.com/tarfeef101/gogs_dark/commit/090a6cfb2c61e211ea31cfaedec973fb01ccf838

[Kos-M]

hey , what about this ?

version: "3"
services:
  gogs:
    image: gogs/gogs:0.12.3
    container_name: gogs
    restart: on-failure
    ports:
      - '3000:3000'
      - '10022:22'
    volumes:
      - './templates:/data/gogs/templates'
      - './public:/data/gogs/public'

setup local in host these 2 folders templates - public (git clone theme ) and hook them directly in compose. is simply enough and you can change theme easily from host.

[tarfeef102]

Eh I don't like having a bunch of bind mounts lying around unnecessarily. The fork I linked is a much cleaner solution, I think. If you wanna work with me to integrate something like that into this repo, I'd be down to help. But at least for myself, I think that's how I'll be deploying from now on. I'd much rather build a new image and keep my host clean than pollute it with extra bind mounts and require extra steps on the host before deployment

[Kos-M]

I see.. We have to update manually these images every time gogs fires new update although ?

[tarfeef102]

Automation for that kind of thing can be set up. The basic setup would be:

• Listen for new commits on branches named release/*
• Clone repo, modify dockerfile, build, push image to Docker Hub

Ideally, we would say "only listen to branches we identify as being supported", so if they release a new major version we'd test to make sure everything still works before adding that new release branch and building images for it. But such a setup is totally possible, yeah

[Kos-M]

Sounds good , we totaly go for that.. I just fired up a new discord server , for not spaming with comments , and more straightforward communication. Join