kartoza / docker-pg-backup

A cron job that will back up databases running in a docker postgres container
GNU General Public License v2.0
452 stars 103 forks source link

Environment variables broken in cron #56

Closed amq closed 2 years ago

amq commented 2 years ago

I'm running the kartoza/pg-backup:14-3.1 image with docker-compose like this:

  postgres-backup:
    image: kartoza/pg-backup:13-3.1
    restart: always
    volumes:
      - postgres-backup:/backups
    environment:
      - POSTGRES_HOST=postgres
      - POSTGRES_USER=postgres
      - POSTGRES_PASS=example
      - REMOVE_BEFORE=15
      - CRON_SCHEDULE='0 0 * * *'

When I exec into the container, all variables are accessible, for example POSTGRES_HOST is correctly postgres instead of db

Also when I manually run /backup-scripts/backups.sh, the backup is created as expected

But for some reason, the env variables are not passed to cron. I could additionally confirm this by running a test script:

#!/bin/bash

source /backup-scripts/env-data.sh

env

Via cron

POSTGRES_USER=docker
POSTGRES_PASS=docker

Manually

POSTGRES_USER=postgres
POSTGRES_PASS=example
NyakudyaA commented 2 years ago

I'm running the kartoza/pg-backup:14-3.1 image with docker-compose like this:

  postgres-backup:
    image: kartoza/pg-backup:13-3.1
    restart: always
    volumes:
      - postgres-backup:/backups
    environment:
      - POSTGRES_HOST=postgres
      - POSTGRES_USER=postgres
      - POSTGRES_PASS=example
      - REMOVE_BEFORE=15
      - CRON_SCHEDULE='0 0 * * *'

When I exec into the container, all variables are accessible, for example POSTGRES_HOST is correctly postgres instead of db

Also when I manually run /backup-scripts/backups.sh, the backup is created as expected

But for some reason, the env variables are not passed to cron. I could additionally confirm this by running a test script:

#!/bin/bash

source /backup-scripts/env-data.sh

env

Via cron

POSTGRES_USER=docker
POSTGRES_PASS=docker

Manually

POSTGRES_USER=postgres
POSTGRES_PASS=example

@amq Are the backups not being created ?. I tried on my local machine and it works as expected with backups being generated

fibfab commented 2 years ago

I can confirm this issue. Backups are not being created because he falls back to the default docker/docker credentials. The cronjob has no access to the env variables. Manually started the backups.sh from inside the container works well.

I also tried it with kartoza/pg-backup:14-3.1

kumpinar commented 2 years ago

I can confirm this issue too. When I run backup in the container, it works. But when backup script triggred via cron, it fails.

Docker compose part:

dbbackups:
    image: kartoza/pg-backup:14-3.1
    networks:
      - web
    environment:
      - DUMPPREFIX=PG_gis
      - POSTGRES_HOST=db
      - POSTGRES_USER=user_name
      - POSTGRES_PASS=password
      - POSTGRES_PORT=5432
    restart: on-failure
    depends_on:
      db:
        condition: service_healthy

Working scenario: > docker exec -it 123containerid123 bash > root@123containerid123:/backup-scripts# ./backups.sh

Via cron:

psql: error: connection to server at "db" (172.29.0.6), port 5432 failed: FATAL: password authentication failed for user "docker" connection to server at "db" (172.29.0.6), port 5432 failed: FATAL: password authentication failed for user "docker"

NyakudyaA commented 2 years ago

I can confirm this issue too. When I run backup in the container, it works. But when backup script triggred via cron, it fails.

Docker compose part:


dbbackups:

    image: kartoza/pg-backup:14-3.1

    networks:

      - web

    environment:

      - DUMPPREFIX=PG_gis

      - POSTGRES_HOST=db

      - POSTGRES_USER=user_name

      - POSTGRES_PASS=password

      - POSTGRES_PORT=5432

    restart: on-failure

    depends_on:

      db:

        condition: service_healthy

Working scenario:

> docker exec -it 123containerid123 bash

> root@123containerid123:/backup-scripts# ./backups.sh

Via cron:

psql: error: connection to server at "db" (172.29.0.6), port 5432 failed: FATAL: password authentication failed for user "docker"

connection to server at "db" (172.29.0.6), port 5432 failed: FATAL: password authentication failed for user "docker"

@kumpinar i can also confirm the issue and it's because the cron runs in a sub shell and the env variables are not exposed. I will provide a fix during the weekend or Monday latest

NyakudyaA commented 2 years ago

@amq can you please build from master and test if the issue is resolved so that I can publish a new image