search

dokku / dokku-postgres

a postgres plugin for dokku
MIT License
490 stars 97 forks source link

postgres backup failed to s3 - Unable to locate credentials #209

Closed sushilbansal closed 4 years ago

sushilbansal commented 4 years ago

Description of problem

Not able to backup existing postgres service

How reproducible

create the post gres service, link it to the ap, setup the AWS auth and then tried to do the backup

Steps to Reproduce

1) dokku postgres:create pg-service 2) dokku postgres:link pg-service myapp 3) postgres:backup-auth pg-service 4) postgres:backup pg-service [--use-iam]

Actual Results

upload failed: - to s3://MyBucketName/postgres-pg-2020-08-09-12-26-58.tgz Unable to locate credentials

i checked the

Expected Results

success

dokku report APP_NAME output

-----> uname: Linux bakbak-server 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux -----> memory: total used free shared buff/cache available Mem: 985 313 96 17 576 509 Swap: 0 0 0 -----> docker version: Client: Version: 18.09.6 API version: 1.39 Go version: go1.10.8 Git commit: 481bc77 Built: Sat May 4 02:35:57 2019 OS/Arch: linux/amd64 Experimental: false

   Server: Docker Engine - Community
    Engine:
     Version:          18.09.6
     API version:      1.39 (minimum version 1.12)
     Go version:       go1.10.8
     Git commit:       481bc77
     Built:            Sat May  4 01:59:36 2019
     OS/Arch:          linux/amd64
     Experimental:     false

-----> docker daemon info: Containers: 13 Running: 4 Paused: 0 Stopped: 9 Images: 14 Server Version: 18.09.6 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84 runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30 init version: fec3683 Security Options: apparmor seccomp Profile: default Kernel Version: 4.15.0-52-generic Operating System: Ubuntu 18.04.2 LTS OSType: linux Architecture: x86_64 CPUs: 1 Total Memory: 985.5MiB Name: bakbak-server ID: NOUJ:LYD3:XCMC:7A7O:3BXF:A37U:AUGL:MQHH:XLAD:LGQN:HQA4:BCWZ Docker Root Dir: /var/lib/docker Debug Mode (client): true Debug Mode (server): false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Product License: Community Engine

-----> sigil version: 0.4.0 -----> herokuish version: herokuish: 0.5.0 buildpacks: heroku-buildpack-multi v1.0.0 heroku-buildpack-ruby v200 heroku-buildpack-nodejs v142 heroku-buildpack-clojure v83 heroku-buildpack-python v149 heroku-buildpack-java v63 heroku-buildpack-gradle v28 heroku-buildpack-scala v84 heroku-buildpack-play v26 heroku-buildpack-php v151 heroku-buildpack-go v104 buildpack-nginx v10 -----> dokku version: dokku version 0.17.9 -----> dokku plugins: plugn: 0.3.2 00_dokku-standard 0.17.9 enabled dokku core standard plugin 20_events 0.17.9 enabled dokku core events logging plugin app-json 0.17.9 enabled dokku core app-json plugin apps 0.17.9 enabled dokku core apps plugin build-env 0.17.9 enabled dokku core build-env plugin buildpacks 0.17.9 enabled dokku core buildpacks plugin certs 0.17.9 enabled dokku core certificate management plugin checks 0.17.9 enabled dokku core checks plugin common 0.17.9 enabled dokku core common plugin config 0.17.9 enabled dokku core config plugin docker-options 0.17.9 enabled dokku core docker-options plugin domains 0.17.9 enabled dokku core domains plugin enter 0.17.9 enabled dokku core enter plugin git 0.17.9 enabled dokku core git plugin letsencrypt 0.9.4 enabled Automated installation of let's encrypt TLS certificates logs 0.17.9 enabled dokku core logs plugin network 0.17.9 enabled dokku core network plugin nginx-vhosts 0.17.9 enabled dokku core nginx-vhosts plugin plugin 0.17.9 enabled dokku core plugin plugin postgres 1.11.5 enabled dokku postgres service plugin proxy 0.17.9 enabled dokku core proxy plugin ps 0.17.9 enabled dokku core ps plugin redis 1.12.5 enabled dokku redis service plugin repo 0.17.9 enabled dokku core repo plugin resource 0.17.9 enabled dokku core resource plugin scheduler-docker-local 0.17.9 enabled dokku core scheduler-docker-local plugin shell 0.17.9 enabled dokku core shell plugin ssh-keys 0.17.9 enabled dokku core ssh-keys plugin storage 0.17.9 enabled dokku core storage plugin tags 0.17.9 enabled dokku core tags plugin tar 0.17.9 enabled dokku core tar plugin trace 0.17.9 enabled dokku core trace plugin =====> bakbakserver app information App dir: /home/dokku/bakbakserver Git sha:
Deploy source:
Locked: false
=====> bakbakserver buildpacks information Buildpacks list:
CN = server.bakbakapp.com error 20 at 0 depth lookup: unable to get local issuer certificate =====> bakbakserver ssl information Ssl dir: /home/dokku/bakbakserver/tls Ssl enabled: true
Ssl hostnames: server.bakbakapp.com
Ssl expires at: Nov 6 18:12:51 2020 GMT Ssl issuer: C = US, O = Lets Encrypt, CN = Lets Encrypt Authority X3 Ssl starts at: Aug 8 18:12:51 2020 GMT Ssl subject: subject=CN = server.bakbakapp.com Ssl verified: self signed
=====> bakbakserver checks information Checks disabled list: none
Checks skipped list: none
=====> bakbakserver docker options information Docker options build: --link dokku.postgres.pg:dokku-postgres-pg --link dokku.redis.red:dokku-redis-red Docker options deploy: --link dokku.postgres.pg:dokku-postgres-pg --link dokku.redis.red:dokku-redis-red --restart=on-failure:10 Docker options run: --link dokku.postgres.pg:dokku-postgres-pg --link dokku.redis.red:dokku-redis-red =====> bakbakserver domains information Domains app enabled: true
Domains app vhosts: server.bakbakapp.com
Domains global enabled: true
Domains global vhosts: bakbakapp.com
=====> bakbakserver git information Git rev env var: GIT_REV
Git deploy branch: master
Git global deploy branch: master
=====> bakbakserver network information Network bind all interfaces: false Network listeners: 172.17.0.5:80 =====> bakbakserver proxy information Proxy enabled: true
Proxy type: nginx
Proxy port map: http:80:80 https:443:80
=====> bakbakserver ps information Processes: 1
Deployed: true
Running: true
Restore: true
Restart policy: on-failure:10
Ps can scale: true
Status web.1: running (CID: 126097b871fd) =====> bakbakserver scheduler-docker-local information Scheduler docker local disable chown:
=====> bakbakserver storage information Storage build mounts:
Storage deploy mounts:
Storage run mounts:

How (deb/make/rpm) and where (AWS, VirtualBox, physical, etc.) was Dokku installed?:

Digitial OCean droplet - single click app - Dokku

postgres:logs

The files belonging to this database system will be owned by user "postgres". This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8". The default database encoding has accordingly been set to "UTF8". The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok creating subdirectories ... ok selecting default max_connections ... 100 selecting default shared_buffers ... 128MB selecting default timezone ... Etc/UTC selecting dynamic shared memory implementation ... posix creating configuration files ... ok running bootstrap script ... ok performing post-bootstrap initialization ... ok syncing data to disk ... ok

WARNING: enabling "trust" authentication for local connections You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

pg_ctl -D /var/lib/postgresql/data -l logfile start

waiting for server to start....2020-08-08 09:09:12.854 UTC [44] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" 2020-08-08 09:09:12.878 UTC [45] LOG: database system was shut down at 2020-08-08 09:09:12 UTC 2020-08-08 09:09:12.883 UTC [44] LOG: database system is ready to accept connections done server started

/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*

2020-08-08 09:09:12.943 UTC [44] LOG: received fast shutdown request waiting for server to shut down....2020-08-08 09:09:12.945 UTC [44] LOG: aborting any active transactions 2020-08-08 09:09:12.948 UTC [44] LOG: background worker "logical replication launcher" (PID 51) exited with exit code 1 2020-08-08 09:09:12.950 UTC [46] LOG: shutting down 2020-08-08 09:09:12.972 UTC [44] LOG: database system is shut down done server stopped

PostgreSQL init process complete; ready for start up.

2020-08-08 09:09:13.058 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 2020-08-08 09:09:13.058 UTC [1] LOG: listening on IPv6 address "::", port 5432 2020-08-08 09:09:13.064 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" 2020-08-08 09:09:13.082 UTC [53] LOG: database system was shut down at 2020-08-08 09:09:12 UTC 2020-08-08 09:09:13.087 UTC [1] LOG: database system is ready to accept connections 2020-08-08 09:09:13.490 UTC [60] LOG: incomplete startup packet 2020-08-08 09:09:14.605 UTC [1] LOG: received smart shutdown request 2020-08-08 09:09:14.609 UTC [1] LOG: background worker "logical replication launcher" (PID 59) exited with exit code 1 2020-08-08 09:09:14.613 UTC [54] LOG: shutting down 2020-08-08 09:09:14.630 UTC [1] LOG: database system is shut down

PostgreSQL Database directory appears to contain a database; Skipping initialization

2020-08-08 09:09:17.056 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 2020-08-08 09:09:17.058 UTC [1] LOG: listening on IPv6 address "::", port 5432 2020-08-08 09:09:17.062 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" 2020-08-08 09:09:17.097 UTC [24] LOG: database system was shut down at 2020-08-08 09:09:14 UTC 2020-08-08 09:09:17.104 UTC [1] LOG: database system is ready to accept connections

Schlepptop commented 4 years ago

Since you are using the --use-iam flag, are you using dokku on an AWS EC2 instance? In your details you say you're on DO, but IAM won't work there iirc.

sushilbansal commented 4 years ago

Hi @Schlepptop , yes i am using Digital Ocean. Is there a way to do the backup from DO to S3.

Schlepptop commented 4 years ago

Yes, you need to create the credentials in AWS and then set them via dokku postgres:backup-auth, passing access key, secret access key etc as arguments to the command. Afterwards, dokku postgres:backup should work.

sushilbansal commented 4 years ago

I tried that but it did not work. i followed the below steps:

1) dokku postgres:create pg-service
2) dokku postgres:link pg-service myapp
3) postgres:backup-auth pg-service <aws-access-key-id> <aws-secret-access-key> <aws-default-region> <aws-signature-version> <endpoint-url>
4) postgres:backup pg-service [--use-iam]
Schlepptop commented 4 years ago

Can you check if the files exist in /var/lib/dokku/services/postgres/your-service/backup?

sushilbansal commented 4 years ago

Yes, it exists and contains AWS_ACCESS_KEY_ID AWS_DEFAULT_REGION AWS_SECRET_ACCESS_KEY AWS_SIGNATURE_VERSION ENDPOINT_URL

sushilbansal commented 4 years ago

without the flag [--use-iam] it worked. Thanks for the help guys :)