databacker / mysql-backup

image to enable automated backups of mysql databases in containers
648 stars 185 forks source link

Database restore from s3 problem #230

Open drmax24 opened 1 year ago

drmax24 commented 1 year ago

I'm trying to restore mysql from s3


6/14/2023, 3:02:46 PM GMT+2 + '[' -d /scripts.d/post-restore/ ']'   c24-int-community-db-backup
6/14/2023, 3:02:46 PM GMT+2 + /bin/rm -f /tmp/restorefile   c24-int-community-db-backup
6/14/2023, 3:02:46 PM GMT+2 + rm -rf /tmp/restore.1 c24-int-community-db-backup
6/14/2023, 3:02:46 PM GMT+2 cat: '/tmp/restore.1/*': No such file or directory  c24-int-community-db-backup
6/14/2023, 3:02:46 PM GMT+2 + cat '/tmp/restore.1/*'    c24-int-community-db-backup
6/14/2023, 3:02:46 PM GMT+2 + mysql -h c24-int-community-instance-1.cjbcawj0mthy.eu-central-1.rds.amazonaws.com -P 3306 -uroot '-pMYFAKEPASSWORD' -Dc24_int_community   c24-int-community-db-backup
6/14/2023, 3:02:46 PM GMT+2 gunzip: invalid magic   c24-int-community-db-backup
6/14/2023, 3:02:46 PM GMT+2 tar: short read c24-int-community-db-backup
6/14/2023, 3:02:46 PM GMT+2 + gunzip    c24-int-community-db-backup
6/14/2023, 3:02:46 PM GMT+2 + tar -C /tmp/restore.1 -xvf -  c24-int-community-db-backup
6/14/2023, 3:02:46 PM GMT+2 + mkdir -p /tmp/restore.1   c24-int-community-db-backup
6/14/2023, 3:02:46 PM GMT+2 + '[' true = true ']'   c24-int-community-db-backup
6/14/2023, 3:02:46 PM GMT+2 + DBDATABASE=-Dc24_int_community    c24-int-community-db-backup
6/14/2023, 3:02:46 PM GMT+2 + workdir=/tmp/restore.1    c24-int-community-db-backup
6/14/2023, 3:02:46 PM GMT+2 + rm -rf /tmp/restore.1 c24-int-community-db-backup
6/14/2023, 3:02:46 PM GMT+2 + [[ -f /tmp/restorefile ]] c24-int-community-db-backup

...THEN GOES copying data from S3...


th 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.9 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.9 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.9 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.9 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.9 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.9 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.7 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.7 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.8 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.8 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.8 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.8 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.8 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.8 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.8 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.8 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.8 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.8 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.8 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.8 MiB/s) with 1 file(s) remaining Completed 2.9 GiB/5.8 GiB (110.8 Mi
...
remaining Completed 5.8 GiB/5.8 GiB (115.6 MiB/s) with 1 file(s) remaining Completed 5.8 GiB/5.8 GiB (115.6 MiB/s) with 1 file(s) remaining Completed 5.8 GiB/5.8 GiB (115.6 MiB/s) with 1 file(s) remaining download: s3://c24-int-community-backup-20230613112316023300000001/c24_prod_community_2023-06-09.sql to tmp/restorefile

the sql is not imported into database the script just stops after copying data from s3

This is my input:

      environment : [
        { name = "DB_USER", value = "root" },
        { name = "DB_SERVER", value = "c24-int-community-instance-1.cjbcawj0mthy.eu-central-1.rds.amazonaws.com"},
        { name = "DB_NAMES", value = replace(local.prefix, "-", "_") },
        { name = "AWS_DEFAULT_REGION", value = local.region },
        { name = "DB_RESTORE_TARGET", value = "s3://${aws_s3_bucket.community_backup.bucket}/c24_prod_community_2023-06-09.sql" },
        { name = "DB_DUMP_DEBUG", value = "true"},
        { name = "SINGLE_DATABASE", value = "true" }
      ]
      secrets : [
        { name = "DB_PASS", valueFrom = data.aws_secretsmanager_secret_version.db_password.arn },
        { name = "AWS_ACCESS_KEY_ID", valueFrom = "${data.aws_secretsmanager_secret.s3.arn}:AWS_S3_ACCESS_KEY_ID::" },
        { name = "AWS_SECRET_ACCESS_KEY", valueFrom = "${data.aws_secretsmanager_secret.s3.arn}:AWS_S3_SECRET_ACCESS_KEY::" },
      ]

Any idas?

deitch commented 1 year ago

Post up the whole debug output. It is as if it got confused about time and order of operations.

Post up the whole debug output please? But make sure that you filtered for any secrets.