maxisam / mgob

MongoDB dockerized backup agent. Runs schedule backups with retention, S3 & SFTP upload, notifications, instrumentation with Prometheus and more.
https://maxisam.github.io/mgob/
MIT License
123 stars 16 forks source link

gpg bin missing in latest image #107

Closed fragolinux closed 2 months ago

fragolinux commented 2 months ago

hi, using chart 2.1.0, with mgob 2.0.18, i see this in pods logs (gpg bin missing, and others), i don't know if my backups are failing because of these errors, or other things...

mgob time="2024-04-16T10:25:20Z" level=info msg="mgob 2.0.18"
mgob time="2024-04-16T10:25:20Z" level=info msg="starting with config: &{LogLevel:info JSONLog:false Host: Port:8090 ConfigPath:/config StoragePath:/storage TmpPath:/tmp DataPath:/data Version:2.0.18 UseAwsCli:fals
e HasGpg:false}"
mgob time="2024-04-16T10:25:20Z" level=info msg="mongodump version: 100.8.0 git version: 732ddfaa6b467ffcd5bfa69a455953320eed85f4 Go version: go1.21.1    os: linux    arch: amd64    compiler: gc "
mgob time="2024-04-16T10:25:20Z" level=info msg="NAME:   mc version - manage bucket versioning  USAGE:   mc version COMMAND [COMMAND FLAGS | -h] [ARGUMENTS...]  COMMANDS:   enable   enable bucket versioning   suspe
nd  suspend bucket versioning   info     show bucket versioning status    FLAGS:   --config-dir value, -C value  path to configuration folder (default: \"/root/.mc\")   --quiet, -q                   disable progres
s bar display   --no-color                    disable color theme   --json                        enable JSON lines formatted output   --debug                       enable debug output   --insecure
   disable SSL certificate verification   --limit-upload value          limits uploads to a maximum rate in KiB/s, MiB/s, GiB/s. (default: unlimited)   --limit-download value        limits downloads to a maximum ra
te in KiB/s, MiB/s, GiB/s. (default: unlimited)   --help, -h                    show help    "
mgob time="2024-04-16T10:25:21Z" level=info msg="aws-cli/1.29.44 Python/3.11.5 Linux/4.18.0-477.15.1.el8_8.x86_64 botocore/1.31.44 "
mgob time="2024-04-16T10:25:21Z" level=warning msg="gpg failed /bin/sh: gpg: not found : exit status 127"
mgob time="2024-04-16T10:25:23Z" level=info msg="Google Cloud SDK 445.0.0 bq 2.0.97 bundled-python3-unix 3.9.16 core 2023.09.01 gcloud-crc32c 1.0.0 gsutil 5.25 "
mgob time="2024-04-16T10:25:24Z" level=info msg="WARNING: You have 2 update(s) available. Consider updating your CLI installation with 'az upgrade' azure-cli                         2.52.0 * "
mgob time="2024-04-16T10:25:24Z" level=info msg="rclone v1.64.0 "
mgob time="2024-04-16T10:25:24Z" level=info msg="Next tmp cleanup run at 2024-04-16 11:00:00 +0000 UTC"
mgob time="2024-04-16T10:25:24Z" level=info msg="Next run at 2024-04-16 11:00:00 +0000 UTC" plan=hourly
mgob time="2024-04-16T10:25:24Z" level=info msg="Starting HTTP server on port 8090"
mgob time="2024-04-16T11:00:00Z" level=info msg="Backup started" plan=hourly
mgob time="2024-04-16T11:00:30Z" level=info msg="new dump" archive= err="after 0 retries, mongodump log : exit status 1" mlog=
mgob time="2024-04-16T11:00:30Z" level=error msg="BACKUP FAILED: after 0 retries, mongodump log : exit status 1" plan=hourly
mgob time="2024-04-16T11:00:30Z" level=info msg="Next run at 2024-04-16 12:00:00 +0000 UTC" plan=hourly
mongodb {"t":{"$date":"2024-04-16T10:25:32.489+00:00"},"s":"I",  "c":"CONTROL",  "id":5945603, "ctx":"main","msg":"Multi threading initialized"}
mongodb {"t":{"$date":"2024-04-16T10:25:32.489+00:00"},"s":"I",  "c":"CONTROL",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":1,"port":27017,"dbPath":"/data/db","architecture":"64-bit"
,"host":"common-mgob-0"}}
mongodb {"t":{"$date":"2024-04-16T10:25:32.489+00:00"},"s":"W",  "c":"CONTROL",  "id":20720,   "ctx":"initandlisten","msg":"Memory available to mongo process is less than total system memory","attr":{"availableMemS
izeMB":512,"systemMemSizeMB":32448}}
fragolinux commented 2 months ago

and that compatibility matrix in main readme: https://github.com/maxisam/mgob#installation is it a must? I mean, i use mongodb 5.0.10, should i stick to that specific mgob version, or that's the lower mgob version supporting that mongodb version? thanks

maxisam commented 2 months ago

Q1, I don't think that is the cause of your issue.

Q2, I think higher mgob can support lower mongodb version but it is possible to work the other way as well, you will have to test it. It is depending on mongo drive

maxisam commented 2 months ago

Could you test this one

https://hub.docker.com/layers/maxisam/mgob/dev.275/images/sha256-1034797c88a9873c719c7e09427362973fda9a991cefcb504a5a4bff000a0afc?context=explore

Thanks!

fragolinux commented 2 months ago

Could you test this one

https://hub.docker.com/layers/maxisam/mgob/dev.275/images/sha256-1034797c88a9873c719c7e09427362973fda9a991cefcb504a5a4bff000a0afc?context=explore

Thanks!

hi, done, seems fine now, but you're checking the "mc" version the wrong way, from the output i see below... it should be "mc --version" and not "mc version", as now... that's actually a real "mc" command, to manage versioning, not to get the tool version :)

time="2024-04-18T07:17:02Z" level=info msg="mgob dev.275"
time="2024-04-18T07:17:02Z" level=info msg="starting with config: &{LogLevel:info JSONLog:false Host: Port:8090 ConfigPath:/config StoragePath:/storage TmpPath:/tmp DataPath:/data Version:dev.275 UseAwsCli:false HasGpg:false}"
time="2024-04-18T07:17:02Z" level=info msg="mongodump version: 100.8.0 git version: 732ddfaa6b467ffcd5bfa69a455953320eed85f4 Go version: go1.21.1    os: linux    arch: amd64    compiler: gc "
time="2024-04-18T07:17:02Z" level=info msg="NAME:   mc version - manage bucket versioning  USAGE:   mc version COMMAND [COMMAND FLAGS | -h] [ARGUMENTS...]  COMMANDS:   enable   enable bucket versioning   suspend  suspend bucket versioning   info     show bucket versioning status    FLAGS:   --config-dir value, -C value  path to configuration folder (default: \"/root/.mc\") [$MC_CONFIG_DIR]   --quiet, -q                   disable progress bar display [$MC_QUIET]   --no-color                    disable color theme [$MC_NO_COLOR]   --json                        enable JSON lines formatted output [$MC_JSON]   --debug                       enable debug output [$MC_DEBUG]   --insecure                    disable SSL certificate verification [$MC_INSECURE]   --limit-upload value          limits uploads to a maximum rate in KiB/s, MiB/s, GiB/s. (default: unlimited) [$MC_LIMIT_UPLOAD]   --limit-download value        limits downloads to a maximum rate in KiB/s, MiB/s, GiB/s. (default: unlimited) [$MC_LIMIT_DOWNLOAD]   --help, -h                    show help    "
time="2024-04-18T07:17:03Z" level=info msg="aws-cli/1.29.44 Python/3.11.8 Linux/4.18.0-477.15.1.el8_8.x86_64 botocore/1.31.44 "
time="2024-04-18T07:17:03Z" level=info msg="gpg (GnuPG) 2.4.4 libgcrypt 1.10.2 Copyright (C) 2024 g10 Code GmbH License GNU GPL-3.0-or-later <https://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.  Home: /root/.gnupg Supported algorithms: Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,         CAMELLIA128, CAMELLIA192, CAMELLIA256 Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224 Compression: Uncompressed, ZIP, ZLIB, BZIP2 "
time="2024-04-18T07:17:04Z" level=info msg="Google Cloud SDK 445.0.0 bq 2.0.97 bundled-python3-unix 3.9.16 core 2023.09.01 gcloud-crc32c 1.0.0 gsutil 5.25 "
time="2024-04-18T07:17:06Z" level=info msg="WARNING: You have 2 update(s) available. Consider updating your CLI installation with 'az upgrade' azure-cli                         2.52.0 * "
time="2024-04-18T07:17:06Z" level=info msg="rclone v1.66.0 "
time="2024-04-18T07:17:06Z" level=info msg="Next tmp cleanup run at 2024-04-18 08:00:00 +0000 UTC"
time="2024-04-18T07:17:06Z" level=info msg="Next run at 2024-04-18 08:00:00 +0000 UTC" plan=hourly
time="2024-04-18T07:17:06Z" level=info msg="Next run at 2024-04-19 00:00:00 +0000 UTC" plan=daily
time="2024-04-18T07:17:06Z" level=info msg="Next run at 2024-04-21 00:00:00 +0000 UTC" plan=weekly
time="2024-04-18T07:17:06Z" level=info msg="Next run at 2025-02-03 00:00:00 +0000 UTC" plan=ondemand
time="2024-04-18T07:17:06Z" level=info msg="Starting HTTP server on port 8090"
fragolinux commented 2 months ago

one last thing: could you please create an other image with NO cloud clients AT ALL? I see now that there are versions for aws, gcp, azure, and an ALL one, which is HUGE... we don't need any cloud provider as everything is on premise clusters, so a smaller image would be appreciated... thanks!

p.s.: az cli is complaining about being old, too, from the previous logs :)

maxisam commented 2 months ago

how about minio ? https://hub.docker.com/layers/maxisam/mgob/2.0.20-minio/images/sha256-e3f64bde000a02bada4c6813cd280978dc34f4f2c9f934c2953ba02714ad299d?context=explore

It is pretty small <100M

Thanks for checking mc version thing. Do you wanna create a PR for it?

Also does everything work now? Just wanna double check.

fragolinux commented 2 months ago

hi, everything works now, thanks! minio, i'd like to have nothing inside the image that's not needed, but in case i'll test that, thanks