Bad exit status reported to docker

[fajfer]

Image and OTRS versions juanluisbaptiste/otrs:6.0.30 juanluisbaptiste/postfix:alpine juanluisbaptiste/otrs-mariadb:latest

Please include the contents of: OTRS_BACKUP_TYPE="fullbackup"

Describe the issue First of all, I'm running your images for over a year now - there are mostly no problems whatsoever or I made some custom fixes to that. This is the first time I'm seeing something like this but it looks worth reporting. OTRS crashed during daily full backup but exited with exit status 0 - the container itself kept running, so docker-compose was unable to restart it.

I'll try to replicate it and perhaps fix it - but this is something that happened for the first time and I'm running several OTRS instances.

Expected behavior I would expect it to: a) Return exit code different than 0, if that wasn't a 'clean' shut down (optional), or b) Finish running the init script after the services were shut down

Screenshots

Oct 31 04:00:02 a93a7b0419fc CROND[6200]: (otrs) CMD ($HOME/bin/otrs.Daemon.pl start >> /dev/null)
Oct 31 04:00:02 a93a7b0419fc CROND[6201]: (root) CMD (. /.backup.env; /otrs_backup.sh)
Oct 31 04:00:02 a93a7b0419fc CROND[6198]: (root) CMDOUT (/.backup.env: line 11: export: `4': not a valid identifier)
Oct 31 04:00:02 a93a7b0419fc CROND[6198]: (root) CMDOUT (/.backup.env: line 11: export: `anaconda-ks.cfg': not a valid identifier)
Oct 31 04:00:02 a93a7b0419fc CROND[6198]: (root) CMDOUT (/.backup.env: line 11: export: `anaconda-ks.cfg': not a valid identifier)
Oct 31 04:00:02 a93a7b0419fc CROND[6198]: (root) CMDOUT (/.backup.env: line 11: export: `anaconda-ks.cfg': not a valid identifier)
Oct 31 04:00:03 a93a7b0419fc CROND[6198]: (root) CMDOUT (#033[46m[INFO]#033[0m #033[38;5;31mOTRS_DB_NAME#033[0m not set, setting value to #033[31motrs#033[0m)
Oct 31 04:00:03 a93a7b0419fc CROND[6198]: (root) CMDOUT (#033[46m[INFO]#033[0m #033[38;5;31mOTRS_DB_USER#033[0m not set, setting value to #033[31motrs#033[0m)
Oct 31 04:00:03 a93a7b0419fc CROND[6198]: (root) CMDOUT (#033[46m[INFO]#033[0m #033[38;5;31mOTRS_DB_PORT#033[0m not set, setting value to #033[31m3306#033[0m)
Oct 31 04:00:03 a93a7b0419fc CROND[6198]: (root) CMDOUT (#033[46m[INFO]#033[0m #033[38;5;31mMYSQL_ROOT_PASSWORD#033[0m not set, setting password to #033[31mchangeme#033[0m)
Oct 31 04:00:03 a93a7b0419fc CROND[6198]: (root) CMDOUT (#033[46m[INFO]#033[0m #033[38;5;31mMYSQL_ROOT_USER#033[0m not set, setting user to #033[31mroot#033[0m)
Oct 31 04:00:03 a93a7b0419fc CROND[6198]: (root) CMDOUT (#033[46m[INFO]#033[0m #033[38;5;31mSetting OTRS_CRON_BACKUP_SCRIPT#033[0m to #033[31m/etc/cron.d/otrs_backup#033[0m)
Oct 31 04:00:03 a93a7b0419fc CROND[6198]: (root) CMDOUT (#033[46m[INFO]#033[0m #033[38;5;31mSetting OTRS_ARTICLE_STORAGE_TYPE#033[0m to #033[31mArticleStorageDB#033[0m)
Oct 31 04:00:03 a93a7b0419fc CROND[6198]: (root) CMDOUT ([2020-10-31_04-00] Starting OTRS backup for host my.properly.configured.fqdn.here...)
Oct 31 04:00:03 a93a7b0419fc CROND[6198]: (root) CMDOUT (#033[46m[INFO]#033[0m Stopping all OTRS services...)
2020-10-31 04:00:07,968 INFO waiting for httpd to stop
2020-10-31 04:00:07,968 INFO waiting for crond to stop
2020-10-31 04:00:07,968 INFO waiting for rsyslog to stop
2020-10-31 04:00:08,536 INFO stopped: crond (exit status 0)
2020-10-31 04:00:08,537 INFO stopped: rsyslog (exit status 0)
[Sat Oct 31 04:00:09.092983 2020] [mpm_prefork:notice] [pid 31421] AH00169: caught SIGTERM, shutting down
2020-10-31 04:00:09,778 INFO stopped: httpd (exit status 0)

[juanluisbaptiste]

Hi,

Did you noticed this error ?

.backup.env: line 11: export:4': not a valid identifier)`

It seems there is an error in your env file, probably that is why the backup process failed. The container startup script saves all the OTRS_* env variables on the .backup.env file so the backup script can read them, it seems one of those vars is broken.

[fajfer]

Hey, it failed again for me :) I have the exact same value here on all other instances, here is the line 11 of .backup.env file:

export OTRS_BACKUP_TIME=0 4 * * *

seems to work on all the other ones like this, except I just noticed this is the only one container running 6.0.30 version that has that problem - 6.0.29 containers are flawless and have the same configuration (except for the passwords). It strips the quotes from the original env file, which is:

OTRS_BACKUP_TIME="0 4 * * *"

but I guess this is not the problem

[fajfer]

This is a recurring problem for my setup - I'll try to set up a fresh instance and check if it's affected as well. Might be something specific or simply an overlook on my side. Do the lines switch somehow or line 11 is always the cron backup timer?

[juanluisbaptiste]

seems to work on all the other ones like this, except I just noticed this is the only one container running 6.0.30 version that has that problem - 6.0.29 containers are flawless and have the same configuration (except for the passwords). It strips the quotes from the original env file, which is:

The only change done from 6.0.29 to 6.0.30 was the OTRS package version, the rest of the container is the same, so it is very strange it is stripping the quotes from that variable out of nothing. I have some installs on 6.0.30 too but I have not changed the cron time with OTRS_BACKUP_TIME, I'm using the default value. I will test it out and see how it behaves for me.

[fajfer]

seems to work on all the other ones like this, except I just noticed this is the only one container running 6.0.30 version that has that problem - 6.0.29 containers are flawless and have the same configuration (except for the passwords). It strips the quotes from the original env file, which is:

The only change done from 6.0.29 to 6.0.30 was the OTRS package version, the rest of the container is the same, so it is very strange it is stripping the quotes from that variable out of nothing. I have some installs on 6.0.30 too but I have not changed the cron time with OTRS_BACKUP_TIME, I'm using the default value. I will test it out and see how it behaves for me.

The stripping is normal behaviour, as verified in previous version. I'll test it out soon on a fresh install as well

[juanluisbaptiste]

I tested it too and it worked without issues, so I'm closing this.

[fajfer]

I started getting these on other containers after the update - which is weird and I don't get it. I'm low-key trying to get to the bottom of this but for now I simply restart the OTRS in cron after doing backup

[juanluisbaptiste]

The last update just fixed a bug where installed addons could break on a restart, but it was only a move of a line of code from one place to another so it should not trigger any issues with the backups.

My best guess is that something else external to the container is making the backup process fail.

[fajfer]

The last update just fixed a bug where installed addons could break on a restart, but it was only a move of a line of code from one place to another so it should not trigger any issues with the backups.

My best guess is that something else external to the container is making the backup process fail.

Sorry for that, I kinda thought of 'since last version' as a timestamp rather than an actual issue on your side. Issue I have is that I'm getting lots of lots of emails to root@localhost that stay in the MailQueue and when they go to a really high number the containers fail to perform a backup. Haven't had the time to sit and fix all this so if someone's reading this I just went with:

docker exec -it <container name> /bin/bash
su – otrs
bin/otrs.Console.pl Maint::Email::MailQueue --delete --filter Recipient::root@localhost

Because this is also making the emails be send with an increasing delay

[juanluisbaptiste]

What is the content of those emails ? I checked some of my installations and none of them has any queued emails.

[fajfer]

Jan 10 15:49:21 62f2a5a77476 OTRS-otrs.Console.pl-Maint::Email::MailQueue-10[4833]: [Error][Kernel::System::CommunicationLog::_LogError][Line:538]: CommunicationLog(ID:978553,AccountType:-,AccountID:-,Direction:Outgoing,Transport:Email,ObjectLogType:Message,ObjectLogID:1065294)::Kernel::System::Email::SMTP => Envelope to 'root@localhost' not accepted by the server: 504, 5.5.2 <root@localhost>: Recipient address rejected: need fully-qualified address#012!
Jan 10 15:49:21 62f2a5a77476 OTRS-otrs.Console.pl-Maint::Email::MailQueue-10[4833]: [Error][Kernel::System::CommunicationLog::_LogError][Line:538]: CommunicationLog(ID:978553,AccountType:-,AccountID:-,Direction:Outgoing,Transport:Email,ObjectLogType:Message,ObjectLogID:1065294)::Kernel::System::Email => Error sending message using backend 'Kernel::System::Email::SMTP'.
Jan 10 15:49:21 62f2a5a77476 OTRS-otrs.Console.pl-Maint::Email::MailQueue-10[4833]: [Error][Kernel::System::CommunicationLog::_LogError][Line:538]: CommunicationLog(ID:978553,AccountType:-,AccountID:-,Direction:Outgoing,Transport:Email,ObjectLogType:Message,ObjectLogID:1065294)::Kernel::System::MailQueue => Message could not be sent! Error message: Envelope to 'root@localhost' not accepted by the server: 504, 5.5.2 <root@localhost>: Recipient address rejected: need fully-qualified address#012!
Jan 10 15:49:21 62f2a5a77476 OTRS-otrs.Console.pl-Maint::Email::MailQueue-10[4833]: [Error][Kernel::System::CommunicationLog::_LogError][Line:538]: CommunicationLog(ID:978553,AccountType:-,AccountID:-,Direction:Outgoing,Transport:Email,ObjectLogType:Message,ObjectLogID:1065294)::Kernel::System::MailQueue => Permanent sending problem or we reached the sending attempt limit. Message will be removed