Allow another logging driver such as journald/syslog/splunk

jean-christophe-manciot commented 5 years ago

ISSUE TYPE

Feature Idea

COMPONENT NAME

Installer

SUMMARY

By default, the logging driver is json-file and destroying a container will also remove logs. Changing it to journald for instance would allow us to access the logs of destroyed containers, on top of everything that is already possible with the default logging driver. In particular, the docker logs command remains available with journald (and only that one). This feature can be made optional in installer/inventory with an option within the [all:vars] section:

logging_driver=journald

The user could even choose another logging driver such as syslog or splunk (the whole list of supported logging drivers is available here), if the user is notified somewhere that the docker logs command does NOT remain available for any other logging driver than journald.

ADDITIONAL INFORMATION

This feature is enabled in the docker-compose.yml file for journald for instance with:

  web:
    logging:
        driver: journald
...
  task:
    logging:
        driver: journald
...
  rabbitmq:
    logging:
        driver: journald
...
  memcached:
    logging:
        driver: journald
...
  postgres:
    logging:
        driver: journald
...

The last entry depends on whether postgresql server is internal or external.

jean-christophe-manciot commented 5 years ago

As a PoC, I performed the following 2.1.0 build from scratch (the variables are of course initialized beforehand):

git checkout 2.1.0
sed -i "s|.*postgres_data_dir=.*|postgres_data_dir=${postgres_data_dir}|g" installer/inventory
sed -i "s|.*docker_compose_dir=.*|docker_compose_dir=${docker_compose_dir}|g" installer/inventory
sed -i "s|.*use_docker_compose=.*|use_docker_compose=true|g" installer/inventory
sed -i "s|.*project_data_dir=.*|project_data_dir=${project_data_dir}|g" installer/inventory
echo "logging_driver=journald" >> installer/inventory
for awx_container in web task rabbitmq memcached postgres
do
        sed -Ei "/^ *${awx_container}:/a\ \ \ \ logging:\n      driver: {{ logging_driver }}" installer/roles/local_docker/templates/docker-compose.yml.j2
done

cd installer
ansible-playbook -i inventory install.yml

So far, it works perfectly:


# docker logs -f docker-compose_postgres_1_8f2a9fce4b53
LOG:  database system was shut down at 2018-11-08 15:02:16 UTC
LOG:  MultiXact member wraparound protections are now enabled
LOG:  autovacuum launcher started
LOG:  database system is ready to accept connections
LOG:  incomplete startup packet
LOG:  incomplete startup packet
ERROR:  duplicate key value violates unique constraint "auth_user_username_key"
DETAIL:  Key (username)=(admin) already exists.
STATEMENT:  INSERT INTO "auth_user" ("password", "last_login", "is_superuser", "username", "first_name", "last_name", "email", "is_staff", "is_active", "date_joined") VALUES ('pbkdf2_sha256$36000$AljspYT1PuCx$l5qD4pIGcXD/+wx2fWtcMBndrih0Sqy0Jh8iZbb4B/Q=', NULL, true, 'admin', '', '', 'root@localhost', true, true, '2018-11-08T15:07:16.168434+00:00'::timestamptz) RETURNING "auth_user"."id"
^C
# journalctl -u docker CONTAINER_NAME=docker-compose_postgres_1_8f2a9fce4b53
-- Logs begin at Thu 2018-11-08 12:02:56 CET, end at Thu 2018-11-08 16:08:08 CET. --
Nov 08 16:07:02 samsung5-debian 901029748f03[5956]: LOG:  database system was shut down at 2018-11-08 15:02:16 UTC
Nov 08 16:07:02 samsung5-debian 901029748f03[5956]: LOG:  MultiXact member wraparound protections are now enabled
Nov 08 16:07:02 samsung5-debian 901029748f03[5956]: LOG:  autovacuum launcher started
Nov 08 16:07:02 samsung5-debian 901029748f03[5956]: LOG:  database system is ready to accept connections
Nov 08 16:07:03 samsung5-debian 901029748f03[5956]: LOG:  incomplete startup packet
Nov 08 16:07:04 samsung5-debian 901029748f03[5956]: LOG:  incomplete startup packet
Nov 08 16:07:16 samsung5-debian 901029748f03[5956]: ERROR:  duplicate key value violates unique constraint "auth_user_username_key"
Nov 08 16:07:16 samsung5-debian 901029748f03[5956]: DETAIL:  Key (username)=(admin) already exists.
Nov 08 16:07:16 samsung5-debian 901029748f03[5956]: STATEMENT:  INSERT INTO "auth_user" ("password", "last_login", "is_superuser", "username", "first_name", "```

megakoresh commented 4 years ago

This resolves also use-case with logging aggregators, since docker would handle formatting, so e.g. in fluentd you don't need to build regexes to parse the output anymore

ansible / awx