search

rija / gigadb-website

Source code for running GigaDB
http://gigadb.org
GNU General Public License v3.0
1 stars 1 forks source link

Configure container logging #200

Closed rija closed 3 years ago

rija commented 3 years ago

Configuration of logging

This PR focus on logging and implements the following:

  1. Setup log rotation for container services
  2. Set YII_DEBUG to true on all environments but the live one
  3. Tighten logging settings for PHP-FPM
  4. Add tests to check some of the logging setup
  5. Update Yii main config to enable logging at info level
  6. Configure nginx to log an error when an error is thrown by the application
  7. Update composer to make some autoload scenario explicit

1. Setup log rotation for container services

This is so that we don't run out of disk space (especially on staging and live deployment to EC2 instances) because of out of control logs. This is done in docker-compose.yml and docker-compose-production-envs.yml. To avoid duplciation, we use YAML anchors to define the settings once in the file (in the x-logging block) and refers to it in the specification of all container services that are to run in detached/daemon mode.

2. Set YII_DEBUG to true on all environments but the live one

This will make entries in application more verbose with more context when debugging and logg debug messages. It work on dev and CI to observe and trace work in progress, and on staging so we can debug ready to go features in a live like configuration. It is disabled in live to avoid performance penalty. Those changes are implmented in the Yii framework entry point file for web application, their template version: index.dev.php.dist, index.CI.php.dist, index.staging.php.dist, index.live.php.dist.

3. Tighten logging settings for PHP-FPM

The configuration of logging in PHP-FPM was relying on default value, and might not be complete. So gigadb.pool.conf and php-7.1-(dev|CI|staging|production).ini files are updated to docker style logging: everything goes to STDOUT/STDERR. /proc/self/fd/2 means STDERR.

4. Add tests to check some of the logging setup

There is a new smoke test tests/logging_check that check that the YII_DEBUG is set to true, that output from application and Composer library are output to the log (that's the UUID part of Sitecontroller.php's change) and that HTTP errors (like 404) appear in the web container log. That test is run automatically on CI by adding in tests/all_and_coverage

5. Update Yii main config to enable logging at info level

Previously only warning and error level of Yii logging appear in logs, which caused many informative notice to be written as warning which is erroneous. Additionally, 'info' type of message are useful for data analytics on application (and its users) behaviour and can help with light debugging on live environment. This change is implemented in main.php.dist

6. Configure nginx to log an error when an error is thrown by the application

configure Nginx to capture and log any error thrown by PHP-FPM to reduce changes for an application problem goes ignored because applicaiton logging was absent or malfunctioning. This change is implemented in nginx.target_deployment.https.conf.dist and nginx.target_deployment.http.conf.dist

7. Update composer to make some autoload scenario explicit

Some Composer librairies may use the autoloader info built up by Composer to know how to find code they need to work with (especially for reflection/debugging/inspections/logging libraries), so I've updated composer.json.dist to help the application code in PSR-0 global namespace we have in protected/models/ and protected/components be discoverable.

pli888 commented 3 years ago

When I try to deploy a local GigaDB website, I get the following error:

...
ERROR: The Compose file is invalid because:
Service gigadb has neither an image nor a build context specified. At least one must be provided.

@pli888, I cannot reproduce that problem. And the CI test run also uses up.sh and it's building and passing on CI. There's no reference to gigadb service in up.sh and docker-compose.yml on that branch (this PR actually removes the gigadb block from the latter file). Can you share the value of your COMPOSE_FILE variable in .env?

rija commented 3 years ago

Hi @pli888,

I've reproduced your error by setting 

COMPOSE_FILE=ops/deployment/docker-compose.yml:ops/deployment/docker-compose.prof.yml

It turns out I forgot to remove the no longer necessary specification of gigadb service in the docker-compose file for profiling. I've fixed it now. Thanks for spotting.