when release to V0.6.9，there wasn't log in docker compose

[skylong5257]

Self Checks

• [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.9

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

1，docker-compose.yaml 1）had set LOG_LEVEL: DEBUG 2，DEBUG : true。had tried to enable it or disable it.

2，entrypoint.sh tried to add log-level in gunicorn else if [[ "${DEBUG}" == "true" ]]; then echo "DEBUG is on" flask run --host=${DIFY_BIND_ADDRESS:-0.0.0.0} --port=${DIFY_PORT:-5001} --debug else echo "DEBUG is off" gunicorn \ --log-level debug \ --bind "${DIFY_BIND_ADDRESS:-0.0.0.0}:${DIFY_PORT:-5001}" \ --workers ${SERVER_WORKER_AMOUNT:-1} \ --worker-class ${SERVER_WORKER_CLASS:-gevent} \ --timeout ${GUNICORN_TIMEOUT:-200} \ --preload \ app:app fi

3，add some log for testing in current code class LoginApi(Resource): """Resource for user login."""

@setup_required
def post(self):
    """Authenticate user and login."""
    logging.info("LoginApi test info")
    logging.debug("LoginApi test debug")
    parser = reqparse.RequestParser()
    parser.add_argument('email', type=email, required=True, location='json')
    parser.add_argument('password', type=valid_password, required=True, location='json')
    parser.add_argument('remember_me', type=bool, required=False, default=False, location='json')
    args = parser.parse_args()

    # todo: Verify the recaptcha

    try:
        logging.info("test ------ login with email: %s", args['email'])

4，below is the output（based on: DEBUG : true） docker-api-1 | None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used. docker-api-1 | WARNING:root:MAIL_TYPE is not set docker-api-1 | Serving Flask app 'app.py' docker-api-1 | Debug mode: on docker-api-1 | INFO:werkzeug:WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. docker-api-1 | Running on all addresses (0.0.0.0) docker-api-1 | Running on http://127.0.0.1:5001 docker-api-1 | Running on http://172.31.0.7:5001 docker-api-1 | INFO:werkzeug:Press CTRL+C to quit docker-api-1 | INFO:werkzeug: Restarting with stat docker-api-1 | None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used. docker-api-1 | WARNING:root:MAIL_TYPE is not set docker-api-1 | WARNING:werkzeug: Debugger is active! docker-api-1 | INFO:werkzeug: Debugger PIN: 743-061-682 docker-api-1 | INFO:werkzeug:172.31.0.10 - - [05/Jun/2024 23:28:52] "GET /console/api/logout HTTP/1.1" 200 - docker-api-1 | INFO:werkzeug:172.31.0.10 - - [05/Jun/2024 23:28:52] "GET /console/api/system-features HTTP/1.1" 200 - docker-api-1 | INFO:werkzeug:172.31.0.10 - - [05/Jun/2024 23:29:09] "POST /console/api/login HTTP/1.1" 200 - docker-api-1 | INFO:werkzeug:172.31.0.10 - - [05/Jun/2024 23:29:10] "GET /console/api/apps?page=1&limit=30&name= HTTP/1.1" 200 - docker-api-1 | INFO:werkzeug:172.31.0.10 - - [05/Jun/2024 23:29:10] "GET /console/api/account/profile HTTP/1.1" 200 - docker-api-1 | INFO:werkzeug:172.31.0.10 - - [05/Jun/2024 23:29:10] "GET /console/api/workspaces/current HTTP/1.1" 200 - docker-api-1 | INFO:werkzeug:172.31.0.10 - - [05/Jun/2024 23:29:10] "GET /console/api/version?current_version=0.6.9 HTTP/1.1" 200 - docker-api-1 | INFO:werkzeug:172.31.0.10 - - [05/Jun/2024 23:29:10] "GET /console/api/tags?type=app HTTP/1.1" 200 - docker-api-1 | INFO:werkzeug:172.31.0.10 - - [05/Jun/2024 23:29:10] "GET /console/api/features HTTP/1.1" 200 - docker-api-1 | INFO:werkzeug:172.31.0.10 - - [05/Jun/2024 23:29:10] "GET /console/api/workspaces HTTP/1.1" 200 - docker-api-1 | INFO:werkzeug:172.31.0.10 - - [05/Jun/2024 23:29:10] "GET /console/api/apps?page=1&limit=30&name= HTTP/1.1" 200 - docker-api-1 | INFO:werkzeug:172.31.0.10 - - [05/Jun/2024 23:29:10] "GET /console/api/datasets/retrieval-setting HTTP/1.1" 200 - docker-api-1 | INFO:werkzeug:172.31.0.10 - - [05/Jun/2024 23:29:10] "GET /console/api/workspaces/current/model-providers HTTP/1.1" 200 - docker-api-1 | INFO:werkzeug:172.31.0.10 - - [05/Jun/2024 23:29:10] "GET /console/api/workspaces/current/models/model-types/llm HTTP/1.1" 200 -

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

[dosubot[bot]]

Hello, @skylong5257! I'm here to help you with any bugs, questions, or contributions you have for the project. Let me know how I can assist you while waiting for a human maintainer.

To ensure that logs are visible in Docker Compose when running version 0.6.9, you need to set the LOG_LEVEL environment variable to an appropriate value such as DEBUG, INFO, WARNING, ERROR, or CRITICAL. This configuration should be present in your docker-compose.yaml file.

Here is an example configuration:

services:
  api:
    image: langgenius/dify-api:0.6.9
    restart: always
    environment:
      LOG_LEVEL: DEBUG
      DEBUG: true

Additionally, ensure that the logging configuration in the create_app function within the api/app.py file is correctly set up to handle the log level and output to sys.stdout, which Docker captures:

log_handlers = [
    logging.StreamHandler(sys.stdout)
]

logging.basicConfig(
    level=app.config.get('LOG_LEVEL', 'DEBUG'),
    format=app.config.get('LOG_FORMAT', '%(asctime)s - %(name)s - %(levelname)s - %(message)s'),
    datefmt=app.config.get('LOG_DATEFORMAT', '%Y-%m-%d %H:%M:%S'),
    handlers=log_handlers
)

Make sure that the entrypoint.sh script does not override these settings. The script should respect the LOG_LEVEL and DEBUG environment variables without additional logging configurations that might interfere.

If you have already set these configurations and still do not see the logs, it might be worth checking if there are any specific dependencies or packages that could interfere with logging output. However, based on the provided information, there are no known issues or specific dependencies in version 0.6.9 that should affect logging output in Docker Compose ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[skylong5257]

@dosu Thanks for your quick response. I think the code in my space was same with your renmensioned. 1，docker-compose.yaml version: '3.1' services:

API service

api: image: dify-api:latest

image: langgenius/dify-api:0.5.6

restart: always
environment:
  # Startup mode, 'api' starts the API server.
  MODE: api
  # The log level for the application. Supported values are `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`
  LOG_LEVEL: DEBUG
  # enable DEBUG mode to output more logs
  DEBUG : true

2，app.py def create_app() -> Flask: app = DifyApp(name) app.config.from_object(Config())

app.secret_key = app.config['SECRET_KEY']

log_handlers = None
log_file = app.config.get('LOG_FILE')
if log_file:
    log_dir = os.path.dirname(log_file)
    os.makedirs(log_dir, exist_ok=True)
    log_handlers = [
        RotatingFileHandler(
            filename=log_file,
            maxBytes=1024 * 1024 * 1024,
            backupCount=5
        ),
        logging.StreamHandler(sys.stdout)
    ]
logging.basicConfig(
    level=app.config.get('LOG_LEVEL'),
    format=app.config.get('LOG_FORMAT'),
    datefmt=app.config.get('LOG_DATEFORMAT'),
    handlers=log_handlers
)

initialize_extensions(app)
register_blueprints(app)
register_commands(app)

return app

After my reading the historical version, I will try to use below code for the testing: logging.basicConfig(level=app.config.get('LOG_LEVEL', 'INFO'))

[dosubot[bot]]

Your configuration looks correct, but there are a few things to double-check and try:

1. Ensure log_handlers is always set: If log_file is not set, log_handlers will be None, and logging.basicConfig won't have any handlers. Ensure log_handlers is always a list of handlers, even if it's just StreamHandler.

2. Set default log level: Ensure the default log level is set to DEBUG if not specified.

Here is an updated version of your create_app function:

def create_app() -> Flask:
    app = DifyApp(__name__)
    app.config.from_object(Config())

    app.secret_key = app.config['SECRET_KEY']

    log_handlers = [
        logging.StreamHandler(sys.stdout)
    ]
    log_file = app.config.get('LOG_FILE')
    if log_file:
        log_dir = os.path.dirname(log_file)
        os.makedirs(log_dir, exist_ok=True)
        log_handlers.append(
            RotatingFileHandler(
                filename=log_file,
                maxBytes=1024 * 1024 * 1024,
                backupCount=5
            )
        )

    logging.basicConfig(
        level=app.config.get('LOG_LEVEL', 'DEBUG'),
        format=app.config.get('LOG_FORMAT', '%(asctime)s - %(name)s - %(levelname)s - %(message)s'),
        datefmt=app.config.get('LOG_DATEFORMAT', '%Y-%m-%d %H:%M:%S'),
        handlers=log_handlers
    )

    initialize_extensions(app)
    register_blueprints(app)
    register_commands(app)

    return app

This ensures that log_handlers always includes StreamHandler, and the default log level is set to DEBUG.

Additionally, make sure your docker-compose.yaml is correctly set up to pass the environment variables:

version: '3.1'
services:
  api:
    image: dify-api:latest
    restart: always
    environment:
      MODE: api
      LOG_LEVEL: DEBUG
      DEBUG: 'true'

Finally, ensure that the entrypoint.sh script does not override these settings. The script should respect the LOG_LEVEL and DEBUG environment variables without additional logging configurations that might interfere.

If you still do not see the logs, it might be worth checking if there are any specific dependencies or packages that could interfere with logging output. However, based on the provided information, there are no known issues or specific dependencies in version 0.6.9 that should affect logging output in Docker Compose ^[1][2].

To continue talking to Dosu, mention @dosu.

[yaohongfenglove]

It maybe a bug, I also encountered it. The problem is that before the execution of the logging.basicConfig() in create.app(), other log configurations were executed, as a result, the configuration was overwritten.  I solved it temporarily like this: Add a line of code to the app. py, to import a class from a_fix_logger.py

app.py add "from a_fix_logger import LogInitializer" before "from commands import register_commands"

import json
import logging
import sys
import threading
import time
import warnings
from logging.handlers import RotatingFileHandler

from flask import Flask, Response, request
from flask_cors import CORS
from werkzeug.exceptions import Unauthorized

from a_fix_logger import LogInitializer  # add
from commands import register_commands
from config import Config

a_fix_logger.py

import logging
import os
import sys
from logging.handlers import RotatingFileHandler

from config import Config

class LogInitializer:

    config = Config()

    log_handlers = None
    log_file = config.LOG_FILE
    if log_file:
        log_dir = os.path.dirname(log_file)
        os.makedirs(log_dir, exist_ok=True)
        log_handlers = [
            RotatingFileHandler(
                filename=log_file,
                maxBytes=1024 * 1024 * 1024,
                backupCount=5
            ),
            logging.StreamHandler(sys.stdout)
        ]

    logging.basicConfig(
        level=config.LOG_LEVEL,
        format=config.LOG_FORMAT,
        datefmt=config.LOG_DATEFORMAT,
        handlers=log_handlers
    )

[skylong5257]

Thanks @yaohongfenglove , it was working based on your workaround. @dosu could you please push to fix this issue? hope your updated version.

[dosubot[bot]]

To address the logging issue in Docker Compose for version 0.6.9 of the Dify application, you can use the workaround provided by @yaohongfenglove. This involves initializing the logging configuration before any other log configurations are executed. Here is the updated approach:

1. Create a new file a_fix_logger.py:

# a_fix_logger.py

import logging
import os
import sys
from logging.handlers import RotatingFileHandler

from config import Config

class LogInitializer:
    config = Config()

    log_handlers = [
        logging.StreamHandler(sys.stdout)
    ]
    log_file = config.LOG_FILE
    if log_file:
        log_dir = os.path.dirname(log_file)
        os.makedirs(log_dir, exist_ok=True)
        log_handlers.append(
            RotatingFileHandler(
                filename=log_file,
                maxBytes=1024 * 1024 * 1024,
                backupCount=5
            )
        )

    logging.basicConfig(
        level=config.LOG_LEVEL,
        format=config.LOG_FORMAT,
        datefmt=config.LOG_DATEFORMAT,
        handlers=log_handlers
    )

2. Modify app.py to import and use LogInitializer:

# app.py

import json
import logging
import sys
import threading
import time
import warnings
from logging.handlers import RotatingFileHandler

from flask import Flask, Response, request
from flask_cors import CORS
from werkzeug.exceptions import Unauthorized

from a_fix_logger import LogInitializer  # Add this line
from commands import register_commands
from config import Config

def create_app() -> Flask:
    app = DifyApp(__name__)
    app.config.from_object(Config())

    app.secret_key = app.config['SECRET_KEY']

    # Initialize logging
    LogInitializer()

    initialize_extensions(app)
    register_blueprints(app)
    register_commands(app)

    return app

This ensures that the logging configuration is set up before any other log configurations can override it.

Additional Steps for Deployment

1. Clone Dify Source Code:

   git clone https://github.com/langgenius/dify.git

2. Start Dify: Navigate to the docker directory in the Dify source code and execute the following command to start Dify:

   cd dify/docker
   docker compose up -d

3. Check Deployment: Verify that all containers are running successfully:

   docker compose ps

4. Upgrade Dify: To upgrade Dify, enter the docker directory of the dify source code and execute the following commands:

   cd dify/docker
   git pull origin main
   docker compose down
   docker compose pull
   docker compose up -d

5. Access Dify: Finally, access http://localhost/install to use the deployed Dify.

6. Customize Dify: The full set of environment variables can be found under docker/dotenvs. To change their values, delete the .example suffix from the corresponding .env.example file name and edit the values in the file directly. Then, restart Dify with:

   docker compose down
   docker compose up -d

This should resolve the logging issue in Docker Compose for version 0.6.9 of the Dify application ^[1][2].

@takatost, this issue seems to require your expertise to provide a solution based on the latest user workaround.

To continue talking to Dosu, mention @dosu.

[zhangever]

Anyone help on this??

[crazywoola]

@zhangever Have you tried the latest version?