504 timeout error when a new incident is created

[CodrinE]

Hello,

I'm new to the cachet. We've recently implemented cachet for our company status page. We have more than 500 subscribers in the system.

When I create a new incident in the system, it throws an error 504 after a few minutes and it does not create an incident after sending out just a few emails.

Cachet Version: 2.4.0-dev

Stack configuration:

version: "3.7"

services:
  cachet:
    image: cachet:staging_v2
    container_name: cachet
    depends_on:
      - memcached
      - redis
      - database
    links:
      - redis
      - database
      - memcached
    ports:
      - 81:8000
    networks:
      - traefik
      - reverse_proxy
    environment:
      - APP_KEY={app.key}
      - APP_URL={app.url}
      - APP_ENV=production
      - CACHE_DRIVER=redis
      - QUEUE_DRIVER=redis
      - SESSION_DRIVER=memcached
      - MEMCACHED_HOST=memcached
      - MEMCACHED_PORT=11211
      - REDIS_HOST=redis
      - REDIS_ENABLED=true
      - REDIS_DATABASE=0
      - REDIS_PORT=6379
      - REDIS_PREFIX='statuspage'
      - CACHET_EMOJI=false
      - CACHET_BEACON=false
      - CACHET_AUTO_TWITTER=true
      - APP_LOG=errorlog
      - APP_DEBUG=false
      - DB_DRIVER=mysql
      - DB_HOST=database
      - DB_DATABASE=cachet
      - DB_USERNAME=cachet
      - DB_PASSWORD={db.pass}
      - MAIL_DRIVER=smtp
      - MAIL_HOST=localhost
      - MAIL_PORT=1025
      - MAIL_USERNAME={mail.username}
      - MAIL_PASSWORD={mail.password}
      - MAIL_ADDRESS={mail.address}
      - MAIL_NAME={mail.name}
      - MAIL_ENCRYPTION=null
      - WRAPPER_MAIL_HOST={wmail.host}
      - WRAPPER_MAIL_PORT=587
      - WEBHOOK_URL=""
      - PHP_MAX_CHILDREN=50
      - EMAIL_INTERVAL=15
      - SESSION_SECURE_COOKIE=true
    deploy:
      replicas: 1
      restart_policy:
        delay: 10s
        max_attempts: 1
        window: 60s
      placement:
        constraints:
          - "node.role == manager"
      labels:
        - "traefik.enable=true"
        - "traefik.port=8000"
        - "traefik.docker.network=reverse_proxy"
        - "traefik.passHostHeader=true"
        - "traefik.protocol=http"
        - "traefik.frontend.rule=Host:my.host.com"
        - "traefik.cachet.frontend.entryPoints=http,https"
        - "traefik.cachet.frontend.headers.SSLRedirect=true"
        - "traefik.backend=cachet"

    volumes:
      - /shared/statuspage/cachet/data/supervisord.conf:/etc/supervisor/supervisord.conf
      - cachet-data:/var/www/html

  cachet-url-monitor:
    image: mtakaki/cachet-url-monitor:release-0.6.9
    container_name: url-mon
    depends_on:                                                    
      - cachet
    networks:
      - traefik
      - reverse_proxy
    volumes:
      - /shared/statuspage/cachet-url-mon:/usr/src/app/config

  redis:
    image: redis:5
    hostname: redis
    user: redis
    container_name: redis
    ports:
      - 6379:6379
    networks:
      - reverse_proxy
      - traefik
    volumes:
      - /shared/statuspage/redis-data:/data
    labels:                                                                             
      - "traefik.enable=false" 

  memcached:
    image: memcached:1.5-alpine
    container_name: memcached
    ports:
      - "11211:11211"
    labels:
      - "traefik.enable=false"
    networks:
      - reverse_proxy
      - traefik

networks:
  traefik:
    external: true
    driver: overlay
    attachable: true
  reverse_proxy:
    external: true
    driver: overlay
    attachable: true
volumes:
  cachet-data:
    driver: local

supervisord.conf configuration:

[program:queue-worker]
process_name=%(program_name)s_%(process_num)02d
command=php artisan queue:work redis --delay=1 --sleep=1 --tries=3 --timeout=1800 --queue=high,low
directory=/var/www/html/
redirect_stderr=true
autostart=true
autorestart=true
numprocs=10
stdout_logfile_maxbytes=0
stderr_logfile_maxbytes=0
stdout_logfile=/dev/stdout
user=www-data

This is the message that I get in the output log when it times out:

[26-Aug-2020 10:10:47] WARNING: [pool www] child 180, script '/var/www/html/public/index.php' (request: "POST /index.php") execution timed out (139.792942 sec), terminating                                       
[26-Aug-2020 10:10:47] WARNING: [pool www] child 180 exited on signal 15 (SIGTERM) after 162.900882 seconds from start                                                                                             
[26-Aug-2020 10:10:47] NOTICE: [pool www] child 525 started  

[CodrinE]

I have increased the Nginx timeout to 300s but it still times out after the time has passed. Any fixes or workarounds?

[jbrooksuk]

Thank you for your input on Cachet 2.x. We are shifting our attention and resources to Cachet 3.x and will no longer be supporting the 2.x version. If your feedback or issue is relevant to the 3.x series, we encourage you to engage with the new branch.

For more information on the Cachet rebuild and our plans for 3.x, you can read the announcement here.

We appreciate your understanding and look forward to your contributions to the new version.