mher / flower

Real-time monitor and web admin for Celery distributed task queue
https://flower.readthedocs.io
Other
6.44k stars 1.08k forks source link

Error 500 on Broker page #1357

Open scorpp opened 7 months ago

scorpp commented 7 months ago

Describe the bug Flower is deployed on Heroku and makes use of Heroku Redis as broker.

Heroku Redis uses self-signed certificates (hence the broker_use_ssl = {"ssl_cert_reqs": "none"} part in config below). And Workers / Tasks tabs work fine. Just the broker part is broken.

To Reproduce Steps to reproduce the behavior:

  1. Go to 'Broker' tab
  2. See error
   [W 240227 06:47:49 connection:268] Secure redis scheme specified (rediss) with no ssl options, defaulting to insecure SSL behaviour.
   [W 240227 06:47:49 connection:268] Secure redis scheme specified (rediss) with no ssl options, defaulting to insecure SSL behaviour.
   [E 240227 06:47:49 broker:31] Unable to get queues: 'Error 1 connecting to ec2-52-45-116-60.compute-1.amazonaws.com:6379. [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1006).'
   [W 240227 06:47:49 connection:268] Secure redis scheme specified (rediss) with no ssl options, defaulting to insecure SSL behaviour.
   [E 240227 06:47:49 web:1875] Uncaught exception GET /broker ([10.1.39.250](https://my.papertrailapp.com/systems/flower-heroku/events?q=%2210.1.39.250%22&focus=1699101004400787463&selected=1699101004400787463))
       HTTPServerRequest(protocol='http', host='flower-heroku.herokuapp.com', method='GET', uri='/broker', version='HTTP/1.1', remote_ip='[10.1.39.250](https://my.papertrailapp.com/systems/flower-heroku/events?q=%2210.1.39.250%22&focus=1699101004400787464&selected=1699101004400787464)')
       Traceback (most recent call last):
         File "/app/.heroku/python/lib/python3.11/site-packages/tornado/web.py", line 1790, in _execute
           result = await result
                    ^^^^^^^^^^^^
         File "/app/.heroku/python/lib/python3.11/site-packages/flower/views/broker.py", line 35, in get
           queues=queues)
                  ^^^^^^
       UnboundLocalError: cannot access local variable 'queues' where it is not associated with a value

Expected behavior Broker info is shown

Screenshots image

System information

$ python -c 'from flower.utils import bugreport; print(bugreport())'
flower   -> flower:2.0.1 tornado:6.4 humanize:4.9.0
software -> celery:5.3.6 (emerald-rush) kombu:5.3.5 py:3.11.8
            billiard:4.2.0 py-amqp:5.2.0
platform -> system:Linux arch:64bit, ELF
            kernel version:5.15.0-1053-aws imp:CPython
loader   -> celery.loaders.app.AppLoader
settings -> transport:amqp results:disabled

deprecated_settings: None

Command line

celery --broker=$REDIS_URL/1 flower --port=$PORT --config flowerconfig

floweconfig.py

import os

basic_auth  = [os.environ['BASIC_AUTH']]

broker_url  = os.environ['REDIS_URL'] + '/1'
broker_use_ssl = {"ssl_cert_reqs": "none"}

timezone    = os.getenv('TIMEZONE', 'US/Eastern')
jspablo commented 6 months ago

Facing same error with RabbitMQ as broker when visiting Broker page

jorge-menjivar commented 4 months ago

Facing the same issue with Redis on Redis Cloud

Legrems commented 4 months ago

Ensure that the broker_api is set accordingly to your config: example:

--broker="amqp://guest:guest@127.0.0.1:5672/

and

--broker-api="http://127.0.0.1:15672/api/

Note that the port change, as well as the auth isn't present in the settings (if you have one)

scorpp commented 4 months ago

i'm not using AMQP. my broker is Redis

fjcapdevila commented 2 months ago

Same problem here using Flower flower==2.0.1 with Sentinel+Redis as broker

[E 240723 14:46:33 web:1875] Uncaught exception GET /flower/broker (1.2.3.4)
    HTTPServerRequest(protocol='http', host='flower.example.com', method='GET', uri='/flower/broker', version='HTTP/1.1', remote_ip='1.2.3.4')
    Traceback (most recent call last):
      File "/home/vocalo/.venv/lib/python3.10/site-packages/tornado/web.py", line 1790, in _execute
        result = await result
      File "/home/vocalo/.venv/lib/python3.10/site-packages/flower/views/broker.py", line 35, in get
        queues=queues)
    UnboundLocalError: local variable 'queues' referenced before assignment
fjcapdevila commented 1 month ago

Quick update: I've also found logs like this:

Unable to get queues: 'unsupported operand type(s) for +: 'NoneType' and 'str''

I was able to fix the issue by setting global_keyprefix = '' I guess global_keyprefix is set to None by default and it makes Broker page fail.