tiangolo / full-stack

Full stack, modern web application generator. Using Flask, PostgreSQL DB, Docker, Swagger, automatic HTTPS and more.
MIT License
523 stars 81 forks source link

Flower not showing tasks/workers in development #16

Closed michaeltoohig closed 4 years ago

michaeltoohig commented 5 years ago

Is that normal for development?

tiangolo commented 5 years ago

Have you already modified the project after generation? Or could you create one that you can share with me so that I can take a look?

michaeltoohig commented 5 years ago

I have modified the project which may be the root of my problem but I couldn't see why. I'll re-download the project and try from baseline.

michaeltoohig commented 5 years ago

I have modified the project which may be the root of my problem but I couldn't see why. I'll re-download the project and try from baseline.

EDIT @tiangolo I did make a new project to test and documented my process of changes below

Download Project

.local/bin/cookiecutter https://github.com/tiangolo/full-stack

Re-download project and select default values

Init Build

cd base-project
docker-compose up -d

Edit files for development

My machine is windows but my development server is a remote nix machine so I need to change some settings following the guide in your README titled Development with a custom IP.

  1. Change DOMAIN in .env to dev.base-project.com
  2. Edit my hosts to add my IP maps to dev.base-project.com
  3. Change VUE_APP_DOMAIN_DEV in frontend/.env to dev.base-project.com

Test Celery Worker

Login to swagger succesfully and succesfully receive a response from the test-celery endpoint. Review logs in celeryworker see my word was recieved.

image

However log into Flower and see no records of workers or tasks. I do notice the /broker section of Flower is trying to connect to a broker at ampq://guest@192.168.0.1:5672 which I am not sure is correct or where that was configured.

image

Try New Celery Task

Add a new task to backend/app/app/worker.py

@celery_app.task(acks_late=True)
def test_2(num: int):
    print("test 2")
    return f"test 2 task return {num}"

Add new endpoint to test this task in backend/app/app/api/api_v1/endpoints/utils.py

@docs.register                                                                                                                        
@doc(description="Test Celery 2", security=security_params, tags=["utils"])
@app.route(f"{config.API_V1_STR}/test-two/", methods=["POST"])
@use_kwargs({"num": fields.Number(required=True)})
@marshal_with(MsgSchema())
@jwt_required
def route_test_2_celery(num):
    celery_app.send_task("app.worker.test_2", args=[num])
    return ({"msg": "Num received"}, 201)

Add new task to task_routes in celery_app. Also I notice the celery broker is defined amqp://guest@queue// which is maybe different than what I saw in flower?

image

Then restart both backend celeryworker. Test the new endpoint from swagger and the task works. So that means I have made a mistake when adding tasks somewhere. Thats a plus and the other issue I opened can be closed.

image

However Flower still does not show anything. about workers or tasks. Here is the repo I just set up.

https://github.com/michaeltoohig/full-stack

tiangolo commented 5 years ago

Sorry for the delay! I was busy building FastAPI.

There was a bug in the Flower service, it didn't have the env vars with the RabbitMQ host. I just fixed it.

The service flower should have the environment variables:

AMQP_ADMIN_HOST=queue
AMQP_HOST=queue

If you are using a recent version of the generator, there is a file env-flower.env, with:

FLOWER_BASIC_AUTH=admin:somesecretpassword

Update it to have:

FLOWER_BASIC_AUTH=admin:somesecretpassword
AMQP_ADMIN_HOST=queue
AMQP_HOST=queue
tiangolo commented 4 years ago

Hi @michaeltoohig , I'll assume you were able to solve your problem and I'll close this issue now.


If you are still using this project, I suggest you check the equivalent project generator for FastAPI that solves the same use cases in a much better way.

Because of that, this Flask-based project generator is now going to be deprecated. You are still free to use it, but it won't receive any new features, changes, or bug fixes.