search

wooey / Wooey

A Django app that creates automatic web UIs for Python scripts.
http://wooey.readthedocs.org
BSD 3-Clause "New" or "Revised" License
2.11k stars 182 forks source link

job expiration not working #328

Closed aydas closed 3 years ago

aydas commented 4 years ago

Hi ,

I still have problem with WOOEY_JOB_EXPIRATION parameter. I set this to:

user_job_expire=timedelta(hours=1) anonymous_job_expire=timedelta(minutes=10)

WOOEY_JOB_EXPIRATION = {'user':user_job_expire,'anonymous':anonymous_job_expire}

But no success. Please help me to fix

Aidas

zz138960 commented 4 years ago

task:cleanup-old-jobs run at midnight each day,so the results will not be verified until the next day celery_app.conf.beat_schedule.update({ 'cleanup-old-jobs': { 'task': 'wooey.tasks.cleanup_wooey_jobs', 'schedule': crontab(hour=0, minute=0), # cleanup at midnight each day }, 'cleanup-dead-jobs': { 'task': 'wooey.tasks.cleanup_dead_jobs', 'schedule': crontab(minute='*/10'), # run every 6 minutes } })

aydas commented 4 years ago

Hi, Thank you for answer. Problem is that system still keep all competed jobs from beginning (more then month) even after I set WOOEY_JOB_EXPIRATION parameter to 1 hour (in timedelta) for user.
So I guess problem is in parameter format or in celery side or somewhere else. Please help me to understand how to fix this.

zz138960 commented 4 years ago

Please confirm whether the beat command works normally，Has your celery received cleanup_wooey_jobs task or cleanup_dead_jobs task?

Chris7 commented 4 years ago

Keep in mind if you are running windows, celery beat is not really supported. I've found some ways to get it running, but the reliability probably varies and is not guaranteed between versions. https://wooey.readthedocs.io/en/latest/running_wooey.html#through-two-separate-processes

`` On Windows, the --beat option may not be supported and the eventlet pool will need to be specified. This looks like:

celery -A your_project_name worker --pool=eventlet -l info

aydas commented 4 years ago

wooey is running on ubuntu. python jobs is running and executed as excepted, Using rabbitmq. The problem is with scheduled tasks like cleanup-old-jobs only. Is there any way to check is cleanup-old-jobs started and completed successfully?

Chris7 commented 4 years ago

Scheduled tasks should be getting recorded in the celery output. Could you provide how you are running wooey and celery?

aydas commented 4 years ago

I setup production server on digitalocean droplet. As broker using rabbitmq. And set supervisor service. Celery see tasks: [tasks] . inovega.wooey_celery_app.debug_task . wooey.tasks.cleanup_dead_jobs . wooey.tasks.cleanup_wooey_jobs . wooey.tasks.submit_script

But log shows wooey.tasks.submit_script tasks only, no other tasks:

[2020-09-11 08:42:02,496: INFO/MainProcess] Received task: wooey.tasks.submit_script[19b1bfd1-fefc-4971-bb84-ec60245b2efa] [2020-09-11 08:42:03,443: INFO/ForkPoolWorker-1] Task wooey.tasks.submit_script[19b1bfd1-fefc-4971-bb84-ec60245b2efa] succeeded

Any ideas, that is wrong?

Aidas

Chris7 commented 3 years ago

How are you invoking hte celery command? I don't see anything indicating the beat option is being used.

aydas commented 3 years ago

Hi,

Thank you for answer. Celery is connected under supervisor service, in supervisor config file is inserted line: celery -A appname worker -l info

Can this service configuration cause inactivity of cleanup-old-jobs?

Aidas

2020-10-04, sk, 17:12 Chris Mitchell notifications@github.com rašė:

How are you invoking hte celery command? I don't see anything indicating the beat option is being used.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/wooey/Wooey/issues/328#issuecomment-703261572, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADGGQ2NVAPV2JOOWWHWHUMLSJB7DBANCNFSM4QMEOEUQ .

Chris7 commented 3 years ago

Hi @aydas ,

You need to add the --beat command to your celery worker. See https://docs.celeryproject.org/en/stable/userguide/periodic-tasks.html#starting-the-scheduler. It should be celery -A appname worker -l info --beat

aydas commented 3 years ago

Thank you, I will try to modify and test again

Aidas

2020-10-05, pr, 15:46 Chris Mitchell notifications@github.com rašė:

Hi @aydas https://github.com/aydas ,

You need to add the --beat command to your celery worker. See https://docs.celeryproject.org/en/stable/userguide/periodic-tasks.html#starting-the-scheduler. It should be celery -A appname worker -l info --beat

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wooey/Wooey/issues/328#issuecomment-703608393, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADGGQ2LWILGP2GMKHKX5GRTSJG54BANCNFSM4QMEOEUQ .

aydas commented 3 years ago

after changes to celery -A appname worker -l info --beat

wooey jobs goes to status waiting...

after changes to celery -A appname worker -l info (without --beat) and restarting supervisor tasks still on waiting?

This rabbitqm or celery problem?

Aidas

2020-10-05, pr, 15:49 Aidas Kaubrys aidas.kaubrys@gmail.com rašė:

Thank you, I will try to modify and test again

Aidas

2020-10-05, pr, 15:46 Chris Mitchell notifications@github.com rašė:

Hi @aydas https://github.com/aydas ,

You need to add the --beat command to your celery worker. See https://docs.celeryproject.org/en/stable/userguide/periodic-tasks.html#starting-the-scheduler. It should be celery -A appname worker -l info --beat

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wooey/Wooey/issues/328#issuecomment-703608393, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADGGQ2LWILGP2GMKHKX5GRTSJG54BANCNFSM4QMEOEUQ .

aydas commented 3 years ago

Hi Chris,

After change supervisor configuration and inserting beat command, scheduled cleanup tasks starts working but scripts stopped with status waiting, so I think problem is with celery configuration.

Aidas

2020-10-05, pr, 15:46 Chris Mitchell notifications@github.com rašė:

Hi @aydas https://github.com/aydas ,

You need to add the --beat command to your celery worker. See https://docs.celeryproject.org/en/stable/userguide/periodic-tasks.html#starting-the-scheduler. It should be celery -A appname worker -l info --beat

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wooey/Wooey/issues/328#issuecomment-703608393, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADGGQ2LWILGP2GMKHKX5GRTSJG54BANCNFSM4QMEOEUQ .

Chris7 commented 3 years ago

It seems to be. The management panel of your broker may be a good place to look to see if tasks are piling up there or being sent at all.

aydas commented 3 years ago

OK. thank you. for support.

I would like to iadd LT language into wooey, can you help me with some advise how-to-do, is it standart Django funkcionality or is something non standart

Aidas

2020-10-06, an, 15:43 Chris Mitchell notifications@github.com rašė:

It seems to be. The management panel of your broker may be a good place to look to see if tasks are piling up there or being sent at all.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wooey/Wooey/issues/328#issuecomment-704242680, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADGGQ2IG2ZNJMOBDRWNDF3LSJMGILANCNFSM4QMEOEUQ .

Chris7 commented 3 years ago

Hi @aydas ,

I love translations :). The translation system is part of the standard Django package. To add a new translation, you'd do this:

Run django-admin makemessages --locale the_locale_to_add from the wooey directory (where git clone drops you). This will create a directory under wooey/locale/the_locale_name with a .po file. This file is where you enter the human readable text. Following translations, you run django-admin compilemessages --locale the_locale_to_add.

How to add a new translations I think would be a great thing to add to the docs as well.

aydas commented 3 years ago

I will try, Thank you

2020-10-13, an, 16:44 Chris Mitchell notifications@github.com rašė:

Hi @aydas https://github.com/aydas ,

I love translations :). The translation system is part of the standard Django package. To add a new translation, you'd do this:

Run django-admin makemessages --locale the_locale_to_add from the wooey directory (where git clone drops you). This will create a directory under wooey/locale/the_locale_name with a .po file. This file is where you enter the human readable text. Following translations, you run django-admin compilemessages --locale the_locale_to_add.

How to add a new translations I think would be a great thing to add to the docs as well.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wooey/Wooey/issues/328#issuecomment-707748551, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADGGQ2P5UGXWVZXQIWCNHGLSKRKTTANCNFSM4QMEOEUQ .