Weekly Scheduled Jobs not Running

ganeshapp commented 4 years ago

Issue Summary

Weekly Scheduled Queries not running for Postgres, Presto and Redshift.

Steps to Reproduce

Create new query in Redash V9. Beta
Save, publish, execute it.
Schedule it for 1 Week job, repeating every Sunday or Monday (any day) etc. and select time 8:00 AM (any time)
Is not queued on Sunday 8:00 AM (day and time is just example).

Any other info e.g. Why do you consider this to be a bug? What did you expect to happen instead? Same query scheduled to run every 30 minutes or every 1 day at 8:00 AM gets queued and executed fine. Same Query scheduled to run 1W on Sunday 8:00 AM runs fine on Redash V8.

Technical details:

Redash Version: redash/preview:9.0.0-beta.b43709, redash/preview:9.0.0-beta.b43494, redash/preview:9.0.0-beta.b42832, redash/preview:9.0.0-beta.b42755
Browser/OS: Chrome, Firefox Mac OS
How did you install Redash: setup.sh to install V8 and then followed steps here to get to V9: https://github.com/getredash/redash/blob/master/CHANGELOG.md#v900-beta---2020-06-11

itssimon commented 4 years ago

We're seeing the same behavior too. Happy to submit a PR if anyone has a clue where this goes wrong?

yyzhou0811 commented 4 years ago

I have the same problem.how to solve it?

xuanyuanaosheng commented 4 years ago

can not wait for the new version

tim5go commented 4 years ago

@ganeshapp Is it related to the JOB EXPIRY TIME (by default, it is 3600 * 12 seconds) ? https://github.com/getredash/redash/blob/90024ebc92ebd952d7ec8c085eb70132642564da/redash/tasks/queries/execution.py#L113

ganeshapp commented 4 years ago

@tim5go If that is the case 3600 * 12 seconds = 12 hours, then even the daily jobs should fail right? But the daily jobs run. Only the weekly jobs don't get scheduled.

susodapop commented 2 years ago

JOB_EXPIRY_TIME doesn't cause this. Because the job isn't enqueued until the query actually needs to run. This is what the scheduler service does. It runs twice each minute and checks which scheduled queries need to be enqueued now.

So what causes it? I don't know. But I've just self-assigned this and will investigate.

mbra commented 2 years ago

Could it be something like this: https://github.com/solute/redash/commit/1a3e13084e7b26c9c0b0e49730b115b5c66332a2

I stumble upon this while fumbling around with parameterized queries (before I read in the docs scheduled parameterized queries are not supported right now, even though you can activate a schedule, it just does not work properly if you do).

So while hunting down my perceived "bug" I found something that seemed wrong to me and "fixed" it with the above commit. If the analysis in the commit message is stupid, I'd also be happy to know why the original code is fine, because I am obviously not getting what is going on then. :-(

getredash / redash