Closed nuno-andre closed 8 months ago
I see another problem too: on the first run, CPU usage goes to 100%. I don't know why, but I'll investigate.
I found the problem: it was some leftover logic from a previous, unsuccessful attempt at implementing configure_task()
. I have to investigate the other problem before I push the fix though.
When I fix the old logic, it still uses up 100% CPU without doing anything useful. This only happens with aiosqlite and sqlite; it works correctly with PostgreSQL.
Something is seriously wrong with the sqlite logic:
DEBUG:apscheduler._schedulers.async_:Sleeping -263.660 seconds until the next fire time (2023-11-12 02:00:46.513514+02:00)
DEBUG:apscheduler._schedulers.async_:Sleeping -263.661 seconds until the next fire time (2023-11-12 02:00:46.513514+02:00)
DEBUG:apscheduler._schedulers.async_:Sleeping -263.663 seconds until the next fire time (2023-11-12 02:00:46.513514+02:00)
It's looking like acquire_schedules()
is returning no results. This is puzzling, as there are several tests passing against the sqlite stores and I don't know why they would pass but this won't work. I suspect it has to do with the lack of date/time handling in sqlite.
Got it: it's due to sqlite doing timestamp comparisons as text. This would've been fine without the time zone offsets, but they make datetime comparison difficult if not impossible on sqlite. The reason this wasn't caught by tests is because the test data used UTC offsets which the data store also uses for queries, so it all worked.
I solved it by special casing sqlite so that the index is created on the expression unix_epoch(next_fire_time)
and the same function is compared against now.timestamp()
. I'll push the fix tomorrow once I have accompanying tests in place.
As soon as I updated the tests to use a positive UTC offset in the timestamps, I discovered that MySQL had a similar issue. My initial fix for that hasn't passed the tests yet.
Ok...so the solution that works for MySQL only works for MySQL 8.0 onwards, and doesn't work on MariaDB :disappointed: Back to the drawing board I guess...
After a really ugly workaround, everything seems to be working now.
Maybe using only UTC internally and making the conversion in the input/output interfaces...
I have been bitten by timezone messes too many times... Now I only use them in user interfaces.
BTW, this seems an amazing refactoring. I've pinned it to start using it ASAP.
Maybe using only UTC internally and making the conversion in the input/output interfaces...
Won't work, as we need the actual time zone information too which would be lost this way.
Things to check first
[X] I have checked that my issue does not already have a solution in the FAQ
[X] I have searched the existing issues and didn't find my bug already reported there
[X] I have checked that my bug is still present in the latest release
Version
https://github.com/agronholm/apscheduler/commit/a67a82c821d5eeb882192d453dea215f7e8987bf
What happened?
When the scheduler tries to configure a task created in a previous run raises a validation error. This doesn't happen when the task is created for the first time.
(I've come across this error both with SQLite and PostgreSQL.)
Commenting the following lines makes the code to run successfully.
https://github.com/agronholm/apscheduler/blob/a67a82c821d5eeb882192d453dea215f7e8987bf/src/apscheduler/_schedulers/async_.py#L307-L309
How can we reproduce the bug?
Run this code (it will create a new DB), stop it, and run it again (with the previously created DB).