dbader / schedule

Python job scheduling for humans.
https://schedule.readthedocs.io/
MIT License
11.77k stars 961 forks source link

Why did three separate python scripts using schedule start at different time eventually converged to be in sync? #614

Open fireattack opened 7 months ago

fireattack commented 7 months ago

This is more of a question than bug report, but this phenomenon has been confusing me for years.

I guess a this is the appropriate place to ask about it.

So I have a script using schedule to check something periodically. It roughly looks like this (pseudo code):

import time 
import sys 

import schedule

def check(url):
    # print current time 
    print(f"\nCurrent Time = {now_to_str()}")
    # fetch and check `url` to see if it meets some conditions...
    if not condition_met:
        return
    # do something...
    return schedule.CancelJob
if __name__ == '__main__':

    schedule.every(26).seconds.do(check, url=sys.argv[1])
    # run it immediately once
    schedule.run_all()

    while schedule.jobs:
        schedule.run_pending()
        time.sleep(.1)

I will typically run three or four of such scripts in parallel (so different Python processes) on my machine. Because I do it manually, they usually start at different time, a few seconds apart.

What I've noticed that if I let the scripts run for a while, say 1 hour or 2, I find that the current time they print at the start of each task eventually became the same.

For example, I parsed the logs from three instances and print the difference between three logs:

log1.txt: find 244 times
log2.txt: find 244 times
log3.txt: find 244 times
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃        log1         ┃        log2         ┃        log3         ┃ diff 1 to 2 ┃ diff 2 to 3 ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ 2024-02-22 21:29:35 │ 2024-02-22 21:29:41 │ 2024-02-22 21:29:45 │     6.0     │     4.0     │
│ 2024-02-22 21:30:02 │ 2024-02-22 21:30:08 │ 2024-02-22 21:30:12 │     6.0     │     4.0     │
│ 2024-02-22 21:30:29 │ 2024-02-22 21:30:35 │ 2024-02-22 21:30:39 │     6.0     │     4.0     │
│ 2024-02-22 21:30:56 │ 2024-02-22 21:31:02 │ 2024-02-22 21:31:06 │     6.0     │     4.0     │
│ 2024-02-22 21:31:23 │ 2024-02-22 21:31:29 │ 2024-02-22 21:31:33 │     6.0     │     4.0     │
│ 2024-02-22 21:31:50 │ 2024-02-22 21:31:56 │ 2024-02-22 21:32:00 │     6.0     │     4.0     │
│ 2024-02-22 21:32:17 │ 2024-02-22 21:32:23 │ 2024-02-22 21:32:27 │     6.0     │     4.0     │
│ 2024-02-22 21:32:44 │ 2024-02-22 21:32:50 │ 2024-02-22 21:32:54 │     6.0     │     4.0     │
│ 2024-02-22 21:33:11 │ 2024-02-22 21:33:17 │ 2024-02-22 21:33:21 │     6.0     │     4.0     │
│ 2024-02-22 21:33:38 │ 2024-02-22 21:33:44 │ 2024-02-22 21:33:48 │     6.0     │     4.0     │
│ 2024-02-22 21:34:05 │ 2024-02-22 21:34:11 │ 2024-02-22 21:34:15 │     6.0     │     4.0     │
│ 2024-02-22 21:34:33 │ 2024-02-22 21:34:38 │ 2024-02-22 21:34:42 │     5.0     │     4.0     │
│ 2024-02-22 21:35:00 │ 2024-02-22 21:35:05 │ 2024-02-22 21:35:09 │     5.0     │     4.0     │
│ 2024-02-22 21:35:27 │ 2024-02-22 21:35:32 │ 2024-02-22 21:35:36 │     5.0     │     4.0     │
│ 2024-02-22 21:35:54 │ 2024-02-22 21:35:59 │ 2024-02-22 21:36:03 │     5.0     │     4.0     │
│ 2024-02-22 21:36:21 │ 2024-02-22 21:36:26 │ 2024-02-22 21:36:30 │     5.0     │     4.0     │
│ 2024-02-22 21:36:48 │ 2024-02-22 21:36:53 │ 2024-02-22 21:36:57 │     5.0     │     4.0     │
│ 2024-02-22 21:37:15 │ 2024-02-22 21:37:20 │ 2024-02-22 21:37:24 │     5.0     │     4.0     │
│ 2024-02-22 21:37:42 │ 2024-02-22 21:37:47 │ 2024-02-22 21:37:51 │     5.0     │     4.0     │
│ 2024-02-22 21:38:10 │ 2024-02-22 21:38:14 │ 2024-02-22 21:38:18 │     4.0     │     4.0     │
│ 2024-02-22 21:38:37 │ 2024-02-22 21:38:41 │ 2024-02-22 21:38:45 │     4.0     │     4.0     │
│ 2024-02-22 21:39:04 │ 2024-02-22 21:39:08 │ 2024-02-22 21:39:12 │     4.0     │     4.0     │
│ 2024-02-22 21:39:31 │ 2024-02-22 21:39:35 │ 2024-02-22 21:39:39 │     4.0     │     4.0     │
│ 2024-02-22 21:39:58 │ 2024-02-22 21:40:02 │ 2024-02-22 21:40:06 │     4.0     │     4.0     │
│ 2024-02-22 21:40:25 │ 2024-02-22 21:40:29 │ 2024-02-22 21:40:33 │     4.0     │     4.0     │
│ 2024-02-22 21:40:52 │ 2024-02-22 21:40:56 │ 2024-02-22 21:41:00 │     4.0     │     4.0     │
│ 2024-02-22 21:41:20 │ 2024-02-22 21:41:23 │ 2024-02-22 21:41:28 │     3.0     │     5.0     │
│ 2024-02-22 21:41:47 │ 2024-02-22 21:41:50 │ 2024-02-22 21:41:55 │     3.0     │     5.0     │
│ 2024-02-22 21:42:14 │ 2024-02-22 21:42:17 │ 2024-02-22 21:42:22 │     3.0     │     5.0     │
│ 2024-02-22 21:42:41 │ 2024-02-22 21:42:44 │ 2024-02-22 21:42:49 │     3.0     │     5.0     │
│ 2024-02-22 21:43:08 │ 2024-02-22 21:43:11 │ 2024-02-22 21:43:16 │     3.0     │     5.0     │
│ 2024-02-22 21:43:35 │ 2024-02-22 21:43:38 │ 2024-02-22 21:43:43 │     3.0     │     5.0     │
│ 2024-02-22 21:44:02 │ 2024-02-22 21:44:05 │ 2024-02-22 21:44:10 │     3.0     │     5.0     │
│ 2024-02-22 21:44:30 │ 2024-02-22 21:44:32 │ 2024-02-22 21:44:37 │     2.0     │     5.0     │
│ 2024-02-22 21:44:57 │ 2024-02-22 21:45:00 │ 2024-02-22 21:45:04 │     3.0     │     4.0     │
│ 2024-02-22 21:45:24 │ 2024-02-22 21:45:27 │ 2024-02-22 21:45:31 │     3.0     │     4.0     │
│ 2024-02-22 21:45:51 │ 2024-02-22 21:45:54 │ 2024-02-22 21:45:58 │     3.0     │     4.0     │
│ 2024-02-22 21:46:18 │ 2024-02-22 21:46:21 │ 2024-02-22 21:46:25 │     3.0     │     4.0     │
│ 2024-02-22 21:46:45 │ 2024-02-22 21:46:48 │ 2024-02-22 21:46:53 │     3.0     │     5.0     │
│ 2024-02-22 21:47:12 │ 2024-02-22 21:47:15 │ 2024-02-22 21:47:19 │     3.0     │     4.0     │
│ 2024-02-22 21:47:39 │ 2024-02-22 21:47:42 │ 2024-02-22 21:47:46 │     3.0     │     4.0     │
│ 2024-02-22 21:48:06 │ 2024-02-22 21:48:09 │ 2024-02-22 21:48:13 │     3.0     │     4.0     │
│ 2024-02-22 21:48:33 │ 2024-02-22 21:48:36 │ 2024-02-22 21:48:40 │     3.0     │     4.0     │
│ 2024-02-22 21:49:00 │ 2024-02-22 21:49:03 │ 2024-02-22 21:49:07 │     3.0     │     4.0     │
│ 2024-02-22 21:49:27 │ 2024-02-22 21:49:30 │ 2024-02-22 21:49:34 │     3.0     │     4.0     │
│ 2024-02-22 21:49:54 │ 2024-02-22 21:49:57 │ 2024-02-22 21:50:01 │     3.0     │     4.0     │
│ 2024-02-22 21:50:22 │ 2024-02-22 21:50:24 │ 2024-02-22 21:50:28 │     2.0     │     4.0     │
│ 2024-02-22 21:50:49 │ 2024-02-22 21:50:51 │ 2024-02-22 21:50:55 │     2.0     │     4.0     │
│ 2024-02-22 21:51:16 │ 2024-02-22 21:51:18 │ 2024-02-22 21:51:22 │     2.0     │     4.0     │
│ 2024-02-22 21:51:43 │ 2024-02-22 21:51:45 │ 2024-02-22 21:51:49 │     2.0     │     4.0     │
│ 2024-02-22 21:52:10 │ 2024-02-22 21:52:12 │ 2024-02-22 21:52:16 │     2.0     │     4.0     │
│ 2024-02-22 21:52:37 │ 2024-02-22 21:52:39 │ 2024-02-22 21:52:43 │     2.0     │     4.0     │
│ 2024-02-22 21:53:04 │ 2024-02-22 21:53:06 │ 2024-02-22 21:53:10 │     2.0     │     4.0     │
│ 2024-02-22 21:53:31 │ 2024-02-22 21:53:33 │ 2024-02-22 21:53:37 │     2.0     │     4.0     │
│ 2024-02-22 21:53:58 │ 2024-02-22 21:54:00 │ 2024-02-22 21:54:04 │     2.0     │     4.0     │
│ 2024-02-22 21:54:26 │ 2024-02-22 21:54:27 │ 2024-02-22 21:54:31 │     1.0     │     4.0     │
│ 2024-02-22 21:54:53 │ 2024-02-22 21:54:54 │ 2024-02-22 21:54:58 │     1.0     │     4.0     │
│ 2024-02-22 21:55:20 │ 2024-02-22 21:55:21 │ 2024-02-22 21:55:26 │     1.0     │     5.0     │
│ 2024-02-22 21:55:47 │ 2024-02-22 21:55:48 │ 2024-02-22 21:55:53 │     1.0     │     5.0     │
│ 2024-02-22 21:56:14 │ 2024-02-22 21:56:15 │ 2024-02-22 21:56:20 │     1.0     │     5.0     │
│ 2024-02-22 21:56:41 │ 2024-02-22 21:56:42 │ 2024-02-22 21:56:47 │     1.0     │     5.0     │
│ 2024-02-22 21:57:08 │ 2024-02-22 21:57:09 │ 2024-02-22 21:57:14 │     1.0     │     5.0     │
│ 2024-02-22 21:57:35 │ 2024-02-22 21:57:36 │ 2024-02-22 21:57:41 │     1.0     │     5.0     │
│ 2024-02-22 21:58:02 │ 2024-02-22 21:58:03 │ 2024-02-22 21:58:08 │     1.0     │     5.0     │
│ 2024-02-22 21:58:29 │ 2024-02-22 21:58:30 │ 2024-02-22 21:58:35 │     1.0     │     5.0     │
│ 2024-02-22 21:58:56 │ 2024-02-22 21:58:57 │ 2024-02-22 21:59:02 │     1.0     │     5.0     │
│ 2024-02-22 21:59:24 │ 2024-02-22 21:59:24 │ 2024-02-22 21:59:29 │     0.0     │     5.0     │
│ 2024-02-22 21:59:51 │ 2024-02-22 21:59:51 │ 2024-02-22 21:59:56 │     0.0     │     5.0     │
│ 2024-02-22 22:00:18 │ 2024-02-22 22:00:18 │ 2024-02-22 22:00:23 │     0.0     │     5.0     │
│ 2024-02-22 22:00:45 │ 2024-02-22 22:00:45 │ 2024-02-22 22:00:50 │     0.0     │     5.0     │
│ 2024-02-22 22:01:12 │ 2024-02-22 22:01:12 │ 2024-02-22 22:01:17 │     0.0     │     5.0     │
│ 2024-02-22 22:01:40 │ 2024-02-22 22:01:40 │ 2024-02-22 22:01:44 │     0.0     │     4.0     │
│ 2024-02-22 22:02:07 │ 2024-02-22 22:02:07 │ 2024-02-22 22:02:11 │     0.0     │     4.0     │
│ 2024-02-22 22:02:34 │ 2024-02-22 22:02:34 │ 2024-02-22 22:02:38 │     0.0     │     4.0     │
│ 2024-02-22 22:03:01 │ 2024-02-22 22:03:01 │ 2024-02-22 22:03:05 │     0.0     │     4.0     │
│ 2024-02-22 22:03:28 │ 2024-02-22 22:03:28 │ 2024-02-22 22:03:32 │     0.0     │     4.0     │
│ 2024-02-22 22:03:55 │ 2024-02-22 22:03:55 │ 2024-02-22 22:04:00 │     0.0     │     5.0     │
│ 2024-02-22 22:04:22 │ 2024-02-22 22:04:22 │ 2024-02-22 22:04:27 │     0.0     │     5.0     │
│ 2024-02-22 22:04:49 │ 2024-02-22 22:04:49 │ 2024-02-22 22:04:54 │     0.0     │     5.0     │
│ 2024-02-22 22:05:16 │ 2024-02-22 22:05:16 │ 2024-02-22 22:05:21 │     0.0     │     5.0     │
│ 2024-02-22 22:05:43 │ 2024-02-22 22:05:43 │ 2024-02-22 22:05:48 │     0.0     │     5.0     │
│ 2024-02-22 22:06:10 │ 2024-02-22 22:06:10 │ 2024-02-22 22:06:15 │     0.0     │     5.0     │
│ 2024-02-22 22:06:38 │ 2024-02-22 22:06:38 │ 2024-02-22 22:06:41 │     0.0     │     3.0     │
│ 2024-02-22 22:07:05 │ 2024-02-22 22:07:05 │ 2024-02-22 22:07:08 │     0.0     │     3.0     │
│ 2024-02-22 22:07:32 │ 2024-02-22 22:07:32 │ 2024-02-22 22:07:35 │     0.0     │     3.0     │
│ 2024-02-22 22:07:59 │ 2024-02-22 22:07:59 │ 2024-02-22 22:08:02 │     0.0     │     3.0     │
│ 2024-02-22 22:08:26 │ 2024-02-22 22:08:26 │ 2024-02-22 22:08:29 │     0.0     │     3.0     │
│ 2024-02-22 22:08:54 │ 2024-02-22 22:08:53 │ 2024-02-22 22:08:56 │    -1.0     │     3.0     │
│ 2024-02-22 22:09:21 │ 2024-02-22 22:09:20 │ 2024-02-22 22:09:23 │    -1.0     │     3.0     │
│ 2024-02-22 22:09:48 │ 2024-02-22 22:09:48 │ 2024-02-22 22:09:50 │     0.0     │     2.0     │
│ 2024-02-22 22:10:15 │ 2024-02-22 22:10:15 │ 2024-02-22 22:10:17 │     0.0     │     2.0     │
│ 2024-02-22 22:10:42 │ 2024-02-22 22:10:42 │ 2024-02-22 22:10:44 │     0.0     │     2.0     │
│ 2024-02-22 22:11:09 │ 2024-02-22 22:11:09 │ 2024-02-22 22:11:11 │     0.0     │     2.0     │
│ 2024-02-22 22:11:36 │ 2024-02-22 22:11:36 │ 2024-02-22 22:11:38 │     0.0     │     2.0     │
│ 2024-02-22 22:12:03 │ 2024-02-22 22:12:03 │ 2024-02-22 22:12:05 │     0.0     │     2.0     │
│ 2024-02-22 22:12:30 │ 2024-02-22 22:12:30 │ 2024-02-22 22:12:32 │     0.0     │     2.0     │
│ 2024-02-22 22:12:58 │ 2024-02-22 22:12:58 │ 2024-02-22 22:12:59 │     0.0     │     1.0     │
│ 2024-02-22 22:13:25 │ 2024-02-22 22:13:25 │ 2024-02-22 22:13:26 │     0.0     │     1.0     │
│ 2024-02-22 22:13:52 │ 2024-02-22 22:13:52 │ 2024-02-22 22:13:53 │     0.0     │     1.0     │
│ 2024-02-22 22:14:19 │ 2024-02-22 22:14:19 │ 2024-02-22 22:14:20 │     0.0     │     1.0     │
│ 2024-02-22 22:14:46 │ 2024-02-22 22:14:46 │ 2024-02-22 22:14:47 │     0.0     │     1.0     │
│ 2024-02-22 22:15:13 │ 2024-02-22 22:15:13 │ 2024-02-22 22:15:14 │     0.0     │     1.0     │
│ 2024-02-22 22:15:40 │ 2024-02-22 22:15:40 │ 2024-02-22 22:15:41 │     0.0     │     1.0     │
│ 2024-02-22 22:16:08 │ 2024-02-22 22:16:07 │ 2024-02-22 22:16:08 │    -1.0     │     1.0     │
│ 2024-02-22 22:16:35 │ 2024-02-22 22:16:35 │ 2024-02-22 22:16:35 │     0.0     │     0.0     │
│ 2024-02-22 22:17:02 │ 2024-02-22 22:17:02 │ 2024-02-22 22:17:02 │     0.0     │     0.0     │
│ 2024-02-22 22:17:29 │ 2024-02-22 22:17:29 │ 2024-02-22 22:17:29 │     0.0     │     0.0     │
│ 2024-02-22 22:17:56 │ 2024-02-22 22:17:56 │ 2024-02-22 22:17:56 │     0.0     │     0.0     │
│ 2024-02-22 22:18:23 │ 2024-02-22 22:18:23 │ 2024-02-22 22:18:23 │     0.0     │     0.0     │
│ 2024-02-22 22:18:50 │ 2024-02-22 22:18:50 │ 2024-02-22 22:18:50 │     0.0     │     0.0     │
│ 2024-02-22 22:19:18 │ 2024-02-22 22:19:17 │ 2024-02-22 22:19:17 │    -1.0     │     0.0     │
│ 2024-02-22 22:19:45 │ 2024-02-22 22:19:45 │ 2024-02-22 22:19:45 │     0.0     │     0.0     │
│ 2024-02-22 22:20:12 │ 2024-02-22 22:20:12 │ 2024-02-22 22:20:12 │     0.0     │     0.0     │
│ 2024-02-22 22:20:39 │ 2024-02-22 22:20:39 │ 2024-02-22 22:20:39 │     0.0     │     0.0     │
│ 2024-02-22 22:21:06 │ 2024-02-22 22:21:06 │ 2024-02-22 22:21:06 │     0.0     │     0.0     │
│ 2024-02-22 22:21:33 │ 2024-02-22 22:21:33 │ 2024-02-22 22:21:33 │     0.0     │     0.0     │
│ 2024-02-22 22:22:00 │ 2024-02-22 22:22:00 │ 2024-02-22 22:22:00 │     0.0     │     0.0     │
│ 2024-02-22 22:22:27 │ 2024-02-22 22:22:28 │ 2024-02-22 22:22:27 │     1.0     │    -1.0     │
│ 2024-02-22 22:22:54 │ 2024-02-22 22:22:55 │ 2024-02-22 22:22:54 │     1.0     │    -1.0     │
│ 2024-02-22 22:23:21 │ 2024-02-22 22:23:22 │ 2024-02-22 22:23:21 │     1.0     │    -1.0     │
│ 2024-02-22 22:23:48 │ 2024-02-22 22:23:49 │ 2024-02-22 22:23:48 │     1.0     │    -1.0     │
│ 2024-02-22 22:24:15 │ 2024-02-22 22:24:16 │ 2024-02-22 22:24:15 │     1.0     │    -1.0     │
│ 2024-02-22 22:24:43 │ 2024-02-22 22:24:43 │ 2024-02-22 22:24:43 │     0.0     │     0.0     │
│ 2024-02-22 22:25:10 │ 2024-02-22 22:25:10 │ 2024-02-22 22:25:10 │     0.0     │     0.0     │
│ 2024-02-22 22:25:37 │ 2024-02-22 22:25:37 │ 2024-02-22 22:25:37 │     0.0     │     0.0     │
│ 2024-02-22 22:26:04 │ 2024-02-22 22:26:04 │ 2024-02-22 22:26:04 │     0.0     │     0.0     │
..............
│ 2024-02-22 23:17:34 │ 2024-02-22 23:17:34 │ 2024-02-22 23:17:34 │     0.0     │     0.0     │
│ 2024-02-22 23:18:01 │ 2024-02-22 23:18:01 │ 2024-02-22 23:18:01 │     0.0     │     0.0     │
│ 2024-02-22 23:18:28 │ 2024-02-22 23:18:28 │ 2024-02-22 23:18:28 │     0.0     │     0.0     │
│ 2024-02-22 23:18:55 │ 2024-02-22 23:18:55 │ 2024-02-22 23:18:55 │     0.0     │     0.0     │
│ 2024-02-22 23:19:22 │ 2024-02-22 23:19:22 │ 2024-02-22 23:19:22 │     0.0     │     0.0     │
└─────────────────────┴─────────────────────┴─────────────────────┴─────────────┴─────────────┘

As you can see, at the beginning 2 is 6 seconds behind 1, and 3 is 4 seconds behind 2. But after merely 1 hours, they become in sync. And they STAY in sync for the rest of the time.

I fully understand schedule isn't meant to have exact timing, so I never expect the interval to be consistent. But I don't understand why would three totally unrelated python processes would eventually converge together.

In the above script I only have timestamp in seconds in my log. But in fact, they are more in sync than that!

In the actual full script, part of the scheduled task is to call a external program (subprocess.run) when the condition is met, which creates a folder called folder_{timestamp_in_ms}. And it has happens to me once, that two different running scripts create the folder at exactly same millisecond due to this convergence.

I can't for the life of me figure it out. I hope someone can provide some insights. Thanks in advance.