Is GitHub raw storage reliable?

[scripting]

I've been building apps on the assumption that if you store something on GitHub, when you get the raw version of that thing, it'll be current. I was getting inconsistent results, so I decided to test that assumption, and it seems that it is not reliable storage.

If you read an object immediately after writing it, you will not always get back what you wrote. It's not just a matter of waiting a second or two, sometimes it's wrong for up to 10 seconds (the delay I programmed into the test).

Here's the Node app I'm running to test this. It wakes up every minute and saves a file to GitHub with the current local time string. If I reload the non-raw version of the page, it usually seems to have the current value. But if I reload the raw version of the page, it often does not have the correct value.

If you see a problem in this app, please let me know. I had big plans for using GitHub for user storage. It'll make a huge difference in interop, common ground in the fast growing Tools for Thought space.

Here are the results of the app for the first 30 minutes.

everyMinute: 1:58:13 PM, status == 200, bad everyMinute: 1:59:00 PM, status == 200, bad everyMinute: 2:00:00 PM, status == 200, bad everyMinute: 2:01:00 PM, status == 200, good everyMinute: 2:02:00 PM, status == 200, bad everyMinute: 2:03:00 PM, status == 200, bad everyMinute: 2:04:00 PM, status == 200, bad everyMinute: 2:05:00 PM, status == 200, bad everyMinute: 2:06:00 PM, status == 200, bad everyMinute: 2:07:00 PM, status == 200, good everyMinute: 2:08:00 PM, status == 200, bad everyMinute: 2:09:00 PM, status == 200, bad everyMinute: 2:10:00 PM, status == 200, bad everyMinute: 2:11:00 PM, status == 200, bad everyMinute: 2:12:00 PM, status == 200, bad everyMinute: 2:13:00 PM, status == 200, good everyMinute: 2:14:00 PM, status == 200, bad everyMinute: 2:15:00 PM, status == 200, bad everyMinute: 2:16:00 PM, status == 200, bad everyMinute: 2:17:00 PM, status == 200, bad everyMinute: 2:18:00 PM, status == 200, bad everyMinute: 2:19:00 PM, status == 200, good everyMinute: 2:20:00 PM, status == 200, bad everyMinute: 2:21:00 PM, status == 200, bad everyMinute: 2:22:00 PM, status == 200, bad everyMinute: 2:23:00 PM, status == 200, bad everyMinute: 2:24:00 PM, status == 200, bad everyMinute: 2:25:00 PM, status == 200, good everyMinute: 2:26:00 PM, status == 200, bad everyMinute: 2:27:00 PM, status == 200, bad everyMinute: 2:28:00 PM, status == 200, bad

You can see that more often than not the raw value is incorrect after ten seconds.

I'm leaving the app running for a while, so if you want to verify this, you can.

I'll post a note in the GitHub thread when I turn the app off.

[scripting]

I turned off the app. There's obviously a cache in there.

everyMinute: 1:58:13 PM, status == 200, bad everyMinute: 1:59:00 PM, status == 200, bad everyMinute: 2:00:00 PM, status == 200, bad everyMinute: 2:01:00 PM, status == 200, good everyMinute: 2:02:00 PM, status == 200, bad everyMinute: 2:03:00 PM, status == 200, bad everyMinute: 2:04:00 PM, status == 200, bad everyMinute: 2:05:00 PM, status == 200, bad everyMinute: 2:06:00 PM, status == 200, bad everyMinute: 2:07:00 PM, status == 200, good everyMinute: 2:08:00 PM, status == 200, bad everyMinute: 2:09:00 PM, status == 200, bad everyMinute: 2:10:00 PM, status == 200, bad everyMinute: 2:11:00 PM, status == 200, bad everyMinute: 2:12:00 PM, status == 200, bad everyMinute: 2:13:00 PM, status == 200, good everyMinute: 2:14:00 PM, status == 200, bad everyMinute: 2:15:00 PM, status == 200, bad everyMinute: 2:16:00 PM, status == 200, bad everyMinute: 2:17:00 PM, status == 200, bad everyMinute: 2:18:00 PM, status == 200, bad everyMinute: 2:19:00 PM, status == 200, good everyMinute: 2:20:00 PM, status == 200, bad everyMinute: 2:21:00 PM, status == 200, bad everyMinute: 2:22:00 PM, status == 200, bad everyMinute: 2:23:00 PM, status == 200, bad everyMinute: 2:24:00 PM, status == 200, bad everyMinute: 2:25:00 PM, status == 200, good everyMinute: 2:26:00 PM, status == 200, bad everyMinute: 2:27:00 PM, status == 200, bad everyMinute: 2:28:00 PM, status == 200, bad everyMinute: 2:29:00 PM, status == 200, bad everyMinute: 2:30:00 PM, status == 200, good everyMinute: 2:31:00 PM, status == 200, bad everyMinute: 2:32:00 PM, status == 200, bad everyMinute: 2:33:00 PM, status == 200, bad everyMinute: 2:34:00 PM, status == 200, bad everyMinute: 2:35:00 PM, status == 200, bad everyMinute: 2:36:00 PM, status == 200, good everyMinute: 2:37:00 PM, status == 200, bad everyMinute: 2:38:00 PM, status == 200, bad everyMinute: 2:39:00 PM, status == 200, bad everyMinute: 2:40:00 PM, status == 200, bad everyMinute: 2:41:00 PM, status == 200, good everyMinute: 2:42:00 PM, status == 200, bad everyMinute: 2:43:00 PM, status == 200, bad everyMinute: 2:44:00 PM, status == 200, bad everyMinute: 2:45:00 PM, status == 200, bad everyMinute: 2:46:00 PM, status == 200, bad everyMinute: 2:47:00 PM, status == 200, bad everyMinute: 2:48:00 PM, status == 200, bad everyMinute: 2:49:00 PM, status == 200, bad everyMinute: 2:50:00 PM, status == 200, bad everyMinute: 2:51:00 PM, status == 200, bad everyMinute: 2:52:00 PM, status == 200, bad everyMinute: 2:53:00 PM, status == 200, bad everyMinute: 2:54:00 PM, status == 200, bad everyMinute: 2:55:00 PM, status == 200, bad everyMinute: 2:56:00 PM, status == 200, bad everyMinute: 2:57:00 PM, status == 200, bad everyMinute: 2:58:00 PM, status == 200, good everyMinute: 2:59:01 PM, status == 200, bad everyMinute: 3:00:00 PM, status == 200, bad everyMinute: 3:01:00 PM, status == 200, bad everyMinute: 3:02:00 PM, status == 200, bad everyMinute: 3:03:00 PM, status == 200, bad everyMinute: 3:04:00 PM, status == 200, good everyMinute: 3:05:00 PM, status == 200, bad everyMinute: 3:06:00 PM, status == 200, bad everyMinute: 3:07:00 PM, status == 200, bad everyMinute: 3:08:00 PM, status == 200, bad everyMinute: 3:09:00 PM, status == 200, bad everyMinute: 3:10:00 PM, status == 200, good everyMinute: 3:11:00 PM, status == 200, bad everyMinute: 3:12:00 PM, status == 200, bad everyMinute: 3:13:00 PM, status == 200, bad everyMinute: 3:14:00 PM, status == 200, bad everyMinute: 3:15:00 PM, status == 200, good everyMinute: 3:16:00 PM, status == 200, bad everyMinute: 3:17:00 PM, status == 200, bad everyMinute: 3:18:00 PM, status == 200, bad everyMinute: 3:19:00 PM, status == 200, bad everyMinute: 3:20:00 PM, status == 200, bad everyMinute: 3:21:00 PM, status == 200, good everyMinute: 3:22:00 PM, status == 200, bad everyMinute: 3:23:00 PM, status == 200, bad everyMinute: 3:24:00 PM, status == 200, bad everyMinute: 3:25:00 PM, status == 200, bad everyMinute: 3:26:00 PM, status == 200, bad everyMinute: 3:27:00 PM, status == 200, good everyMinute: 3:28:00 PM, status == 200, bad everyMinute: 3:29:00 PM, status == 200, bad everyMinute: 3:30:00 PM, status == 200, bad everyMinute: 3:31:00 PM, status == 200, bad everyMinute: 3:32:00 PM, status == 200, good everyMinute: 3:33:00 PM, status == 200, bad everyMinute: 3:34:00 PM, status == 200, bad everyMinute: 3:35:00 PM, status == 200, bad everyMinute: 3:36:00 PM, status == 200, bad everyMinute: 3:37:00 PM, status == 200, bad everyMinute: 3:38:00 PM, status == 200, good everyMinute: 3:39:00 PM, status == 200, bad everyMinute: 3:40:00 PM, status == 200, bad everyMinute: 3:41:00 PM, status == 200, bad everyMinute: 3:42:00 PM, status == 200, bad everyMinute: 3:43:00 PM, status == 200, good everyMinute: 3:44:00 PM, status == 200, bad everyMinute: 3:45:00 PM, status == 200, bad everyMinute: 3:46:00 PM, status == 200, bad everyMinute: 3:47:00 PM, status == 200, bad everyMinute: 3:48:00 PM, status == 200, bad everyMinute: 3:49:00 PM, status == 200, bad everyMinute: 3:50:00 PM, status == 200, bad everyMinute: 3:51:00 PM, status == 200, bad everyMinute: 3:52:00 PM, status == 200, bad everyMinute: 3:53:00 PM, status == 200, bad everyMinute: 3:54:00 PM, status == 200, bad everyMinute: 3:55:00 PM, status == 200, bad everyMinute: 3:56:00 PM, status == 200, bad everyMinute: 3:57:00 PM, status == 200, bad everyMinute: 3:58:00 PM, status == 200, bad everyMinute: 3:59:00 PM, status == 200, bad everyMinute: 4:00:00 PM, status == 200, good everyMinute: 4:01:00 PM, status == 200, bad everyMinute: 4:02:00 PM, status == 200, bad everyMinute: 4:03:00 PM, status == 200, bad everyMinute: 4:04:00 PM, status == 200, bad everyMinute: 4:05:00 PM, status == 200, bad everyMinute: 4:06:00 PM, status == 200, good everyMinute: 4:07:00 PM, status == 200, bad everyMinute: 4:08:00 PM, status == 200, bad everyMinute: 4:09:00 PM, status == 200, bad everyMinute: 4:10:00 PM, status == 200, bad everyMinute: 4:11:00 PM, status == 200, bad everyMinute: 4:12:00 PM, status == 200, good everyMinute: 4:13:00 PM, status == 200, bad everyMinute: 4:14:00 PM, status == 200, bad everyMinute: 4:15:00 PM, status == 200, bad everyMinute: 4:16:00 PM, status == 200, bad everyMinute: 4:17:00 PM, status == 200, bad everyMinute: 4:18:00 PM, status == 200, good everyMinute: 4:19:00 PM, status == 200, bad everyMinute: 4:20:00 PM, status == 200, bad everyMinute: 4:21:00 PM, status == 200, bad everyMinute: 4:22:00 PM, status == 200, bad everyMinute: 4:23:00 PM, status == 200, bad everyMinute: 4:24:00 PM, status == 200, good everyMinute: 4:25:00 PM, status == 200, bad everyMinute: 4:26:00 PM, status == 200, bad everyMinute: 4:27:00 PM, status == 200, bad everyMinute: 4:28:00 PM, status == 200, bad everyMinute: 4:29:00 PM, status == 200, bad everyMinute: 4:30:00 PM, status == 200, good everyMinute: 4:31:00 PM, status == 200, bad everyMinute: 4:32:00 PM, status == 200, bad everyMinute: 4:33:00 PM, status == 200, bad everyMinute: 4:34:00 PM, status == 200, bad everyMinute: 4:35:00 PM, status == 200, bad everyMinute: 4:36:00 PM, status == 200, good everyMinute: 4:37:00 PM, status == 200, bad everyMinute: 4:38:00 PM, status == 200, bad everyMinute: 4:39:00 PM, status == 200, bad everyMinute: 4:40:00 PM, status == 200, bad everyMinute: 4:41:00 PM, status == 200, bad everyMinute: 4:42:00 PM, status == 200, good everyMinute: 4:43:00 PM, status == 200, bad everyMinute: 4:44:00 PM, status == 200, bad everyMinute: 4:45:00 PM, status == 200, bad everyMinute: 4:46:01 PM, status == 200, bad everyMinute: 4:47:00 PM, status == 200, bad everyMinute: 4:48:00 PM, status == 200, bad everyMinute: 4:49:00 PM, status == 200, bad everyMinute: 4:50:00 PM, status == 200, bad everyMinute: 4:51:00 PM, status == 200, bad everyMinute: 4:52:00 PM, status == 200, bad everyMinute: 4:53:00 PM, status == 200, bad everyMinute: 4:54:00 PM, status == 200, bad everyMinute: 4:55:00 PM, status == 200, bad everyMinute: 4:56:00 PM, status == 200, bad everyMinute: 4:57:00 PM, status == 200, bad everyMinute: 4:58:00 PM, status == 200, bad everyMinute: 4:59:00 PM, status == 200, bad everyMinute: 5:00:00 PM, status == 200, good everyMinute: 5:01:00 PM, status == 200, bad everyMinute: 5:02:00 PM, status == 200, bad everyMinute: 5:03:00 PM, status == 200, bad everyMinute: 5:04:00 PM, status == 200, bad everyMinute: 5:05:00 PM, status == 200, bad everyMinute: 5:06:00 PM, status == 200, good everyMinute: 5:07:00 PM, status == 200, bad everyMinute: 5:08:00 PM, status == 200, bad everyMinute: 5:09:00 PM, status == 200, bad everyMinute: 5:10:00 PM, status == 200, bad everyMinute: 5:11:00 PM, status == 200, bad everyMinute: 5:12:00 PM, status == 200, good everyMinute: 5:13:00 PM, status == 200, bad everyMinute: 5:14:00 PM, status == 200, bad everyMinute: 5:15:00 PM, status == 200, bad everyMinute: 5:16:00 PM, status == 200, bad everyMinute: 5:17:00 PM, status == 200, bad everyMinute: 5:18:00 PM, status == 200, good everyMinute: 5:19:01 PM, status == 200, bad everyMinute: 5:20:00 PM, status == 200, bad everyMinute: 5:21:00 PM, status == 200, bad everyMinute: 5:22:00 PM, status == 200, bad everyMinute: 5:23:00 PM, status == 200, bad everyMinute: 5:24:00 PM, status == 200, good everyMinute: 5:25:00 PM, status == 200, bad everyMinute: 5:26:00 PM, status == 200, bad everyMinute: 5:27:00 PM, status == 200, bad everyMinute: 5:28:00 PM, status == 200, bad everyMinute: 5:29:00 PM, status == 200, bad everyMinute: 5:30:00 PM, status == 200, good everyMinute: 5:31:00 PM, status == 200, bad everyMinute: 5:32:00 PM, status == 200, bad everyMinute: 5:33:00 PM, status == 200, bad everyMinute: 5:34:00 PM, status == 200, bad everyMinute: 5:35:00 PM, status == 200, good everyMinute: 5:36:00 PM, status == 200, bad everyMinute: 5:37:00 PM, status == 200, bad everyMinute: 5:38:00 PM, status == 200, bad everyMinute: 5:39:00 PM, status == 200, bad everyMinute: 5:40:00 PM, status == 200, bad everyMinute: 5:41:00 PM, status == 200, good everyMinute: 5:42:00 PM, status == 200, bad everyMinute: 5:43:00 PM, status == 200, bad everyMinute: 5:44:00 PM, status == 200, bad everyMinute: 5:45:00 PM, status == 200, bad everyMinute: 5:46:00 PM, status == 200, bad everyMinute: 5:47:00 PM, status == 200, good everyMinute: 5:48:00 PM, status == 200, bad everyMinute: 5:49:00 PM, status == 200, bad everyMinute: 5:50:00 PM, status == 200, bad everyMinute: 5:51:00 PM, status == 200, bad everyMinute: 5:52:00 PM, status == 200, bad everyMinute: 5:53:00 PM, status == 200, bad everyMinute: 5:54:00 PM, status == 200, bad everyMinute: 5:55:00 PM, status == 200, bad everyMinute: 5:56:00 PM, status == 200, bad everyMinute: 5:57:00 PM, status == 200, bad everyMinute: 5:58:00 PM, status == 200, bad everyMinute: 5:59:00 PM, status == 200, bad everyMinute: 6:00:00 PM, status == 200, bad everyMinute: 6:01:00 PM, status == 200, bad everyMinute: 6:02:00 PM, status == 200, bad everyMinute: 6:03:00 PM, status == 200, good everyMinute: 6:04:00 PM, status == 200, bad everyMinute: 6:05:00 PM, status == 200, bad everyMinute: 6:06:00 PM, status == 200, bad everyMinute: 6:07:00 PM, status == 200, bad everyMinute: 6:08:00 PM, status == 200, bad everyMinute: 6:09:00 PM, status == 200, good everyMinute: 6:10:00 PM, status == 200, bad everyMinute: 6:11:00 PM, status == 200, bad everyMinute: 6:12:00 PM, status == 200, bad everyMinute: 6:13:00 PM, status == 200, bad everyMinute: 6:14:00 PM, status == 200, bad everyMinute: 6:15:00 PM, status == 200, good everyMinute: 6:16:00 PM, status == 200, bad everyMinute: 6:17:00 PM, status == 200, bad everyMinute: 6:18:00 PM, status == 200, bad everyMinute: 6:19:00 PM, status == 200, bad everyMinute: 6:20:00 PM, status == 200, good everyMinute: 6:21:00 PM, status == 200, bad everyMinute: 6:22:00 PM, status == 200, bad everyMinute: 6:23:00 PM, status == 200, bad everyMinute: 6:24:00 PM, status == 200, bad everyMinute: 6:25:00 PM, status == 200, bad everyMinute: 6:26:00 PM, status == 200, good everyMinute: 6:27:00 PM, status == 200, bad everyMinute: 6:28:00 PM, status == 200, bad everyMinute: 6:29:00 PM, status == 200, bad everyMinute: 6:30:00 PM, status == 200, bad everyMinute: 6:31:00 PM, status == 200, bad everyMinute: 6:32:00 PM, status == 200, good everyMinute: 6:33:00 PM, status == 200, bad everyMinute: 6:34:00 PM, status == 200, bad everyMinute: 6:35:00 PM, status == 200, bad everyMinute: 6:36:00 PM, status == 200, bad everyMinute: 6:37:00 PM, status == 200, good everyMinute: 6:38:00 PM, status == 200, bad everyMinute: 6:39:00 PM, status == 200, bad everyMinute: 6:40:00 PM, status == 200, bad everyMinute: 6:41:00 PM, status == 200, bad everyMinute: 6:42:00 PM, status == 200, bad everyMinute: 6:43:00 PM, status == 200, good everyMinute: 6:44:00 PM, status == 200, bad everyMinute: 6:45:00 PM, status == 200, bad everyMinute: 6:46:00 PM, status == 200, bad everyMinute: 6:47:00 PM, status == 200, bad everyMinute: 6:48:00 PM, status == 200, bad everyMinute: 6:49:00 PM, status == 200, bad everyMinute: 6:50:00 PM, status == 200, bad everyMinute: 6:51:00 PM, status == 200, bad everyMinute: 6:52:00 PM, status == 200, bad everyMinute: 6:53:00 PM, status == 200, bad everyMinute: 6:54:00 PM, status == 200, bad everyMinute: 6:55:00 PM, status == 200, bad everyMinute: 6:56:00 PM, status == 200, bad everyMinute: 6:57:00 PM, status == 200, bad everyMinute: 6:58:00 PM, status == 200, bad everyMinute: 6:59:00 PM, status == 200, bad everyMinute: 7:00:00 PM, status == 200, bad everyMinute: 7:01:00 PM, status == 200, bad everyMinute: 7:02:00 PM, status == 200, bad everyMinute: 7:03:00 PM, status == 200, bad everyMinute: 7:04:00 PM, status == 200, bad everyMinute: 7:05:00 PM, status == 200, good everyMinute: 7:06:00 PM, status == 200, bad everyMinute: 7:07:00 PM, status == 200, bad everyMinute: 7:08:00 PM, status == 200, bad everyMinute: 7:09:00 PM, status == 200, bad everyMinute: 7:10:00 PM, status == 200, bad everyMinute: 7:11:00 PM, status == 200, good everyMinute: 7:12:00 PM, status == 200, bad everyMinute: 7:13:00 PM, status == 200, bad everyMinute: 7:14:00 PM, status == 200, bad everyMinute: 7:15:00 PM, status == 200, bad everyMinute: 7:16:00 PM, status == 200, bad everyMinute: 7:17:00 PM, status == 200, good everyMinute: 7:18:00 PM, status == 200, bad everyMinute: 7:19:00 PM, status == 200, bad everyMinute: 7:20:00 PM, status == 200, bad everyMinute: 7:21:00 PM, status == 200, bad everyMinute: 7:22:00 PM, status == 200, bad everyMinute: 7:23:00 PM, status == 200, good everyMinute: 7:24:00 PM, status == 200, bad everyMinute: 7:25:00 PM, status == 200, bad everyMinute: 7:26:00 PM, status == 200, bad everyMinute: 7:27:00 PM, status == 200, bad everyMinute: 7:28:00 PM, status == 200, bad everyMinute: 7:29:00 PM, status == 200, bad everyMinute: 7:30:00 PM, status == 200, bad everyMinute: 7:31:00 PM, status == 200, bad everyMinute: 7:32:00 PM, status == 200, bad everyMinute: 7:33:00 PM, status == 200, bad everyMinute: 7:34:00 PM, status == 200, good everyMinute: 7:35:00 PM, status == 200, bad everyMinute: 7:36:00 PM, status == 200, bad everyMinute: 7:37:00 PM, status == 200, bad everyMinute: 7:38:00 PM, status == 200, bad everyMinute: 7:39:00 PM, status == 200, bad everyMinute: 7:40:00 PM, status == 200, good everyMinute: 7:41:00 PM, status == 200, bad everyMinute: 7:42:00 PM, status == 200, bad everyMinute: 7:43:00 PM, status == 200, bad everyMinute: 7:44:00 PM, status == 200, bad everyMinute: 7:45:00 PM, status == 200, bad everyMinute: 7:46:00 PM, status == 200, good everyMinute: 7:47:00 PM, status == 200, bad everyMinute: 7:48:00 PM, status == 200, bad everyMinute: 7:49:00 PM, status == 200, bad everyMinute: 7:50:00 PM, status == 200, bad everyMinute: 7:51:00 PM, status == 200, bad everyMinute: 7:52:00 PM, status == 200, bad everyMinute: 7:53:00 PM, status == 200, bad everyMinute: 7:54:00 PM, status == 200, bad everyMinute: 7:55:00 PM, status == 200, bad everyMinute: 7:56:00 PM, status == 200, bad everyMinute: 7:57:00 PM, status == 200, bad everyMinute: 7:58:00 PM, status == 200, bad everyMinute: 7:59:00 PM, status == 200, bad everyMinute: 8:00:00 PM, status == 200, bad everyMinute: 8:01:00 PM, status == 200, bad everyMinute: 8:02:00 PM, status == 200, bad everyMinute: 8:03:00 PM, status == 200, good everyMinute: 8:04:00 PM, status == 200, bad everyMinute: 8:05:00 PM, status == 200, bad everyMinute: 8:06:00 PM, status == 200, bad everyMinute: 8:07:00 PM, status == 200, bad everyMinute: 8:08:00 PM, status == 200, bad everyMinute: 8:09:00 PM, status == 200, good everyMinute: 8:10:00 PM, status == 200, bad everyMinute: 8:11:00 PM, status == 200, bad everyMinute: 8:12:00 PM, status == 200, bad everyMinute: 8:13:00 PM, status == 200, bad everyMinute: 8:14:00 PM, status == 200, bad everyMinute: 8:15:00 PM, status == 200, good everyMinute: 8:16:00 PM, status == 200, bad everyMinute: 8:17:00 PM, status == 200, bad everyMinute: 8:18:00 PM, status == 200, bad everyMinute: 8:19:00 PM, status == 200, bad everyMinute: 8:20:00 PM, status == 200, bad everyMinute: 8:21:00 PM, status == 200, good everyMinute: 8:22:00 PM, status == 200, bad everyMinute: 8:23:00 PM, status == 200, bad everyMinute: 8:24:00 PM, status == 200, bad everyMinute: 8:25:00 PM, status == 200, bad everyMinute: 8:26:00 PM, status == 200, bad everyMinute: 8:27:00 PM, status == 200, good everyMinute: 8:28:00 PM, status == 200, bad everyMinute: 8:29:00 PM, status == 200, bad everyMinute: 8:30:00 PM, status == 200, bad everyMinute: 8:31:00 PM, status == 200, bad everyMinute: 8:32:00 PM, status == 200, bad everyMinute: 8:33:00 PM, status == 200, good everyMinute: 8:34:00 PM, status == 200, bad everyMinute: 8:35:00 PM, status == 200, bad everyMinute: 8:36:00 PM, status == 200, bad everyMinute: 8:37:00 PM, status == 200, bad everyMinute: 8:38:00 PM, status == 200, good everyMinute: 8:39:00 PM, status == 200, bad everyMinute: 8:40:00 PM, status == 200, bad everyMinute: 8:41:00 PM, status == 200, bad everyMinute: 8:42:00 PM, status == 200, bad everyMinute: 8:43:00 PM, status == 200, bad everyMinute: 8:44:00 PM, status == 200, good everyMinute: 8:45:00 PM, status == 200, bad everyMinute: 8:46:00 PM, status == 200, bad everyMinute: 8:47:00 PM, status == 200, bad everyMinute: 8:48:00 PM, status == 200, bad everyMinute: 8:49:00 PM, status == 200, bad everyMinute: 8:50:00 PM, status == 200, good everyMinute: 8:51:00 PM, status == 200, bad everyMinute: 8:52:00 PM, status == 200, bad everyMinute: 8:53:00 PM, status == 200, bad everyMinute: 8:54:00 PM, status == 200, bad everyMinute: 8:55:00 PM, status == 200, bad everyMinute: 8:56:00 PM, status == 200, bad everyMinute: 8:57:00 PM, status == 200, bad everyMinute: 8:58:00 PM, status == 200, bad everyMinute: 8:59:00 PM, status == 200, bad everyMinute: 9:00:00 PM, status == 200, bad everyMinute: 9:01:00 PM, status == 200, bad everyMinute: 9:02:00 PM, status == 200, bad everyMinute: 9:03:00 PM, status == 200, bad everyMinute: 9:04:00 PM, status == 200, bad everyMinute: 9:05:00 PM, status == 200, bad everyMinute: 9:06:00 PM, status == 200, bad everyMinute: 9:07:00 PM, status == 200, good everyMinute: 9:08:00 PM, status == 200, bad everyMinute: 9:09:00 PM, status == 200, bad everyMinute: 9:10:00 PM, status == 200, bad everyMinute: 9:11:00 PM, status == 200, bad everyMinute: 9:12:00 PM, status == 200, good everyMinute: 9:13:00 PM, status == 200, bad everyMinute: 9:14:00 PM, status == 200, bad everyMinute: 9:15:00 PM, status == 200, bad everyMinute: 9:16:00 PM, status == 200, bad everyMinute: 9:17:00 PM, status == 200, bad everyMinute: 9:18:00 PM, status == 200, good everyMinute: 9:19:00 PM, status == 200, bad everyMinute: 9:20:00 PM, status == 200, bad everyMinute: 9:21:00 PM, status == 200, bad everyMinute: 9:22:00 PM, status == 200, bad everyMinute: 9:23:00 PM, status == 200, good everyMinute: 9:24:00 PM, status == 200, bad everyMinute: 9:25:00 PM, status == 200, bad everyMinute: 9:26:00 PM, status == 200, bad everyMinute: 9:27:00 PM, status == 200, bad everyMinute: 9:28:00 PM, status == 200, bad everyMinute: 9:29:00 PM, status == 200, good everyMinute: 9:30:00 PM, status == 200, bad everyMinute: 9:31:00 PM, status == 200, bad everyMinute: 9:32:00 PM, status == 200, bad everyMinute: 9:33:00 PM, status == 200, bad everyMinute: 9:34:00 PM, status == 200, bad everyMinute: 9:35:00 PM, status == 200, good everyMinute: 9:36:00 PM, status == 200, bad everyMinute: 9:37:00 PM, status == 200, bad everyMinute: 9:38:00 PM, status == 200, bad everyMinute: 9:39:00 PM, status == 200, bad everyMinute: 9:40:00 PM, status == 200, bad everyMinute: 9:41:00 PM, status == 200, good everyMinute: 9:42:00 PM, status == 200, bad everyMinute: 9:43:00 PM, status == 200, bad everyMinute: 9:44:00 PM, status == 200, bad everyMinute: 9:45:00 PM, status == 200, bad everyMinute: 9:46:00 PM, status == 200, bad everyMinute: 9:47:00 PM, status == 200, good everyMinute: 9:48:00 PM, status == 200, bad everyMinute: 9:49:00 PM, status == 200, bad everyMinute: 9:50:00 PM, status == 200, bad everyMinute: 9:51:00 PM, status == 200, bad everyMinute: 9:52:00 PM, status == 200, bad everyMinute: 9:53:00 PM, status == 200, bad everyMinute: 9:54:00 PM, status == 200, bad everyMinute: 9:55:00 PM, status == 200, bad everyMinute: 9:56:00 PM, status == 200, bad everyMinute: 9:57:00 PM, status == 200, bad everyMinute: 9:58:00 PM, status == 200, bad everyMinute: 9:59:00 PM, status == 200, bad everyMinute: 10:00:00 PM, status == 200, bad everyMinute: 10:01:00 PM, status == 200, bad everyMinute: 10:02:00 PM, status == 200, bad everyMinute: 10:03:00 PM, status == 200, bad everyMinute: 10:04:00 PM, status == 200, bad everyMinute: 10:05:00 PM, status == 200, good everyMinute: 10:06:00 PM, status == 200, bad everyMinute: 10:07:00 PM, status == 200, bad everyMinute: 10:08:00 PM, status == 200, bad everyMinute: 10:09:00 PM, status == 200, bad everyMinute: 10:10:00 PM, status == 200, bad everyMinute: 10:11:00 PM, status == 200, good everyMinute: 10:12:00 PM, status == 200, bad everyMinute: 10:13:00 PM, status == 200, bad everyMinute: 10:14:00 PM, status == 200, bad everyMinute: 10:15:00 PM, status == 200, bad everyMinute: 10:16:00 PM, status == 200, bad everyMinute: 10:17:00 PM, status == 200, good everyMinute: 10:18:00 PM, status == 200, bad everyMinute: 10:19:00 PM, status == 200, bad everyMinute: 10:20:00 PM, status == 200, bad everyMinute: 10:21:00 PM, status == 200, bad everyMinute: 10:22:00 PM, status == 200, bad everyMinute: 10:23:00 PM, status == 200, good everyMinute: 10:24:00 PM, status == 200, bad everyMinute: 10:25:00 PM, status == 200, bad everyMinute: 10:26:00 PM, status == 200, bad everyMinute: 10:27:00 PM, status == 200, bad everyMinute: 10:28:00 PM, status == 200, bad everyMinute: 10:29:00 PM, status == 200, good everyMinute: 10:30:00 PM, status == 200, bad everyMinute: 10:31:00 PM, status == 200, bad everyMinute: 10:32:00 PM, status == 200, bad everyMinute: 10:33:00 PM, status == 200, bad everyMinute: 10:34:00 PM, status == 200, bad everyMinute: 10:35:00 PM, status == 200, good everyMinute: 10:36:00 PM, status == 200, bad everyMinute: 10:37:00 PM, status == 200, bad everyMinute: 10:38:00 PM, status == 200, bad everyMinute: 10:39:00 PM, status == 200, bad everyMinute: 10:40:00 PM, status == 200, good everyMinute: 10:41:00 PM, status == 200, bad everyMinute: 10:42:00 PM, status == 200, bad everyMinute: 10:43:00 PM, status == 200, bad

[allenwb]

This is pure speculation, but this variable latency might be intentional and designed to discourage to developers from doing exactly what you want to do.

[scripting]

@allenwb I’ve had the same thought.

[imsickofmaps]

I think if you hit the raw link for the latest commit, you'll always get a true version. E.g. https://raw.githubusercontent.com/scripting/instantOutlines/00631584b899d8037b9ae1c9dd7f3ac3aab50365/misc/now.txt Is it too complex for the Node app to know what the latest commit SHA was and link to it directly?

Raw pages are served with a 300 second cache-control value.

[scripting]

@imsickofmaps -- that's an interesting bit of info. I'm going to try to test it and will report back.

However it's not a solution. The URL by which apps reference the data must remain fixed, it's like the URL of an RSS feed. I could add another layer of interpretation to make it work, set up a web service to translate, maybe I should do that? Maybe.

Thanks for the tip...

[scripting]

BTW, this is the JSON response you get from GitHub after uploading a file.

https://gist.github.com/scripting/9f21e03f4180cc7c3da84140d2247b86

[scripting]

@imsickofmaps -- it is a solution.

Here's the result of the first ten minutes...

everyMinute: 10:34:30 AM, status == 200, good everyMinute: 10:35:00 AM, status == 200, good everyMinute: 10:36:00 AM, status == 200, good everyMinute: 10:37:00 AM, status == 200, good everyMinute: 10:38:00 AM, status == 200, good everyMinute: 10:39:00 AM, status == 200, good everyMinute: 10:40:00 AM, status == 200, good everyMinute: 10:41:00 AM, status == 200, good everyMinute: 10:42:00 AM, status == 200, good everyMinute: 10:43:00 AM, status == 200, good everyMinute: 10:44:00 AM, status == 200, good

At first I thought I wouldn't be able to use it, but now I have an idea.

I'll buy you a beer next time I see you. Thanks. :-)

[scripting]

BTW -- here's the relevant code --

uploadToGithub ("misc/now.txt", nowtext, "text/plain", function (err, githubResponse) {
    var githubRawUrl = "https://raw.githubusercontent.com/scripting/instantOutlines/" + githubResponse.commit.sha + "/misc/now.txt";
    setTimeout (function () { //wait ten seconds to check
        request (githubRawUrl, function (err, response, data) {
            var result;
            if (data.toString () == nowtext) {
                result = "good";
                }
            else {
                result = "bad";
                }
            console.log ("everyMinute: " + nowtext + ", status == " + response.statusCode + ", " + result); 
            });
        }, 1000 * 10);
    });

[andrewshell]

I'm finding that if instead of using raw.githubusercontent.com I'm writing to the gh-pages branch and reading through GitHub pages it's live within 20-30 seconds

I just added ref: "gh-pages" to options and changed the logic to:

readConfig ("config.json", config, function () {
    console.log ("config == " + utils.jsonStringify (config));

    var nowtext = new Date ().toLocaleTimeString ();
    var start = Date.now();
    uploadToGithub ("now.txt", nowtext, "text/plain", function (err) {
        setInterval (function () { //wait ten seconds to check
            request ("https://andrewshell.github.io/testgit/now.txt", function (err, response, data) {
                var result;
                let end = Date.now()
                let elapsed = (end - start) / 1000;
                console.log(elapsed + ': ' + data.toString ());
                if (data.toString () == nowtext) {
                    result = "good";
                    }
                else {
                    result = "bad";
                    }
                console.log (elapsed + ": " + nowtext + ", status == " + response.statusCode + ", " + result);
                });
            }, 1000 * 10);
        });
    });

Seeing responses like these:

11.198: 9:43:44 AM
11.198: 9:46:06 AM, status == 200, bad
21.118: 9:46:06 AM
21.118: 9:46:06 AM, status == 200, good
31.127: 9:46:06 AM
31.127: 9:46:06 AM, status == 200, good

11.21: 9:46:06 AM
11.21: 9:46:42 AM, status == 200, bad
21.063: 9:46:06 AM
21.063: 9:46:42 AM, status == 200, bad
30.959: 9:46:42 AM
30.959: 9:46:42 AM, status == 200, good

[scripting]

I've been running a tougher test.

The previous version waited ten seconds after writing before reading, the new version does not wait.

everyMinute: 10:56:24 AM, status == 200, good everyMinute: 10:57:00 AM, status == 200, good everyMinute: 10:58:00 AM, status == 200, good everyMinute: 10:59:00 AM, status == 200, good everyMinute: 11:00:00 AM, status == 200, good everyMinute: 11:01:00 AM, status == 200, good everyMinute: 11:02:00 AM, status == 200, good everyMinute: 11:03:00 AM, status == 200, good everyMinute: 11:04:00 AM, status == 200, good everyMinute: 11:05:00 AM, status == 200, good everyMinute: 11:06:00 AM, status == 200, good everyMinute: 11:07:00 AM, status == 200, good everyMinute: 11:08:00 AM, status == 200, good everyMinute: 11:09:00 AM, status == 200, good everyMinute: 11:10:00 AM, status == 200, good everyMinute: 11:11:00 AM, status == 200, good everyMinute: 11:12:00 AM, status == 200, good everyMinute: 11:13:00 AM, status == 200, good everyMinute: 11:14:00 AM, status == 200, good everyMinute: 11:15:00 AM, status == 200, good everyMinute: 11:16:00 AM, status == 200, good everyMinute: 11:17:00 AM, status == 200, good everyMinute: 11:18:00 AM, status == 200, good everyMinute: 11:19:00 AM, status == 200, good

The results as you can see are perfect.

[imsickofmaps]

@scripting 👍 -- as long as your routing code knows the latest SHA that seems like a solution. It could be a Node app or I was even thinking if you wanted to keep it pure HTTP you could "publish" a web server config file (things like nginx or apache etc.) that does a 303 or 307 to your SHA inclusive URL.

[scripting]

@imsickofmaps -- yes, but --

the point of this application is to share data with apps using github.

if the data doesn't have a github url, it pretty much could be anywhere, nothing magic about using github.

crossing this line makes things sloppy and confusing, when they were already pretty confusing to begin with.

what we really need is a system that was thought-out to be good for this use.

[scripting]

actually there is something nice about using github. the webhooks interface is a big part of what's needed out of the storage system.

[imsickofmaps]

Understood and agree! The thing that I like about the path of using git is you can, if you want, use the standard git protocol stuff for local vs remote etc. The only reason I've thought about this stuff before was from working alongside a team on a large scale CMS publishing infrastructure across poorly connected nodes on the African continent that used a similar approach build on top of the git protocol.

[scripting]

@imsickofmaps --

I just checked my systems that rely on this, and now they seem to be performing well.

A few days ago, they weren't working well at all. What did I change? Nothing! ;-)

So I'm going to proceed as we were before, and aware there is a backup plan if things get iffy again.

[scripting]

@andrewshell -- thanks for taking a look at this. in the interim @imsickofmaps offered a perfect solution with zero latency, so I have to go that way. :boom: