What time does spacer use?

[tkellogg]

I'm unsure if this is a bug or a feature, but spacer seems to use CPU wall clock time when determining how long it's waited. I tend to think of this as a bug because it's idea of "time passed" is difficult to articulate or reason about. I honestly can't explain what spacer is doing, aside from randomly inserting breaks (which is still helpful, but it bugs me because i can't explain it).

Sample output from my HN scraper. This is using tail -f keyword_links.txt | spacer -a 3600. There's 2 days between these 2 spacers, spacer is set to 1 hour, yet the script seems to have been periodically active throughout the entire weekend while my laptop was asleep.

{"ts": "2023-06-18T09:41:17.663962", "title": "How to Create a UTF-16 Surrogate Pair by Hand, with Python (oilshell.org)", "href": "https://www.oilshell.org/blog/2023/06/surrogate-pair.html", "comments": "https://news.ycombinator.com/item?id=36371661"}
2023-06-18 12:00:30 4.1h ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
{"ts": "2023-06-18T12:10:37.432999", "title": "Gas Stoves Emit Benzene and Increases Indoor Air Pollution (acs.org)", "href": "https://pubs.acs.org/doi/10.1021/acs.est.2c09289#", "comments": "https://news.ycombinator.com/item?id=36381189"}
{"ts": "2023-06-18T12:10:37.634425", "title": "Hexa Lift: Single person drone (liftaircraft.com)", "href": "https://www.liftaircraft.com", "comments": null}
{"ts": "2023-06-18T12:44:41.840386", "title": "OpenLLaMA 13B Released (huggingface.co)", "href": "https://huggingface.co/openlm-research/open_llama_13b", "comments": "https://news.ycombinator.com/item?id=36381136"}
{"ts": "2023-06-18T15:04:59.489717", "title": "The WFH Future Is Destroying Bosses' Brains (wheresyoured.at)", "href": "https://wheresyoured.at/p/the-work-from-home-future-is-destroying", "comments": null}
{"ts": "2023-06-18T19:10:01.413910", "title": "You're neither in control nor responsible for their failures (sudarkoff.blog)", "href": "https://www.sudarkoff.blog/p/youre-neither-in-control-nor-responsible", "comments": null}
{"ts": "2023-06-18T19:10:02.593397", "title": "LLM Fine Tuning Guide for Enterprises in 2023 (aimultiple.com)", "href": "https://research.aimultiple.com/llm-fine-tuning/", "comments": "https://news.ycombinator.com/item?id=36383298"}
{"ts": "2023-06-18T19:10:03.794215", "title": "ChatGPT, Google Bard Generates Generic Windows 11, Windows 10 Pro Keys (tomshardware.com)", "href": "https://www.tomshardware.com/news/chatgpt-generates-windows-11-pro-keys", "comments": "https://news.ycombinator.com/item?id=36385032"}
{"ts": "2023-06-19T00:42:58.508460", "title": "Releasing an indie game on 3 consoles at once and failing financially (juicybeast.com)", "href": "https://juicybeast.com/2016/01/11/releasing-an-indie-game-on-3-consoles-at-once-and-failing-financially/#2-years", "comments": "https://news.ycombinator.com/item?id=36387030"}
{"ts": "2023-06-19T01:51:33.392465", "title": "Analogical Diagram from Tobias Cohen\u2019s Ma\u2019aseh Tuviyah (1708) (publicdomainreview.org)", "href": "https://publicdomainreview.org/collection/body-as-house-diagram", "comments": null}
{"ts": "2023-06-19T01:51:34.123682", "title": "WSL 1.3.10 Brings Experimental Memory Reclaim, Updated DXCore and Linux Kernel (phoronix.com)", "href": "https://www.phoronix.com/news/Microsoft-WSL-1.3.10", "comments": "https://news.ycombinator.com/item?id=36387197"}
{"ts": "2023-06-19T06:10:40.154780", "title": "OpenLLM (github.com/bentoml)", "href": "https://github.com/bentoml/OpenLLM", "comments": "https://news.ycombinator.com/item?id=36388219"}
{"ts": "2023-06-19T06:10:41.385718", "title": "MusicGen-Looper: Generate fixed-bpm loops from text prompts (replicate.com)", "href": "https://replicate.com/andreasjansson/musicgen-looper", "comments": "https://news.ycombinator.com/item?id=36388919"}
{"ts": "2023-06-19T06:10:42.455880", "title": "Invisibility Training for Motorcyclists [video] (youtube.com)", "href": "https://www.youtube.com/watch?v=x94PGgYKHQ0", "comments": "https://news.ycombinator.com/item?id=36375118"}
{"ts": "2023-06-19T06:10:43.999618", "title": "Popeye the Sailor Meets Ali Baba's Forty Thieves (1937) (nstr.no)", "href": "https://video.nstr.no/videos/watch/4ba21e69-2813-4242-b792-1f0301b2350c", "comments": "https://news.ycombinator.com/item?id=36381141"}
{"ts": "2023-06-19T06:10:44.386096", "title": "Merged magnetic resonance and light sheet microscopy of the whole mouse brain (pnas.org)", "href": "https://www.pnas.org/doi/10.1073/pnas.2218617120", "comments": null}
{"ts": "2023-06-19T07:44:44.192225", "title": "We tried to book a train ticket and ended up with a 245,000 records data breach (zerforschung.org)", "href": "https://zerforschung.org/posts/freundschaftspass-en/", "comments": "https://news.ycombinator.com/item?id=36388828"}
{"ts": "2023-06-19T08:53:33.165641", "title": "We need more of Richard Stallman, not less (ploum.net)", "href": "https://ploum.net/2023-06-19-more-rms.html", "comments": "https://news.ycombinator.com/item?id=36389805"}
{"ts": "2023-06-19T09:28:28.295303", "title": "Twitter\u2019s Lawyers Admit They\u2019re Overwhelmed; 2000 Laid Off Employees File Claims (techdirt.com)", "href": "https://www.techdirt.com/2023/06/16/twitters-lawyers-admit-theyre-overwhelmed-as-nearly-2000-laid-off-employees-file-arbitration-claims/", "comments": "https://news.ycombinator.com/item?id=36390308"}
{"ts": "2023-06-19T11:06:45.754068", "title": "Plain Text Journaling in Vim (peppe.rs)", "href": "https://peppe.rs/posts/plain_text_journaling/", "comments": "https://news.ycombinator.com/item?id=36390405"}
{"ts": "2023-06-19T11:06:46.943889", "title": "KDE Plasma 6 Has Reached the Point of Being \u201cFairly Livable\u201d (phoronix.com)", "href": "https://www.phoronix.com/news/KDE-Plasma-6-Fairly-Livable", "comments": "https://news.ycombinator.com/item?id=36391492"}
{"ts": "2023-06-19T11:06:48.084486", "title": "The Coolest Library on Earth (hakaimagazine.com)", "href": "https://hakaimagazine.com/features/the-coolest-library-on-earth/", "comments": "https://news.ycombinator.com/item?id=36380892"}
{"ts": "2023-06-19T12:31:48.674197", "title": "Paris air show takes off with biggest ever plane order (reuters.com)", "href": "https://www.reuters.com/business/aerospace-defense/paris-air-show-returns-with-jets-missiles-demand-2023-06-19/", "comments": null}
{"ts": "2023-06-19T13:24:16.328866", "title": "Power companies spend millions to fight Maine\u2019s proposed not for profit utility (theguardian.com)", "href": "https://www.theguardian.com/us-news/2023/jun/18/maine-state-run-utility-power-companies", "comments": "https://news.ycombinator.com/item?id=36392577"}
{"ts": "2023-06-19T13:24:17.336574", "title": "Meta introduces Voicebox: state-of-the-art generative AI model for speech (facebook.com)", "href": "https://ai.facebook.com/blog/voicebox-generative-ai-model-speech/", "comments": null}
{"ts": "2023-06-19T14:25:19.850143", "title": "Hurl: A terrible but cute idea for a language (ntietz.com)", "href": "https://ntietz.com/blog/introducing-hurl/", "comments": "https://news.ycombinator.com/item?id=36393673"}
{"ts": "2023-06-19T14:25:20.015015", "title": "Show HN: Python package for interfacing with ChatGPT with minimized complexity (github.com/minimaxir)", "href": "https://github.com/minimaxir/simpleaichat", "comments": null}
{"ts": "2023-06-19T15:17:35.531759", "title": "AI: First New UI Paradigm in 60 Years, Jakob Nielsen (nngroup.com)", "href": "https://www.nngroup.com/articles/ai-paradigm/", "comments": null}
{"ts": "2023-06-19T16:54:17.770726", "title": "Dunning\u2013Kruger effect and flat-earthers: An exploratory analysis (sagepub.com)", "href": "https://journals.sagepub.com/doi/10.1177/09636625231166255", "comments": "https://news.ycombinator.com/item?id=36395856"}
{"ts": "2023-06-19T18:16:11.137002", "title": "Thousands of Azure Websites Are Still Vulnerable to Subdomain Takeovers (keytos.io)", "href": "https://www.keytos.io/blog/2023/05/23/azure-domains-still-vulnerable-to-subdomain-takeover.html", "comments": null}
{"ts": "2023-06-19T19:02:57.779949", "title": "SVB customers who lost their deposits remain on the hook for loans (wsj.com)", "href": "https://www.wsj.com/articles/svb-customers-who-lost-their-deposits-remain-on-the-hook-for-loans-5df7963b", "comments": "https://news.ycombinator.com/item?id=36396932"}
{"ts": "2023-06-19T19:51:49.391127", "title": "Mum\u2019s microbes might boost brain development of C-section babies (nature.com)", "href": "https://www.nature.com/articles/d41586-023-01972-4", "comments": null}
{"ts": "2023-06-20T00:21:26.081601", "title": "A daytime nap is good for the brain (bbc.com)", "href": "https://www.bbc.com/news/health-65950168", "comments": null}
{"ts": "2023-06-20T00:21:27.387384", "title": "TerraUSD: South Korea 'cryptocrash king\u2019 Do Kwon jailed (bbc.co.uk)", "href": "https://www.bbc.co.uk/news/business-65957999", "comments": "https://news.ycombinator.com/item?id=36399514"}
{"ts": "2023-06-20T01:38:51.213969", "title": "ChatGPT: Fear Litany (openai.com)", "href": "https://chat.openai.com/share/312e82f0-cc5e-47f3-b368-b2c0c0f4ad3f", "comments": null}
{"ts": "2023-06-20T03:57:53.971820", "title": "ChatGPT: Fear Litany (archive.org)", "href": "https://web.archive.org/web/20230620052350/https://chat.openai.com/share/312e82f0-cc5e-47f3-b368-b2c0c0f4ad3f", "comments": "https://news.ycombinator.com/item?id=36400053"}
{"ts": "2023-06-20T08:55:37.692695", "title": "Media Objectivity and Bias in Western Coverage of the Russian-Ukrainian Conflict (diva-portal.org)", "href": "http://sh.diva-portal.org/smash/record.jsf?pid=diva2%3A1769389&dswid=-966", "comments": null}
2023-06-20 10:41:21 6.8h ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
{"ts": "2023-06-20T10:52:26.220057", "title": "Undb \u2013 open-source airtable alternative (github.com/undb-xyz)", "href": "https://github.com/undb-xyz/undb", "comments": null}

[samwho]

This could certainly be more clear.

When you set -a 3600 it will output a spacer after 3600 seconds of no activity. So if 1 hour passes with nothing being written to the terminal, a spacer will be output.

Any time a new line gets output to the terminal, the timer is reset.

Does that help? I'm not entirely certain I've answered your question.

[tkellogg]

Shouldn't there have been a spacer between these records then?

{"ts": "2023-06-19T01:51:34.123682", "title": "WSL 1.3.10 Brings Experimental Memory Reclaim, Updated DXCore and Linux Kernel (phoronix.com)", "href": "https://www.phoronix.com/news/Microsoft-WSL-1.3.10", "comments": "https://news.ycombinator.com/item?id=36387197"}
{"ts": "2023-06-19T06:10:40.154780", "title": "OpenLLM (github.com/bentoml)", "href": "https://github.com/bentoml/OpenLLM", "comments": "https://news.ycombinator.com/item?id=36388219"}

(Also, I'm not sure how often MacOS wakes up to process these ongoing scripts. That could definitely be playing into it as well)

[samwho]

Yes, there certainly should.

The way spacer is implemented uses a background thread to check if enough time has passed to print a spacer. When your Mac is asleep, this thread may not get a chance to run. It's plausible, but tricky to verify with the existing spacer binary (there's no debug logging of any kind).

If sleep is preventing this, and when it wakes up the main thread goes first, reads a line from STDIN, and resets the timer, that could cause the behaviour you're seeing. I'm not sure how I'd go about "fixing" this.

[tkellogg]

This could be a terrible idea, but can you use the notify crate to be notified when changes happen, rather than sleeping?

[samwho]

The thread that does the printing would still need to sleep. It prints the spacer after a given timeout, so it needs to wake up independently of anything happening to STDIN. I don't think the notify crate would help.

[samwho]

Hey Tim, checking in on this. I don't think there's much help I can offer without more info. Are you able to reproduce this behaviour on the latest version and without your laptop being asleep?

[tkellogg]

We can close this issue