Closed lemon24 closed 4 months ago
Logic is easy enough (turning update_after etc. to/from seconds not shown):
import random
from dataclasses import dataclass
@dataclass
class Feed:
url: str
update_period: int
update_after: int = 0
# last_updated is only set when the feed is actually updated
# (not when it's not modified, not when there was an exception)
# https://github.com/lemon24/reader/blob/3.12/src/reader/_update.py#L276
# https://github.com/lemon24/reader/blob/3.12/src/reader/_update.py#L445
last_retrieved: int = 0
def get_feeds_for_update(feeds, now):
return [f for f in feeds if f.update_after <= now]
def next_period(feed, now, jitter_ratio=0):
jitter = random.random() * jitter_ratio
current_period_no = now // feed.update_period
return (current_period_no + 1 + jitter) * feed.update_period
def update_feeds(feeds, now, get_update_after=next_period):
to_update = get_feeds_for_update(feeds, now)
for feed in to_update:
feed.last_retrieved = now
feed.update_after = get_update_after(feed, now)
return to_update
def set_update_period(feed, update_period):
feed.update_period = update_period
feed.update_after = next_period(feed, feed.last_retrieved)
On to the API!
Update: We can get rid of last_retrieve and rely on the current time in set_update_period(); all tests pass with minimal changes.
API
Add update_feeds(scheduled: bool | None = None)
argument that filters feeds to update:
.reader.update
is set (regardless of value), assume true, else assume falseupdate_after < now
update_after
filtering)In reader 4.0 (#291), we can make scheduled
default to True (or just change the default behavior).
To configure the update interval, we can use a .reader.update
tag:
interval: int
(seconds), jitter: float|bool
(in [0,1])
@hourly, @daily, @weekly, @monthly
values{interval: 3600, jitter: 0.25}
Using tags is good because it allows configuring stuff without additional UI.
Possible cons (WIP):
update_after
when the update interval changes (a dedicated set_feed_update_period()
method can take care of this out of the box). One solution may be being able to register hooks on tag updates.In the (low level) storage API:
update_after
and last_retrieved
to FeedUpdateIntent and FeedForUpdateget_feeds_for_update(after: datetime | None)
(also returns nulls)
new
uses last_updated
, it should use last_retrieved
set_feed_update_after()
(used when setting the interval)To do (minimal):
Later:
update_after and last_retrieved should go on FeedUpdateIntent, and in turn FeedUpdateIntent should have union of FeedData-with-extra-stuff or exception or None, but I'm having trouble finding a good name for FeedData-with-extra-stuff.
For reference, here's all the feed-related data classes and how they're used:
.-- FeedForUpdate ---.
v |
parser |
| |
ParsedFeed storage -.
(has FeedData) ^ |
v | |
updater | |
| | |
|- FeedUpdateIntent -' Feed
| (has ??? (has FeedData) |
| or ExceptionInfo |
| or None) |
| v
'---- UpdateResult -----> user
(has UpdatedFeed)
307 needs a way of changing the feed update frequency. Just like https://github.com/lemon24/reader/issues/96#issuecomment-1236304134, this should be a configurable global/per-feed strategy.
Use cases:
update_feeds(new=True)
update_feeds(updates_enabled=True)
A unified way of updating feeds would be nice as well, e.g. "just call this every minute". Currently, I'm running the following in a cron:
update
update --new-only; search update