dariusk / rss-to-activitypub

An RSS to ActivityPub converter.
MIT License
556 stars 27 forks source link

Refresh header support #57

Open SoniEx2 opened 4 years ago

SoniEx2 commented 4 years ago

:joy:

I feel like the Refresh header would fit in great with RSS feeds, and other systems where things should be refreshed every now and then. I added this to my software recently: ```python class RemoteDataSource(ObjectDataSource): def __init__(self, uri): super().__init__({}) self.uri = uri self.remote_exists = False self.next_update = 0 def update(self): if self.next_update > time.time(): return # I long for the day when toml has a registered media type response = requests.get(self.uri, headers={'user-agent': 'ganarchy/0.0.0', 'accept': '*/*'}) self.remote_exists = response.status_code == 200 seconds = 3600 if (refresh := response.headers.get('Refresh', None)) is not None: try: seconds = int(refresh) except ValueError: refresh = refresh.split(';', 1) try: seconds = int(refresh[0]) except ValueError: pass self.next_update = time.time() + seconds if self.remote_exists: response.encoding = 'utf-8' try: self._obj = qtoml.loads(response.text) except (UnicodeDecodeError, qtoml.decoder.TOMLDecodeError) as e: self._obj = {} return e else: return response def exists(self): return self.remote_exists def __repr__(self): return "RemoteDataSource({!r})".format(self.uri) ``` maybe you too can make use of the Refresh header?