broken feed

[dzwdz]

So, rachelbythebay's feed broke for me.

feed details:
http://rachelbythebay.com/w/atom.xml

Title: 
Description: ""
Refresh: Jan 01, 00:00:00 UTC
0 Items:

So, first off, a small feature request - it'd be nice to be able to see the last error (if any), and the last status code. Without that, I have to diagnose the problem blind, but I think the issue is not supporting Last-Modified/ETag (note that the former can be tricky). Apart from fixing this particular feed, this will reduce your bandwidth usage, and lessen the load on people's servers, which is a win-win.

[biox]

ah yep - i've noticed this (& even read the same rachelbythebay post!) but i haven't had the time to implement ETag support. bubbling fetch failures up to users is another feature that i've long wanted to add, but haven't had the time. i think your intuition is spot on - i invited you to be a collaborator on this repo btw, in case you have the motivation to implement these features yourself / help guide other people.

i may not have time for it for a few months, given that it's summer & my day job is picking up steam. fetch failures is higher priority imho. i think the implementation should probably be something like:

• add a bool column to the feed table called last_fetch_failed that defaults to false
• if last_fetch_failed is equal to true, surface the fetch_failure (this column already exists) on the /feeds page under the appropriate feed.

i was also thinking about adding an attempt fetch button so that users may try re-fetching, but i'm hesitant because i don't want a user to smash that button a whole bunch and spam requests to random endpoints. vore already has SSRF issues ;)

[dzwdz]

Thanks for the invitation, I'll consider it. But also, a few thoughts:

• Isn't last_fetch_failed redundant? Why not check if fetch_error is not null, and if so, prioritize the feed?
• The attempt fetch button seems like trouble. Maybe the refresh time could be lowered for dead feeds, instead?
• How do you handle schema changes?

[biox]

How do you handle schema changes?

i added a migration system in 8f3db736

Isn't last_fetch_failed redundant? Why not check if fetch_error is not null, and if so, prioritize the feed?

good call. for now, i've exposed fetch failures to users, implemented in dff68a9b

The attempt fetch button seems like trouble. Maybe the refresh time could be lowered for dead feeds, instead?

agreed, but there needs to be a way to tell the difference between an actively dead feed and a feed that failed a fetch at some point in the past.