Handling of HTTP errors when harvesting a feed

[ldodds]

The specification could give some indication about what consumers should do if/when HTTP errors are encountered when processing a feed, e.g:

• In case of a a server error, the client might retry at a later date
• In case of a 404, when following a next page link, should the client retry or is this a more severe problem?
• If a client receives a redirect, then should the use the original request URL or the new target as the comparison URL when checking to see if they're reached the end of the feed?

[ldodds]

I see Section 5.1.4 addresses some of this already.

I'd suggest adding 500 as an option for other server side errors than just overloading/maintenance. Helps clarify with clients that there may be something to report.

Spec currently constrains implementations from not using redirects, so that isn't likely to be an option. Although I think this should be allowed.

[ldodds]

Decided not to specify 500 error as the expected behaviour is close to 503.

@nickevansuk how would you feel about revising this section slightly to say:

• Clients should expect 5XX errors and back off from polling
• Servers should use 503 to indicate temporary errors (and indicate a window)
• Servers should use 404 or better 410 for feeds that are no longer active.

It's a minor point but gives servers a bit more flexibility. I think in practice they're likely to issue a range of 5XX errors. Pretty sure I've seen a gateway or vanilla 500 error.

[nickevansuk]

Agree @ldodds, 5XX errors allow more flexibility and have also seen gateways and CDNs produce 500s.