search

huginn / huginn

Create agents that monitor and act on your behalf. Your agents are standing by!
MIT License
43.11k stars 3.75k forks source link

HTTP 504 breaks RSS Agent #3309

Open level20peon opened 1 year ago

level20peon commented 1 year ago

I have a list of RSS URLs which I want to grab with the RSS Agent. Some of those URLs produce intermittent 504 Gateway Timeouts. If that happens, the Agent will time out at some point, but then won't "return", so it essentially becomes blocked and no further jobs are being processed.

Is there a way to handle errors such as this in huginn? Right now I work around this by retreiving those URLs via wget (error handling via bash script), dump them to a file and then read those dumps with huginn via LocalFileAgent / FileReadAgent.

arvindcj commented 1 year ago

@level20peon could you give a few examples of problematic URL's?

level20peon commented 1 year ago

@arvindcj I can't do that since those feeds contain sensitive information. However, simulating a 504 is pretty easy. For example you can find a readily available 504 "simulator" here:

http://httpbin.org/status/504