Open danshearer opened 2 years ago
Hi @danshearer,
I've been meaning to give this gem some love. I definitely intend to maintain it.
This particular error looks like it might not be an issue with wayback_archiver, it rather looks like the request timed out. Net::HTTP will raise that exception if it takes more than 60 seconds to open a connections (documentation for Net:HTTP for Ruby 2.7).
Some questions to help me debug if there's an issue 👇
How are you invoking wayback_archiver
? CLI or with Ruby code and can you paste it here?
Does this happen consistently? Does it work if you try archiving another site?
Thanks!
Oh it is certainly a timeout issue.
But I was thinking that really this should be caught and presented as an error, and potentially even a back off and retry? It's a different case to being hard rartelimited, and is hopefully transient.
I was thinking that a backtrace was a little brutal, given this context?
I see where you come from. I'm not entirely sure that I'd want to try to capture and re-raise each possible exception, however there's definitely a use case for retrying failed requests. I've added that as a possible feature in the v2 roadmap 👉 #68.
I got this traceback:
/usr/lib/ruby/2.7.0/net/http.rb:960:in `initialize': execution expired (Net::OpenTimeout)
This was with the wayback_archive 1.4.0 gem, reproducible with latest github code.
I'm not sure if this project is still maintained, but thanks anyway.
Dan