Switch from open-uri to HTTParty - as it seems to be able to handle protocol downgrade redirects (HTTPS > HTTP) natively - Closes #16
Change error handling, since HTTParty does not raise exceptions by default. Result to the user should be the same unless we screwed something.
There code that was responsible for converting relative links to absolute was completely wrong. Now using URI.join instead - this means that previously snapcrawl sometimes crawled deeper than it should have.
open-uritoHTTParty- as it seems to be able to handle protocol downgrade redirects (HTTPS > HTTP) natively - Closes #16URI.joininstead - this means that previously snapcrawl sometimes crawled deeper than it should have.