search

benbalter / sitemap-parser

Ruby Gem to parse sitemaps.org compliant sitemaps
MIT License
29 stars 46 forks source link

The Typhoeus dependency breaks SSL in Ruby 2.7 #27

Open heaven opened 1 year ago

heaven commented 1 year ago

Describe the bug

I didn't dig deeply yet but after performing a Typhoeus::Request all other attempts to open an URL even with the URI.open fail.

Steps to reproduce the behavior

URI.open("https://removeglassdoorreviews.com/sitemap.xml")
 => #<StringIO:0x00007f0ae7fd82e0 ...

SitemapParser.new("https://removeglassdoorreviews.com/sitemap.xml", recurse: true).urls
Traceback (most recent call last):
/app/vendor/bundle/ruby/2.7.0/gems/sitemap-parser-0.5.6/lib/sitemap-parser.rb:20:in `block in raw_sitemap': HTTP request to https://removeglassdoorreviews.com/sitemap.xml failed (RuntimeError)

URI.open("https://removeglassdoorreviews.com/sitemap.xml")
Traceback (most recent call last):
/app/vendor/ruby-2.7.8/lib/ruby/2.7.0/net/protocol.rb:44:in `connect_nonblock': Connection reset by peer - SSL_connect (Errno::ECONNRESET)

Expected behavior

I'd say the Typhoeus requirement is quite unnecessary here. The same could be accomplished with the standard library. Especially considering the number of issues https://github.com/typhoeus/typhoeus/issues

sitemap-parser 0.5.6 typhoeus 1.4.0 rails 5.2.8.1 ruby 2.7.8

benbalter commented 1 year ago

I'd say the Typhoeus requirement is quite unnecessary here. The same could be accomplished with the standard library. Especially considering the number of issues https://github.com/typhoeus/typhoeus/issues

Agreed. I won't be able to do that myself, but I'd be happy to review a PR that does so.