search

everypolitician / scraped_page_archive

Create an archive of HTML pages scraped by a Ruby scraper
MIT License
1 stars 0 forks source link

Error when running with Poltergeist #35

Open chrismytton opened 8 years ago

chrismytton commented 8 years ago

Problem

https://github.com/everypolitician-scrapers/uganda-parliament-scraper/pull/6

When I use the Capybara::Session#reset! method an exception is raised. This only seems to happen when I've got a require 'scraped_page_archive/capybara' line in the script, so I think it's a bug with this repo, not with Poltergeist. The odd thing is the error seems to be coming from the original command method, not our monkeypatched version. So something about the state the program is in is making poltergeist unhappy.

$ bundle exec ruby scraper.rb 
/Users/chris/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/poltergeist-1.6.0/lib/capybara/poltergeist/browser.rb:323:in `command': There was an error inside the PhantomJS portion of Polte
rgeist. This is probably a bug, so please report it.  (Capybara::Poltergeist::BrowserError)

TypeError: undefined is not an object (evaluating 'this._responseHeaders.forEach')
responseHeaders@phantomjs://code/web_page.js:244:26
response_headers@phantomjs://code/browser.js:490:62
runCommand@phantomjs://code/browser.js:51:28
runCommand@phantomjs://code/main.js:21:37
commandReceived@phantomjs://code/connection.js:16:33
phantomjs://code/connection.js:1:66
        from /Users/chris/Developer/scraped_page_archive/lib/scraped_page_archive/capybara.rb:53:in `command'
        from /Users/chris/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/poltergeist-1.6.0/lib/capybara/poltergeist/browser.rb:262:in `response_headers'
        from /Users/chris/Developer/scraped_page_archive/lib/scraped_page_archive/capybara.rb:36:in `get_details'
        from /Users/chris/Developer/scraped_page_archive/lib/scraped_page_archive/capybara.rb:58:in `block in command'
        from /Users/chris/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/vcr-3.0.3/lib/vcr/util/variable_args_block_caller.rb:9:in `call'
        from /Users/chris/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/vcr-3.0.3/lib/vcr/util/variable_args_block_caller.rb:9:in `call_block'
        from /Users/chris/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/vcr-3.0.3/lib/vcr.rb:189:in `use_cassette'
        from /Users/chris/Developer/scraped_page_archive/lib/scraped_page_archive.rb:30:in `record'
        from /Users/chris/Developer/scraped_page_archive/lib/scraped_page_archive.rb:20:in `record'
        from /Users/chris/Developer/scraped_page_archive/lib/scraped_page_archive/capybara.rb:57:in `command'
        from /Users/chris/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/poltergeist-1.6.0/lib/capybara/poltergeist/browser.rb:195:in `reset'
        from /Users/chris/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/poltergeist-1.6.0/lib/capybara/poltergeist/driver.rb:177:in `reset!'
        from /Users/chris/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/capybara-2.5.0/lib/capybara/session.rb:108:in `reset!'
        from scraper.rb:39:in `block in scrape_list'
        from scraper.rb:37:in `map'
        from scraper.rb:37:in `scrape_list'
        from scraper.rb:32:in `block in scrape_all_terms'
        from scraper.rb:31:in `upto'
        from scraper.rb:31:in `each'
        from scraper.rb:31:in `map'
        from scraper.rb:31:in `scrape_all_terms'
        from scraper.rb:15:in `scrape'
        from scraper.rb:125:in `<main>'