search

dannyvassallo / insta_scrape

The instagram swiss army knife. Restores all deprecated hashtag functionality and grants public api access from instagram's front end without any of the authorization.
https://rubygems.org/gems/insta_scrape
MIT License
60 stars 27 forks source link

Capybara::Poltergeist::JavascriptError when trying anything #18

Closed gchiaren closed 7 years ago

gchiaren commented 7 years ago

I added insta_scrape 1.1.2 to my current Rails project, Rails 5.0.3 with ruby 2.3.1p112. When trying to run any insta_scrape methods from a console session, I get the following error:

Capybara::Poltergeist::JavascriptError: Capybara::Poltergeist::JavascriptError
    from /Users/me/.rvm/gems/ruby-2.3.1/gems/poltergeist-1.9.0/lib/capybara/poltergeist/browser.rb:351:in `command'
    from /Users/me/.rvm/gems/ruby-2.3.1/gems/poltergeist-1.9.0/lib/capybara/poltergeist/browser.rb:34:in `visit'
    from /Users/me/.rvm/gems/ruby-2.3.1/gems/poltergeist-1.9.0/lib/capybara/poltergeist/driver.rb:95:in `visit'
    from /Users/me/.rvm/gems/ruby-2.3.1/gems/capybara-2.7.1/lib/capybara/session.rb:233:in `visit'
    from /Users/me/.rvm/gems/ruby-2.3.1/gems/capybara-2.7.1/lib/capybara/dsl.rb:52:in `block (2 levels) in <module:DSL>'
    from /Users/me/.rvm/gems/ruby-2.3.1/gems/insta_scrape-1.1.2/lib/insta_scrape.rb:105:in `scrape_user_info'
    from /Users/me/.rvm/gems/ruby-2.3.1/gems/insta_scrape-1.1.2/lib/insta_scrape.rb:35:in `user_info'

In this particular instance I ran InstaScrape.user_info("foofighters").

Interestingly, if I go to line 105 of insta_scrape.rb and run extend Capybara::DSL followed by visit "https://www.instagram.com/foofighters/", occasionally it works, and occasionally it throws the same error.

What could be the issue?

dannyvassallo commented 7 years ago

The gem includes capybara and poltergeist as dependencies, do you also have them included in your gemfile? Maybe a potential version mismatch? Try removing your own instance of capybara if thats the case and see if it still behaves. Let me know if you have any success -- otherwise I can look into this further for you. @gchiaren

gchiaren commented 7 years ago

Yes, I have capybara (2.7.1) and poltergeist (1.9.0) in my bundle. I can see those aren't the latest versions of those gems, but based on the gemspec they're what you recommend.

I can try explicitly including them in my gemfile as you suggested, and report back with the results.

gchiaren commented 7 years ago

No luck. I tried installing the latest versions of capybara and poltergeist, followed by insta_scrape, and that made insta_scrape completely not work. For some reason it installed version 0.1.0 of insta_scrape, and running InstaScrape.user_info("foofighters") yielded:

NoMethodError: undefined method 'user_info' for InstaScrape:Class

dannyvassallo commented 7 years ago

Remove them as dependencies from your app. Instascrape installs them on its own.

dannyvassallo commented 7 years ago

Do you have a repo I can look at? @gchiaren

gchiaren commented 7 years ago

@dannyvassallo I unfortunately can't as it's private. :/ Happy to share any relevant parts of it however, anything you think that might help?

dannyvassallo commented 7 years ago

Remove those gems. Install just instascrape. Don't include capybara or poltergeist in your gemfile. Let me know how that goes.

gchiaren commented 7 years ago

That's what I did initially, and it implicitly installed the dependencies capybara (2.7.1) and poltergeist (1.9.0), as I would expect.

dannyvassallo commented 7 years ago

Do you also have phantomjs installed as a dependency?

gchiaren commented 7 years ago

Yes, I saw that was a dependency, so I installed it via homebrew before installing insta_scrape.

dannyvassallo commented 7 years ago

All of the gem dependencies required for instascrape are bundled with instascrape. You don't need to install anything extra. Remove any gems that are listed as dependencies for instascrape and then you should be fine. It sounds like you're causing version conflicts by double including the gems that instascrape requires. It works by itself. You don't need to install anything BUT instascrape.

On Jun 1, 2017 11:02 AM, "gchiaren" notifications@github.com wrote:

Yes, I saw that was a dependency, so I installed it via homebrew before installing insta_scrape.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dannyvassallo/insta_scrape/issues/18#issuecomment-305521320, or mute the thread https://github.com/notifications/unsubscribe-auth/AFk4r54BmrhARNH8hivvDsJz9h_bTXjQks5r_tKYgaJpZM4Nr7YT .

gchiaren commented 7 years ago

Ok, I have more details:

I went back and looked at the sequence of events when I first added insta_scrape to my gemfile. When I did bundle install, along with installing insta_scrape 1.1.2 it automatically included the dependencies: phantomjs 2.1.1.0, capybara 2.7.1, and poltergeist 1.9.0.

I then fired up a console session and ran my first insta_scrape command, InstaScrape.user_posts("foofighters"). It auto-installed Phantomjs then immediately went into the same error, like so:

Phantomjs does not appear to be installed in /Users/me/.phantomjs/2.1.1/darwin/bin/phantomjs, installing!
...
Successfully installed phantomjs. Yay!
Removed temporarily downloaded files.
Capybara::Poltergeist::JavascriptError: Capybara::Poltergeist::JavascriptError
    from /Users/me/.rvm/gems/ruby-2.3.1/gems/poltergeist-1.9.0/lib/capybara/poltergeist/browser.rb:351:in `command'
    from /Users/me/.rvm/gems/ruby-2.3.1/gems/poltergeist-1.9.0/lib/capybara/poltergeist/browser.rb:34:in `visit'
    from /Users/me/.rvm/gems/ruby-2.3.1/gems/poltergeist-1.9.0/lib/capybara/poltergeist/driver.rb:95:in `visit'
    from /Users/me/.rvm/gems/ruby-2.3.1/gems/capybara-2.7.1/lib/capybara/session.rb:233:in `visit'
    from /Users/me/.rvm/gems/ruby-2.3.1/gems/capybara-2.7.1/lib/capybara/dsl.rb:52:in `block (2 levels) in <module:DSL>'
    from /Users/me/.rvm/gems/ruby-2.3.1/gems/insta_scrape-1.1.2/lib/insta_scrape.rb:173:in `scrape_user_posts'
    from /Users/me/.rvm/gems/ruby-2.3.1/gems/insta_scrape-1.1.2/lib/insta_scrape.rb:48:in `user_posts'

To make sure I did not double include anything, I fired up a new rails project with a clean gemset, and ONLY added the insta_scrape gem to the gemfile. I did a bundle install, started a console session, ran an insta_scrape command, and same error :(

gchiaren commented 7 years ago

Not sure if this made any difference, but I updated the app to Rails 5.1.0. Tried again, and 1 out of 9 attempts actually worked. The fact that it's not a 100% failure rate is peculiar. Does it work 100% of the time when you try it, @dannyvassallo?

screen shot 2017-06-01 at 1 43 58 pm
beydogan commented 7 years ago

I've dig into this and found that its throwing a javascript error on Instagram side sometimes. If you catch the JavascriptError exception;

Its contents are;

#<Capybara::Poltergeist::JSErrorItem:0x000000036bf2d0 @message="Facebook Pixel Error: ReferenceError: Can't find variable: Set", @stack="Facebook Pixel Error: ReferenceError: Can't find variable: Set\n    at https://connect.facebook.net/en_US/fbevents.js:24 in fc">]

So, js_errors option should be passed to the driver initialization at: https://github.com/dannyvassallo/insta_scrape/blob/master/lib/init/poltergeist.rb#L2

dannyvassallo commented 7 years ago

@gchiaren this is in the new release 1.1.3