rubycdp / ferrum

Headless Chrome Ruby API
https://ferrum.rubycdp.com
MIT License
1.7k stars 122 forks source link

Not actually loading pages, returns string instead #305

Closed austinarchibald closed 1 year ago

austinarchibald commented 1 year ago

Several websites are returning a string that I cannot interact with. Here's one example. Any help?

browser = Ferrum::Browser.new
browser = browser.go_to("https://www.basketball-reference.com/leagues/NBA_2023.html")
=> "1A41C2B0782CD6E9E359634394008AAE"
page = Nokogiri::HTML(browser.body)
NoMethodError (undefined method `body' for "1A41C2B0782CD6E9E359634394008AAE":String)
browser.screenshot(path: "bref.png")
NoMethodError (undefined method `screenshot' for "1A41C2B0782CD6E9E359634394008AAE":String)
route commented 1 year ago

Please read the docs

austinarchibald commented 1 year ago

Can you point me in the right direction? I've been reading and researching for an hour or so and cannot find what I am doing wrong. Not having this issue with other web pages, just a couple of them.

route commented 1 year ago

browser = Ferrum::Browser.new

austinarchibald commented 1 year ago

That's exactly what I did. Still only returns a string that I cannot interact with. I'm not having an issue with most web pages. It doesn't work for this specific web page and a couple others, and I don't know why.

route commented 1 year ago

But that’s not what you exactly did assigning a string to browser variable

austinarchibald commented 1 year ago

Ok, I needed remove browser = before the go_to command. Thanks.

The website yields a human verification challenge, which is another issue. Is there a way to successfully pass captcha each time? Screenshots: bref bref2

Update: doing headless: false works on my local environment, get no security challenge. Will try that in production.