Simple 'browser.evaluate' throws a 'NoMethodError'

umairkhalid598 commented 2 years ago

Steps to reproduce:

require "ferrum"
browser = Ferrum::Browser.new
browser.go_to("http://grmdaily.com/")
browser.evaluate("pbjs.getConfig()") #throws error stack below

Error Stack:

2022-08-22T15:47:44.660Z pid=86509 tid=3xh WARN: NoMethodError: undefined method `[]' for nil:NilClass
2022-08-22T15:47:44.660Z pid=86509 tid=3xh WARN: /usr/share/rvm/gems/ruby-2.7.2/gems/ferrum-0.11/lib/ferrum/frame/runtime.rb:193:in `block in handle_response'
/usr/share/rvm/gems/ruby-2.7.2/gems/ferrum-0.11/lib/ferrum/frame/runtime.rb:220:in `block in reduce_props'
/usr/share/rvm/gems/ruby-2.7.2/gems/ferrum-0.11/lib/ferrum/frame/runtime.rb:218:in `each'
/usr/share/rvm/gems/ruby-2.7.2/gems/ferrum-0.11/lib/ferrum/frame/runtime.rb:218:in `reduce'
/usr/share/rvm/gems/ruby-2.7.2/gems/ferrum-0.11/lib/ferrum/frame/runtime.rb:218:in `reduce_props'
/usr/share/rvm/gems/ruby-2.7.2/gems/ferrum-0.11/lib/ferrum/frame/runtime.rb:192:in `handle_response'
/usr/share/rvm/gems/ruby-2.7.2/gems/ferrum-0.11/lib/ferrum/frame/runtime.rb:144:in `block in call'
/usr/share/rvm/gems/ruby-2.7.2/gems/ferrum-0.11/lib/ferrum.rb:145:in `with_attempts'
/usr/share/rvm/gems/ruby-2.7.2/gems/ferrum-0.11/lib/ferrum/frame/runtime.rb:124:in `call'
/usr/share/rvm/gems/ruby-2.7.2/gems/ferrum-0.11/lib/ferrum/frame/runtime.rb:50:in `evaluate'
/usr/share/rvm/rubies/ruby-2.7.2/lib/ruby/2.7.0/forwardable.rb:235:in `evaluate'
/usr/share/rvm/rubies/ruby-2.7.2/lib/ruby/2.7.0/forwardable.rb:235:in `evaluate'

The line pbjs.getConfig() returns an object on actual website: http://grmdaily.com/ Seems to be issue in reducing received props. Am I missing something?

ttilberg commented 2 years ago

A little more research:

The resulting object contains property getters, which when moving through the Ferrum code returns nil for properties that are function calls.

In Ferrum::Frame::Runtime#reduce_props:

          props["result"].reduce(to) do |memo, prop|
            next(memo) unless prop["enumerable"]
            yield(memo, prop["name"], prop["value"])  # <-- prop["value"] is nil for getter function props
          end

All values in props['result'] return true for #["enumerable"], so each one makes it to the yield call.

Here's what those keys and values look like:

(rdbg) props["result"][0]
{"name"=>"_debug",
 "value"=>{"type"=>"boolean", "value"=>false},
 "writable"=>true,
 "configurable"=>true,
 "enumerable"=>true,
 "isOwn"=>true}

(rdbg) props["result"][1]
{"name"=>"debug",
 "get"=>
  {"type"=>"function",
   "className"=>"Function",
   "description"=>"get debug(){return this._debug}",
   "objectId"=>"5453901104118849959.2.31"},
 "set"=>
  {"type"=>"function",
   "className"=>"Function",
   "description"=>"set debug(e){this._debug=e}",
   "objectId"=>"5453901104118849959.2.32"},
 "configurable"=>true,
 "enumerable"=>true,
 "isOwn"=>true}

props["result"].map{|prop| prop["name"]}

["_debug",
 "debug",
 "_bidderTimeout",
 "bidderTimeout",
 "_publisherDomain",
 "publisherDomain",
 "_priceGranularity",
 "priceGranularity",
 "_customPriceBucket",
 "customPriceBucket",
 "_mediaTypePriceGranularity",
 "mediaTypePriceGranularity",
 "_sendAllBids",
 "enableSendAllBids",
 "_useBidCache",
 "useBidCache",
 "_bidderSequence",
 "bidderSequence",
 "_timeoutBuffer",
 "timeoutBuffer",
 "_disableAjaxTimeout",
 "disableAjaxTimeout",
 "userSync",
 "s2sConfig"]

props["result"].map{|prop| prop["value"]}

[{"type"=>"boolean", "value"=>false},
 nil,
 {"type"=>"number", "value"=>3000, "description"=>"3000"},
 nil,
 {"type"=>"string", "value"=>"https://grmdaily.com"},
 nil,
 {"type"=>"string", "value"=>"medium"},
 nil,
 {"type"=>"object", "className"=>"Object", "description"=>"Object", "objectId"=>"5453901104118849959.2.39"},
 nil,
 {"type"=>"object", "className"=>"Object", "description"=>"Object", "objectId"=>"5453901104118849959.2.41"},
 nil,
 {"type"=>"boolean", "value"=>true},
 nil,
 {"type"=>"boolean", "value"=>false},
 nil,
 {"type"=>"string", "value"=>"random"},
 nil,
 {"type"=>"number", "value"=>400, "description"=>"400"},
 nil,
 {"type"=>"boolean", "value"=>false},
 nil,
 {"type"=>"object", "className"=>"Object", "description"=>"Object", "objectId"=>"5453901104118849959.2.54"},
 {"type"=>"object", "className"=>"Object", "description"=>"Object", "objectId"=>"5453901104118849959.2.55"}]

These nil values are being returned to the calling frame as values, where value["objectid"] raises.

            reduce_props(object_id, {}) do |memo, key, value|
              value = value["objectId"] ? handle_response(value) : value["value"]
              memo.merge(key => value)
            end

@route What should the resulting Ruby object look like when the result references a function as above? Is it as simple as adding a guard such as value && value['objectid'] to show a nil value?

ttilberg commented 2 years ago

For what it's worth, I checked how objects with getters/setters are handled in both master and released v0.11. This test passes for both. I was hoping it would raise the NoMethodError from above.

      expect(browser.evaluate(<<~JS)).to eq({"_a" => "Class with Getter"})
        new (class {
          constructor(a) {
            this._a = a;
          }
          get a() { return this._a }
        })("Class with Getter")
      JS

@umairkhalid598 In the meantime, if you're stuck, you could serialize as JSON to pass back and forth:

 JSON.parse(browser.evaluate("JSON.stringify(pbjs.getConfig())"))

umairkhalid598 commented 2 years ago

Thanks for the support and workaround solution. Love!

rubycdp / ferrum

Simple 'browser.evaluate' throws a 'NoMethodError' #283