Passing variables to page for page.evaluate

[ariya]

roejame...@gmail.com commented:

There's no current easy way to pass variables from the start script to the page. Currently we can only transport from the page to the start script.

This is a problem because in order to pass, for example, a command line arg to page.evaluate, the only way to do it is to make the anonymous function in page.evaluate a string, and append the variable. It would be much better to be able to just pass the variable to the script, that way the workaround isn't needed.

I haven't thought a lot about this yet, but I imagine something like..

page.open('http://site.com', function(status) { var thisvar = 't'; // as it is now page.evaluate('function() {' + 'var thisvar = \''+thisvar+'\';' + '}'); // I propose (can have multiple comma separated args), e.g. page.send(thisvar, anothervar, ..) page.send(thisvar); page.evaluate(function() { console.log(thisvar); }); });

If that won't work, then we could try passing an object..

var obj = new Object(); obj.thisvar = thisvar; page.send(obj);

Disclaimer: This issue was migrated on 2013-03-15 from the project's former issue tracker on Google Code, Issue #132. :star2:   28 people had starred this issue at the time of migration.

[ariya]

ariya.hi...@gmail.com commented:

Alternatively, as argument to the evaluate function, e.g:

page.evaluate(function(args) { console.log(JSON.stringify(args)); });

[ariya]

roejame...@gmail.com commented:

Absolutely! Much better idea than the ones I had before. :)

Would this be good?:

page.evaluate(function() { console.log(args[0]); }, arg, ...);

It would then be accessible with an args array. Python can do infinite args easily with args. This *might be doable in C++ by va_start.

http://www.cplusplus.com/reference/clibrary/cstdarg/va_start/

[ariya]

roejame...@gmail.com commented:

Actually I don't know how well (Py)Qt would work with that method. But if it's doable it'd be pretty nice.

[ariya]

ariya.hi...@gmail.com commented:

Since we only allow primitive object (no function, no closure) as the return value for WebPage.evaluate(), we might as well make it a requirement that the object passed to the evaluate() is the same.

If people need complicated object, they can always use JSON.

[ariya]

roejame...@gmail.com commented:

 

 
Metadata Updates

• Milestone updated: Release1.3 (was: FutureRelease)

[ariya]

roejame...@gmail.com commented:

"If people need complicated object, they can always use JSON."

I completely agree. No need to make it anymore complicated than it needs to be. :)

[ariya]

roejame...@gmail.com commented:

 

 
Metadata Updates

• Owner updated: roejame...@gmail.com

[huntc]

hunt...@gmail.com commented:

In relation to this issue, I found myself having to embed the scope of any functions I wanted to call within evaluate within its scope of course i.e.

page.evaluate(function() { function a() { }

a(); });

The problem with this approach is that the code can get a little messy - especially if there are a few functions within the evaluation's body. Referencing functions in the outer scope of the evaluation fails of course.

Can anyone think of a way to express functions that are available within the evaluation, but declared in its outer scope? Would others find this a useful feature?

[ariya]

roejame...@gmail.com commented:

The only workaround seems to be converting the function to a string then passing it in, which would work with our plans.

However, notice that JSON is incompatible with functions, so it has to be turned into a string another way, then passed as a string, and converted from.

afunc = function() { console.log('afunc'); }; page.evaluate(function(afunc) { afunc = new Function(afunc); afunc(); }, String(afunc));

Hopefully that's good enough for you?

[ariya]

roejame...@gmail.com commented:

I forgot to note; if I remember right, the above approach doesn't technically work, but should get the idea across. There may be a similar way to do that.

[focusaurus]

p...@peterlyons.com commented:

Here's my hack workaround to pass data from phantom.args into the page.evaluate (coffeescript)

toRun = ->
  $('#id_username').val 'some.email@example.com'
  $('#id_password').val 'PASSWORD'
  $('#login').submit()
page.evaluate toRun.toString().replace('PASSWORD', phantom.args[0])

[huntc]

hunt...@gmail.com commented:

Thanks. I'm also wondering if passing in query parameters would be useful in this scenario now...

[ariya]

ariya.hi...@gmail.com commented:

This apparently requires a lot of effort and won't be done in time for 1.3 so sadly I have to defer it to 1.4.

Once we get issue 31 solved, it would be an easier effort.

 
Metadata Updates

• Milestone updated: Release1.4 (was: Release1.3)

[voronkovm]

voronk...@gmail.com commented:

Here is a fast-dirty hack-wrapper:

evaluateWithVars = function(page, func, vars) { var fstr = func.toString() for(var v in vars) { switch(typeof(vars[v])) { case "string": fstr = fstr.replace("VARS"+v, "'"+vars[v]+"'") break default: fstr = fstr.replace("VARS"+v, vars[v]) } } return page.evaluate(fstr) }

var page = new WebPage(); page.onConsoleMessage = function (msg) { console.log(msg); }; page.open("about:blank", function(status) { var query = "test" evaluateWithVars( page, function() { console.log(_VARS_query) }, { "query": query } ); })

Does anyone know how to bind evaluateWithVars to WebPage() or override basic WebPage::evaluate()? WebPage.prototype is not working.

[n1k0]

nperria...@gmail.com commented:

For the records, I implemented a similar method in Casper.js: https://github.com/n1k0/casperjs/blob/master/casper.js#L239

[voronkovm]

voronk...@gmail.com commented:

More accurate variant without types dependencies and VARS prefix:

evaluateWithVars = function(page, func, vars) { var fstr = func.toString() //console.log(fstr.replace("function () {", "function () {\n"+vstr)) var evalstr = fstr.replace( new RegExp("function ((.*?)) {"), "function $1 {\n" + "var vars = JSON.parse('" + JSON.stringify(vars) + "')\n" + "for (var v in vars) window[v] = vars[v]\n" + "\n" ) console.log(evalstr) return page.evaluate(evalstr) }

var page = new WebPage(); page.onConsoleMessage = function (msg) { console.log(msg); }; page.open("about:blank", function(status) { var query = "test" evaluateWithVars( page, function(sdf) { console.log(query1) }, { "query1": [1,2,3, function(a) {console.log(a)}] } ); })

[ariya]

ariya.hi...@gmail.com commented:

WebPage is just a wrapper, not a "native" JavaScript object. Thus, its prototype is unavaiable/can't be extended.

[ariya]

roejame...@gmail.com commented:

 

 
Metadata Updates

• Owner updated: ---

[ariya]

ariya.hi...@gmail.com commented:

Issue 258 has been merged into this issue.

[ariya]

neli...@gmail.com commented:

Another work-around is to use eval():

eval("function fn() { $('#id').val('" + value + "');}"); page.evaluate(fn);

[jgonera]

jgon...@gmail.com commented:

Another version. It prepends the injected vars with $ but can work without prepending too (it's similar to voronkovm's version but does not pollute window). I just find those vars easier to distinguish with $ before them ;)

It can also inject a function inside another function, so that you can have chains of such functions with injected arguments (I find it useful), e.g.:

var query = new Query(page, 'h1'); console.log(query.text);

var Query = function(page, query) { this.query = query; this.page = page; };

Query.prototype = { _evaluate: function(fn) { return this._page.evaluate(injectArgs({ query: this.query, fn: fn }, function() { var element = document.querySelector($query);

  return $fn(element);
}));

},

get text() { return this._evaluate(injectArgs({ say: 'hello' }, function(element) { console.log($say); return element.textContent; })); } };

injectArgs() implementation:

var injectArgs = function(args, fn) { var stringifyArgs = function(argsString) { var splittedArgs = argsString.split(','); for (var i=0; i<splittedArgs.length; ++i) { splittedArgs[i] = JSON.stringify(splittedArgs[i].trim()); } return splittedArgs.join(', '); };

var newFn, normalArgs, code = fn.toString();

code = code.replace(/function .((.?)) {/, function(str, p1) { var name, arg, newStr = ""; normalArgs = p1; for (name in args) { arg = args[name]; if (arg instanceof Function) { newStr += "var $" + name + " = " + arg.toString() + ";\n"; } else { newStr += "var $" + name + " = " + JSON.stringify(arg) + ";\n"; } } return newStr; }); code = code.slice(0, -1); if (normalArgs === '') { newFn = new Function(code); } else { newFn = eval('new Function(' + stringifyArgs(normalArgs) + ', code)'); } //console.log(newFn.toString()); return newFn; };

[firedfox]

wangyang...@gmail.com commented:

just ran across this post and i'm glad to share my implementation. it's more elegant because it's simpler and there's no limit to type or number of parameters.

function evaluate(page, func) { var args = [].slice.call(arguments, 2); var str = 'function() { return (' + func.toString() + ')('; for (var i = 0, l = args.length; i < l; i++) { var arg = args[i]; if (/object|string/.test(typeof arg)) { str += 'JSON.parse(\'' + JSON.stringify(arg) + '\'),'; } else { str += arg + ','; } } str = str.replace(/,$/, '); }'); return page.evaluate(str); }

[firedfox]

wangyang...@gmail.com commented:

add some comments to previous evaluate() function.

suppose the function is: function funcA(x, y) { ... } just call it like this: evaluate(page, funcA, 0, "1");

this is a simple test that i used to verify it.

var page = require('webpage').create(); page.onConsoleMessage = function(msg) { console.log(msg); }; var func = function() { console.log('hello, ' + document.title + '\n'); for (var i = 0, l = arguments.length; i < l; i++) { var arg = arguments[i]; console.log(typeof arg + ':\t' + arg); } }; page.onLoadFinished = function() { evaluate(page, func, true, 0, 'string', [0,1,2], {a:0}, function(){}, undefined, null); phantom.exit(0); }; page.open('http://www.google.com/');

[n1k0]

nperria...@gmail.com commented:

For the records I've just added arguments passing for evaluation into CasperJS, I used this little class which will parse the function args and inject the corresponding values from a passed context: https://github.com/n1k0/casperjs/commit/d8d083331dc4ac399fc803b3ce4bfc211c939d89#L0R1652

[jgonera]

jgon...@gmail.com commented:

wangyang, your solution is simple but:

a) doesn't let you pass a function as an argument b) does not have named arguments which means that you have to remember the number of each argument

nperria, could you provide some use example?

[n1k0]

nperria...@gmail.com commented:

jgon> sure, the example in the commit message should be self explanatory: https://github.com/n1k0/casperjs/commit/d8d083331dc4ac399fc803b3ce4bfc211c939d89

[jgonera]

jgon...@gmail.com commented:

Oh, I didn't scroll up, thanks ;)

[firedfox]

wangyang...@gmail.com commented:

hi jgon, with my implementation in comment 28,

a) you can pass a function as an argument:

page.onConsoleMessage = function(msg) { console.log(msg); } function f0(x, y) { console.log(typeof x); x(); console.log(typeof y); y(); } function a0() { console.log('this is function a0 as an argument'); } function a1() { console.log('this is function a1 as another one'); } evaluate(page, f0, a0, a1);

b) named arguments are also supported because you can pass objects as arguments:

page.onConsoleMessage = function(msg) { console.log(msg); } function f0(x) { console.log(typeof x); console.log(x.a0); console.log(x.a1); } var params = { a0:'i have a name', a1:'i have, too' }; evaluate(page, f0, params);

[jgonera]

jgon...@gmail.com commented:

wangyang, you're right, I didn't read the code carefully enough. Seems like a good solution to me. Will test it later in my code.

[ariya]

ariya.hi...@gmail.com commented:

Not enough time to resolve for 1.4. Postpone to 1.5.

 
Metadata Updates

• Milestone updated: Release1.5 (was: Release1.4)

[detro]

detroniz...@gmail.com commented:

Hi all,

I have been a bit away, so I'm going through all the past communication.

After having re-read this whole issue/thread, I realise that we all seem to have reached the same conclusion: JSON Objects are enough.

Above here there are many possible solution to this problem (every example has it's own pros and cons). What is to decide is where do we fit this in.

CasperJS, that is now a rolling project on it's own, has decided it's implementation. How do we proceed here?

I have a proposal in few points:

• we should provide an overloadable "evaluate" method on the "webpage" object
• we should handle var serialization/deserialization ourself
• we should handle "bad" usages gracefully

The this point is very important to me: we have to assume that some people might end up passing a non "plain" JSON object: in the evaluate function we should device a mechanism to extract the "plain" part of an object and provide that to the internal scope.

In alternative, we should discard non plain objects.

What do you think?

[ariya]

ariya.hi...@gmail.com commented:

I suggest waiting for issue 226. Workarounds to implement the variable passing will break sooner or later.

 
Metadata Updates

• Label(s) added:
  • Module-WebPage
  • Component-Logic

[ariya]

joniscoo...@googlemail.com commented:

FWIW, this is how Poltergeist is doing this (code in CoffeeScript):

evaluate: (fn, args...) -> @native.evaluate("function() { return #{this.stringifyCall(fn, args)} }")

execute: (fn, args...) -> @native.evaluate("function() { #{this.stringifyCall(fn, args)} }")

stringifyCall: (fn, args) -> if args.length == 0 "(#{fn.toString()})()" else

The JSON.stringify happens twice because the second time we are essentially

  # escaping the string.
  "(#{fn.toString()}).apply(this, JSON.parse(#{JSON.stringify(JSON.stringify(args))}))"

[theicfire]

theicf...@gmail.com commented:

@Comment 28 Works for me. Thanks!

[marksteward]

markstew...@gmail.com commented:

As touched on in #40, line 7 in #28 should read:

str += 'JSON.parse(' + JSON.stringify(JSON.stringify(arg)) + '),';

or it'll blow up on apostrophes.

[ariya]

nonp...@gmail.com commented:

28 can be simplified using the approach in #40, e.g.

function evaluate(page, func) { var args = [].slice.call(arguments, 2); var fn = "function() { return (" + func.toString() + ").apply(this, " + JSON.stringify(args) + ");}"; return page.evaluate(fn); }

It's then used like this (setting the value of field "q" in the form "f" to a specified value):

var sum = evaluate(page, function(text) { document.forms["f"].elements["q"].value = text; }, "PhantomJS");

[firedfox]

wangyang...@gmail.com commented:

as mark mentioned in #40, apostrophes can break my evaluate function in #28. thanks to his modification, a bug fix version is as follows:

function evaluate(page, func) { var args = [].slice.call(arguments, 2); var str = 'function() { return (' + func.toString() + ')('; for (var i = 0, l = args.length; i < l; i++) { var arg = args[i]; if (/object|string/.test(typeof arg)) { str += 'JSON.parse(' + JSON.stringify(JSON.stringify(arg)) + '),'; } else { str += arg + ','; } } str = str.replace(/,$/, '); }'); return page.evaluate(str); }

[firedfox]

wangyang...@gmail.com commented:

hi nonp, the function in #43 doesn't handle function and undefined correctly. please use my test case in #30 to check it.

[ariya]

ariya.hi...@gmail.com commented:

No time for 1.5. Rescheduled.

 
Metadata Updates

• Milestone updated: Release1.6 (was: Release1.5)

[ariya]

rfl109....@gmail.com commented:

W Le 20 mars 2012 01:14, phantomjs@googlecode.com a écrit :

[firedfox]

wangyang...@gmail.com commented:

just pulled a request for this issue. https://github.com/ariya/phantomjs/pull/231 it's based on #44, with some further modifications, though. usage: page.evaluate(func[, arg0, arg1, arg2, ...])

maybe it's not a 'once-for-all' solution, but it works. the only problem is that, you cannot pass an object with another variable referred by it. however, i believe 90% of the demand could be satisfied now.

[ariya]

ariya.hi...@gmail.com commented:

I'd like to have this implemented as close to the metal as possible (possible right in the layer after the JavaScript engine). However, looking at the pull request above, I see that this implementation is already very good and should cover most of the cases.

I'll do some testing and if no serious regression is found, I'll likely merge it. Thanks a lot!

[detro]

detroniz...@gmail.com commented:

I reviewed the code and I made some very small remarks. Code style stuff, so I'm sure he can still iron those out before merging.

[firedfox]

wangyang...@gmail.com commented:

hi detro,

just fixed them and updated. many thanks for your comments!

[ariya]

ariya.hi...@gmail.com commented:

Danny's implementation is landed in https://github.com/ariya/phantomjs/commit/81794f9096. Thanks, Danny!

 
Metadata Updates

• Status updated: Fixed