Question: Passing object references

[shellscape]

I've got a general question about working with the module and passing object references from an environment like Node, to evaluated scripts and/or scripts running on a page. The specific use case is for developing a module that leverages mocha and headless chrome for CI automation testing. In that case, the Mocha script running in the browser needs to be provided a stdout interface.

There are solutions for that particular use case, such as using DOM events to send and listen for a 'stdout' event, coupled with https://github.com/kumavis/browser-stdout. But I thought it would be particularly useful if passing references was possible. Is that possible now, and if not, what needs to happen to make that possible?

Component	Version
Operating system	Mac OS 10.12.5
Node.js	7.8.0
Chrome/Chromium/...	Version 59.0.3071.115
chrome-remote-interface	latest

[cyrus-and]

AFAIK it's not possible, only JSON object can be passed, as of why I'm not the authoritative source but I think that the problem is that Node.js and the browser are two distinct JavaScript execution environments. You might have more luck if you ask this in the Google Group.

Speaking of general purpose workarounds, I think this problem can be split in two directions:

1. from Node.js to the browser;
2. from the browser to Node.js.

Case 1 is easy, you can just call Runtime.evaluate({expression: '/* browser code */'}) whenever you want, e.g., upon a Node.js-related event.

Case 2 is more complex because the interaction must be started (or at least expected) by Node.js, for example, in the browser:

function doNodeAction(param) {
    if (window._nodeAction) {
        _nodeAction(param);
    }
}

While in Node.js:

async function prepareNodeAction(action) {
    // wait for browser
    const param = await Runtime.evaluate({
        expression: `new Promise((fulfill, reject) => {
            _nodeAction = fulfill;
        }).then(() => {
            _nodeAction = null;
        })`,
        awaitPromise: true
    });
    // actually perform the action
    action(param);
}

In this way you can even implement a very limited browser event handler. I do really hope there are better ways to accomplish that...

[shellscape]

That's really a pretty solid way to go about it in the near term though. I could in theory create a stub of stdout that simply piped out to node in much the same way. Thanks for giving that some consideration.

[cyrus-and]

You're welcome!

The main problem is that you have to re-prepare the Promise after every browser invocation of doNodeAction(param); a while (...) loop would probably do as long as you await prepareNodeAction inside it.

Another problem is that you should somewhat buffer the browser invocations of doNodeAction(param) when window._nodeAction is null.

This is an interesting problem, if you come out with a neater solution or simply want to link to another discussion thread feel free to post it here.

[shellscape]

Tangential thought: It really is a shame that there's no built-in event bus for clients. That'd make so much sense.

[cyrus-and]

That'd make so much sense.

True but I see why it has not been implemented yet, at the very least the browser's JavaScript environment should be aware of being inspected and thus provide an API to interact with it, something like window.devtools.postMessage(...) which would trigger some CDP event.

Another hackish solution would be using the Runtime.consoleAPICalled and send events to Node.js using the Console API:

console.log({type: 'event', data: {...}});

Then in Node.js:

await Runtime.enable();
await Runtime.consoleAPICalled(({type, args}) => {
    if (type !== 'log' || args.length != 1 || args[0].type !== 'object' || /*...*/) {
        // skip other Console API calls based on the expected arguments
        return;
    }
    const object = args[0].preview.properties;
    // ...
});

[shellscape]

And that's exactly what I ended up doing last night - piggy backing on the Console API. Thanks very much for being so active on this repo and for humoring so many questions about the protocol. It's incredibly helpful.

[shellscape]

@cyrus-and I found an interesting quirk with the Runtime.consoleAPICalled method: it would appear that CDP is performing some kind of cropping of text values.

For instance:

let json = JSON.stringify({"suites":1,"tests":1,"passes":1,"pending":0,"failures":0,"start":"2017-08-01T02:35:50.914Z","end":"2017-08-01T02:35:50.996Z","duration":82});
console.log({ json });`

Ends up looking like this:

{ type: 'object',
  description: 'Object',
  overflow: false,
  properties: 
   [ { name: 'json',
       type: 'string',
       value: '{"suites":1,"tests":1,"passes":1,"pending":0,"fail…","end":"2017-08-01T02:35:50.996Z","duration":82}' } ] }

Notice the "fail…" bit in the resulting object. JSON.parse will naturally fail, as the string has been made invalid JSON for some reason. It's a bizarre quirk and I can't find any info regarding it.

Any ideas?

[cyrus-and]

I notice this behavior (i.e., string members of objects may be shortened for the sake of preview) in DevTools itself but not via the CDP, I tried with the following and I'm not able to reproduce it:

const CDP = require('chrome-remote-interface');

CDP(async (client) => {
    const {Page, Runtime} = client;
    Runtime.consoleAPICalled(({args}) => {
        console.log(args[0].value);
    });
    try {
        await Page.enable();
        await Page.navigate({url: 'http://example.com'});
        await Page.loadEventFired();
        await Runtime.enable();
        await Runtime.evaluate({
            expression: `let json = JSON.stringify({"suites":1,"tests":1,"passes":1,"pending":0,"failures":0,"start":"2017-08-01T02:35:50.914Z","end":"2017-08-01T02:35:50.996Z","duration":82});
                         console.log(json);`
        });
    } catch (err) {
        console.error(err);
    } finally {
        client.close();
    }
}).on('error', (err) => {
    console.error(err);
});

[shellscape]

In my case the console.log is coming from a script block within a simple html file loaded by Page.navigate. eg (truncated for brevity):

<!doctype>
<html>
<body>
  <script>
    console.log(...);
  </script>
</body>
</html>

await Page.navigate({ url: 'file://' + path.join(__dirname, 'test.html') });

I wasn't able to find a consistent pattern to the string length that would trigger that quirk, but I was able to reproduce that quirk consistently. I don't think the issue lies with this module. At the very least we have it documented here as a potential issue.

My workaround, for the purposes of the event bus simulation, was to use localStorage in conjunction with DOMStorage.domStorageItemUpdated since there's practically no limit on the size of a thing you can set there. I set a localStorage item in the client/html/script localStorage.setItem('bus', ''); to init the value, and reset the value every time the event bus needed to "emit" back to the node app. The value for each set had to be stringified and then parsed in the node app, but it's very effective, and it's significantly simpler to work with than the Console solution - the main reason being there's no need for parsing a complex item tree in the preview property, and no need for a module like https://github.com/ironSource/node-chrome-unmirror to facilitate that.

[cyrus-and]

OK I made a mistake in my previous snippet, I'm not returning an object but just a string. If I return an object then I'm able to reproduce it.

Well, we have accidentally found a solution then: just pass the JSON string directly instead of {json: '{...}'}. In this case no alteration is performed and you can easily get the value with:

Runtime.consoleAPICalled(({args}) => {
    const object = JSON.parse(args[0].value);
    console.log(object);
});

My workaround, for the purposes of the event bus simulation, ...

That's a smart solution, thanks for sharing.