File download

[kogeva]

Hello, please add ability for download file in Slimer. For example in PhantomJs https://github.com/ariya/phantomjs/issues/10052

[laurentj]

We should wait after PhantomJS to know what will be the API.

[DarrenCook]

Hello, please add ability for download file in Slimer. For example in PhantomJs https://github.com/ariya/phantomjs/issues/10052

That does seem to have stalled with Vitallium saying he planned a refactor, but not giving enough details on it. But also reading the comment it might not be as useful as being able to intercept the content of any downloaded resource, which is what these are talking about:

https://github.com/ariya/phantomjs/issues/10158 https://github.com/ariya/phantomjs/pull/11484

(I think you have to manually patch and compile PhantomJS to get this functionality, it is not in latest release.)

The SlimerJS docs say onResourceReceived has a "body" element, that is only available in SlimerJS:

http://docs.slimerjs.org/nightly/api/webpage.html#webpage-onresourcereceived

I just tried this (using 0.9 RC), but body is only set for the initial html request, not for any of the CSS, images, etc. Is that a bug or expected behaviour?

(My test script is below.)

Darren

---

var url="...";

var page = require('webpage').create();

page.onResourceReceived = function(response) { //console.log('Response (#' + response.id + ', stage "' + response.stage

• '"): ' + JSON.stringify(response)); console.log('Response (#' + response.id + ', stage "' + response.stage + '"): '+response.body); };

page.onResourceRequested = function(requestData, networkRequest) { console.log('Request (#' + requestData.id + '): ' + JSON.stringify(requestData)); };

page.open(url,function(){ console.log('Loaded!!!'); phantom.exit(); });

[laurentj]

This is an expected behavior. In fact I forgot to document it: you have to indicate the mime type of resources you want to retrieve into webpage.captureContent. This is an array containing regular expression that should match mime type of these resources.

webpage.captureContent = [ /css/, /png/ ]

There is this limitation I added because a webpage could contain huge resources (images etc), and so if we retrieve content of everything, it could takes many memory resources (uselessly if we don't use the body property).

But it seems that PhantomJS retrieves always everything, so probably I should remove this limitation to have full compatibility...

[DarrenCook]

Thanks Laurent. I just blogged with a full example script: http://darrendev.blogspot.jp/2013/11/saving-downloaded-files-in-slimerjs-and.html P.S. I originally wrote the regex is applied to the URL; I notice you say it is the mime-type. I was going to say URL is more intuitive... but that can be filtered on in your own function, so I don't suppose it matters which.

[Brade]

@DarrenCook Your method is appreciated, but it doesn't seem to work for files that would normally show a download prompt. I'm simply trying to get a CSV from here (click the "Export Data" link): http://www.fangraphs.com/projections.aspx?pos=all&stats=bat&type=steameru

It's extremely discouraging that both PhantomJS and SlimerJS can't handle this scenario. With Phantom you can at least see the text/csv stuff come across in onResourceReceived (I guess because webkit doesn't bring up a download prompt, it just downloads the file), but as of now there's no way to get the actual body content, as per issue #10052 mentioned in the first comment. But Slimer shows no CSV data coming across.

[kavuri]

+1 Any inputs on how to download a file with a click on a button (javascript based with a POST method)?

[laurentj]

As indicated into issue #617 , there is also a onDownloadFinished callback to implement.

Implementation proposal into Phantomjs: https://github.com/ariya/phantomjs/pull/14225