HTTP get binary files

[GoogleCodeExporter]

I try:
new File('1.ico')
    .open('wb')
    .write(
        new HTTP.ClientRequest('http://code.google.com/favicon.ico').send(true).data);

thats provide different (corrupted) result with:
new (require('process').Process)().exec('wget 
http://code.google.com/favicon.ico -O 2.ico');

you can try:
system.stdout([
    '1 size: ', Util.serialize(new File('1.ico').stat()), '\n',
    '2 size: ', Util.serialize(new File('2.ico').stat()), '\n'
].join(''));

Original issue reported on code.google.com by mr.ve...@gmail.com on 17 Jan 2010 at 1:20

[GoogleCodeExporter]

We need a working Binary datatype before bugs like this can be fixed.

Original comment by ondrej.zara on 17 Jan 2010 at 3:39

• Changed state: Accepted

[GoogleCodeExporter]

Is this somehow solved or does a workaround exist?

I hacked/solved it by changing the "send" function inside http.js to include 
something like:

//--------------------------------------
    //----- CHANGE
    var binary = [];
    //------

    do {
        var part = s.receive(1024, true);

        for (var i=0;i<part.length;i++) {
            received += String.fromCharCode(part[i]);
            // ----- CHANGE
            binary.push(part[i]); 
            // ------
        }
    } while (part.length > 0);

        s.close();
    received = Util.utf8encode(received);

    return this._handleResponse(received, binary, follow) // CHANGE - passing "binary" so its a memeber of HTTP.ClientResponse
//------------------------------------------------

so you can eventually do:

var bnry = new 
HTTP.ClientRequest('http://code.google.com/favicon.ico').send(true).binary;

***** HOWEVER you would also need to remove the HTTP headers from the 
binary-response by applying something like the following and only then save the 
return value to a file:

  function removeHeaders(bnry) { // {i know its ugly}
    var noheaders = [];
    var eoh = false;

    for (var i = 0; i < binary.length; i++) {
        if (eoh){
        noheaders.push(binary[i]);
        }

        if (binary[i] == 13 && binary[i + 1] == 10 && binary[i + 2] == 13 && binary[i + 3] == 10) {
        response.write('end of headers');
        i = i + 3;
        eoh = true;
        }
    }

    return noheaders;
    }

Original comment by ayalgel...@gmail.com on 12 Sep 2010 at 4:16

[GoogleCodeExporter]

This workaround does not look bad at all. One can detect where the headers end 
in ClientResponse constructor - and remove adequate bytes form the beginning of 
"binary" argument...

Generally, a completly correct solution requires rewriting the whole http.js by 
scanning the headers first and deciding text/binary mode based on response's 
Content-type. Unfortunately, I am now out of time to implement this, but I at 
least assign this issue to myself :)

Original comment by ondrej.zara on 12 Sep 2010 at 4:27

[GoogleCodeExporter]

HTTP module now retrieves the data in binary mode (as instances of a Buffer).

Original comment by ondrej.zara on 9 Dec 2010 at 8:03

• Changed state: Fixed