Use zlib instead of pako to read zip compression 4.5× faster

[jardicc]

I tested Tiff file with ZIP compression: 3000 × 4000px, 14.6MB input, 45.7MB output

Inflate with "pako" took 230-240ms. Inflate with "zlib" took 47-50ms. So it is about 4.5× faster while producing same results.

Code to test performance:

const read = require("fs/promises").readFile;
const write = require("fs/promises").writeFile;
const UTIF = require("./UTIF");

(async () => {
    const buf = await read("c:/yourPath/in.tif");

    var ifds = UTIF.decode(buf);
    UTIF.decodeImage(buf, ifds[0])
    var rgba = UTIF.toRGBA8(ifds[0]);  // Uint8Array with RGBA pixels
    const outBuf = UTIF.encodeImage(rgba, ifds[0].width, ifds[0].height);

    await write("c:/yourPath/out.tif", Buffer.from(outBuf));    
})()

Also I do recomend to wrap inflate() command into console.time() and console.timeEnd() to measure performance only for inflate. (line 185)

[photopea]

Hi, what is the "zlib" library? Where can I find the code? I do not use NPM. Also, where can I find your TIFF file, that you used for testing?

[jardicc]

That is the best part about it. You don't need npm since it is built-in inside Node.js :-) https://nodejs.org/api/zlib.html

I can't send you that exact file but maybe I could find you alternative.

[jardicc]

Test file: http://sklad.bereza.cz/00-jarda/10-JB/in.zip

This one is only 2× faster. It was generated by Photoshop. Compression level is much higher (maybe 7-9 instead of 2). Also it contains multiple strips instead of single one that contains whole image. Thefore time measument should be as wrapper around the for-loop.

[photopea]

I dont use Node.js either :( In our case, we run UTIF.js in a web browser, so we need the JS files.

[jardicc]

I think you can use only node.exe (if you are on windows) all other files are not necessary and then you run CLI. Save test code into e.g. resave.js so you would run within command line node.exe resave.js (also modify image file paths)

[photopea]

I think the "zlib" in Node.js refers to the "zlib" compression library written by Mark Adler in C: https://en.wikipedia.org/wiki/Zlib It was written back in the 90s and it never uses more than 32 kB of RAM.

Much faster decompression is possible, if you allow yourself to use more than 32 kB of RAM. There exist C libraries which are 3x - 5x faster than Zlib.

Nonetheless, these are C libraries and UTIF.js is a Javascript library. We can not distribute UTIF.js with some EXE programs, or our own web browser, etc.

[jardicc]

@photopea 1) you can set memory levels to increase it: https://nodejs.org/api/zlib.html#for-zlib-based-streams I don't know what is the fastest library in the world but this one is faster everytime than UTIF default implementation

2) If you look here https://www.npmjs.com/package/utif there is 1 000 000+ downloads every week by people who will run it in Node.js anyway. No additional C library is needed this is already included in NodeJS. Your code has require("pako") and as far I know this will work only in NodeJS. And also I added contitions into source code to recognize whether it is running in web browser or NodeJS. So it is optional.

[photopea]

At Photopea.com , we run UTIF.js in a browser. You can create a Node.js version of UTIF.js if you want. One user suggested me to add require("pako"), I dont know what it means.

[jardicc]

Yes I know. But the problems with separated versions are that in this repo: 1) no changelog 2) no commit explanations 3) no tests 4) everything is in one file 5) no versioning

So that means that I would manually change Utif.js everytime you upload new file but I have no guarantee that your changes won't break my changes so I would need to carefully check all changes everytime. I don't blame you... providing library is your good will. And like capabilities of this library a lot. But working with it and keeping separated version that is updated is time demanding.

[photopea]

I have made the deflate decompression about 25% faster. I have my own implementation of a deflate, which can make it even more fast, but it would add around 16 kB of JS (but we would remove the pako.js dependency).

Also, around a half of UTIF.js code is related to RAW images and is not needed for TIFF decoding.