tinify / tinify-nodejs

Node.js client for the Tinify API.
https://tinypng.com/developers
MIT License
421 stars 74 forks source link

Images occasionally cut off without an error thrown. #25

Closed Frustrated-Programmer closed 4 years ago

Frustrated-Programmer commented 4 years ago

So almost about every 10 images 1 of them gets cut off vertically. I get NO error, it just continues with the next card. As far as I know, there is no consistency. As it sometimes doesn't happen for another 20 imgs and others just 3 images in a row. And I can delete the image and restart my code, and it will tinify it again but this time not have the cut-off part. But both times count towards my tinied images. Tinified: Originals:

Edit I just counted, I have 124 images messed up this way, and 385 images tinified. (Some of the 385 was re-doing the ones that were messed up.)

My code:

const tinify = require("tinify");
const fs = require("fs");
tinify.key = "My_Key_Here";

function upgradeFile(path, imgPath, fileName){
    return new Promise(function(cb, rj){
        console.log("Upgrading file: " + path);
        if(alreadyTiny(imgPath, fileName)){
            console.log("Already upgraded... Returning.");
            cb();
        }
        else{
            let spl = fileName.split(".");
            let exe = spl[spl.length - 1];
            if(exe !== "png" && exe !== "jpg" || exe !== "jpeg"){
                console.error(`Invalid file type: ${exe}, skipping`);
                cb();
            }
            else{
                const source = tinify.fromFile(path);
                let dir = "./../tinyfied/" + imgPath + "/";
                if(!fs.existsSync(dir)){
                    fs.mkdirSync(dir);
                }
                source.toFile(dir + fileName).then(cb);
            }
        }
    });
}
function alreadyTiny(imgPath, fileName){
    return fs.existsSync("./../tinyfied/" + imgPath + "/" + fileName);
}
function readDir(path, imgPath){
    return new Promise(function(cb, rj){
        console.log(`reading path: ${path}`);
        fs.readdir(path, {withFileTypes: true}, async function(err, files){
            for(let i = 0; i < files.length; i++){
                let dirent = files[i];
                let isDir = dirent.isDirectory();
                console.log(`checking ${isDir ? "dir" : "file"} ${i}/${files.length}: ${path + "/" + dirent.name}`);
                if(isDir) await readDir(path + "/" + dirent.name, dirent.name + "/").catch(rj);
                else await upgradeFile(path + "/" + dirent.name, imgPath, dirent.name).catch(rj);
            }
            cb();
        });
    });
}
readDir("./../images", "/");
emielvanlankveld commented 4 years ago

Hi! That is odd, it looks like the output image is somehow corrupted, either during the compression process or when downloading it. The images that are cut off can't be compressed again (they can't be decoded).

Can you still reproduce the issue? I compressed your original images with your exact script and I can't reproduce it (perhaps because I only have 3). Once an image is compressed the result gets cached so any further compression attempts of the same image will not incur another compression. The fact that you say that compressing them again counts towards your compressed images again makes me think the images may actually get corrupted during the compression process.

If you can send a link to all the images you compressed to support@tinify.com then I can try and reproduce it with the entire sample size. Otherwise if you can reproduce it yourself please email the output link to the same address (before it expires after ~8 hours) so we can check it out that way. If you send us your account details we can give you some credits to allow you to keep compressing within your limit to test it out.

Frustrated-Programmer commented 4 years ago

Email sent, though I wanted this part to be public:

Somethings not right about the cards counting when I had to fix them. I did the math I was at 491 cards this morning before I started trying to re-produce the issue. I needed a total of 439 cards tinied, although I edited some images and re-tinied them again (I don't have a count tho, prob 15-20).

So I'm assuming 460 images I used legit but instead have 491 used. I had over 130(counted from my recycling bin) images broken that I had to re-tiny. So I honestly at this point I don't know what happened to 31 images. I had assumed it was the cards when I re-uploaded them to be fixed again but that math doesn't add up.

I honestly don't care what happened to the 31 count, maybe I forgot something I did. But I wanted to let ya know it's prob NOT what I said in the github issue:

I can delete the image and restart my code, and it will tinify it again but this time not have the cut-off part. But both times count towards my tinied images. 

emielvanlankveld commented 4 years ago

Thank you for looking into it, that points toward the images being compressed correctly on the server but something going wrong during the download step.

Frustrated-Programmer commented 4 years ago

Tried reproducing on a different day, couldn't replicate.

Sorry...