jimp-dev / jimp

An image processing library written entirely in JavaScript for Node, with zero external or native dependencies.
http://jimp-dev.github.io/jimp/
MIT License
13.94k stars 762 forks source link

Could not find MIME for buffer (null) #643

Closed Allaphy closed 5 years ago

Allaphy commented 5 years ago

I have this problem "Could not find MIME for buffer (null)" and i don't know how to fix it. I believe that the problem is in jimp.read. that is the part of the code where i read the images:

var imgs = [pa1, pa2]; // pa1 and pa2 are images URLs. var jimps = []; for (var i = 0; i < imgs.length; i++) { jimps.push(await jimp.read(imgs[i])); }

'imgs[]' are the URLs of the images to read.

hipstersmoothie commented 5 years ago

I’m probably going to need a link to one of your images. I’m pretty sure image urls are working so I’ll need a test image On Wed, Oct 10, 2018 at 8:44 PM Allaphy notifications@github.com wrote:

I have this problem "Could not find MIME for buffer (null)" and i don't know how to fix it. I believe that the problem is in jimp.read. that is the part of the code where i read the images:

var imgs = [pa1, pa2]; // pa1 and pa2 are images URLs. var jimps = [];

for (var i = 0; i < imgs.length; i++) { jimps.push(await jimp.read(imgs[i])); }

'imgs[]' are the URLs of the images to read.

  • Jimp Version: 0.5.3
  • Operating System: Windows 8.1
  • Node version: 8.11.4

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/oliver-moran/jimp/issues/643, or mute the thread https://github.com/notifications/unsubscribe-auth/ABIyBOBpUO5i3bT431rTurwMkJ8aolZsks5ujr63gaJpZM4XWuiO .

DavidOndrus commented 5 years ago

I experience it too but with files saved locally. I've been using old version (0.2.28) for a long time, then on 2nd October I made an update to 0.5.3. It never happened to me before. It's not reproducible when someone sends you a file because in my situation, for example only 2 of 5 tries crash. When it crashes and then I try the same file, after some time it works. It doesn't make sense.

Right now I tried the file I'm attaching, on 2nd try it worked. 636629e3-dcee-4a34-9c07-99f7aadfbd99.zip

This was successfully printed after file upload:

node_1   | debug: UploadedFileMetadata {
node_1   |   fd: '/app/.tmp/uploads/636629e3-dcee-4a34-9c07-99f7aadfbd99.jpg',
node_1   |   size: 305735,
node_1   |   type: 'image/jpeg',
node_1   |   filename: '1539245498891.jpg',
node_1   |   status: 'finished',
node_1   |   field: 'photo',
node_1   |   extra: undefined }

Then this error:

node_1   | debug: Error: Could not find MIME for Buffer <null>
node_1   |     at Jimp.parseBitmap (/app/node_modules/@jimp/core/dist/utils/image-bitmap.js:108:15)
node_1   |     at Jimp.parseBitmap (/app/node_modules/@jimp/core/dist/index.js:498:32)
node_1   |     at /app/node_modules/@jimp/core/dist/index.js:440:15
node_1   |     at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:511:3)
Allaphy commented 5 years ago

the urls aren't aways the same, are urls from discord's avatars (message.author.displayAvatarURL), for example: https://cdn.discordapp.com/avatars/472158887966015508/a64800726ff83aedd350092730484a1e.jpg?size=2048

with me, just some times crashes too, some times works fine, and some times it wont.

hipstersmoothie commented 5 years ago

The following runs without any errors. I'll do a run with a 1000 but I'm not sure this will reproduce it.

Are you able to reproduce this outside the discord server?

const Jimp = require('jimp');

test();

async function test() {
  for (let i = 0; i < 100; i++) {
    const image = await Jimp.read(
      'https://cdn.discordapp.com/avatars/472158887966015508/a64800726ff83aedd350092730484a1e.jpg?size=2048'
    );

    const buffer = await image.getBufferAsync(Jimp.AUTO);
    console.log(i, buffer);
  }
}
hipstersmoothie commented 5 years ago

I feel like maybe the request is failing? you said it had to do with upload. maybe sometimes you do it too soon and the resource isnt quite there yet?

DavidOndrus commented 5 years ago

I'm able to reproduce it with local files, without network so it's not an issue with online files, it's a general Jimp.read issue.

hipstersmoothie commented 5 years ago

You you give me a script like the one i provided above? Currently 700 requests and no errors

DavidOndrus commented 5 years ago

Download my file and use your script with Jimp.read('/app/.tmp/uploads/636629e3-dcee-4a34-9c07-99f7aadfbd99.jpg')

hipstersmoothie commented 5 years ago

Still no luck. How often does it happen for you?

screen shot 2018-10-11 at 11 35 57 pm
hipstersmoothie commented 5 years ago

Can you put together a repo that demonstrates the issue? I so far have not been able to reproduce. I am also not on windows

DavidOndrus commented 5 years ago

I would not try to reproduce it with tests because even if one of many tries fails, you are not able to repeat the same process with the same result. I mean, if one line printed "null" in the test output above, it's not very useful information I think. I've searched this repo and found a line with the error string we've posted so I would go for investigating the stack trace. If similar errors occur, it's often a problem with thread (asynchronous operations) because it's reproducible e.g. only once per 10 tries.

hipstersmoothie commented 5 years ago

The thing is I can't get it to go wrong even once so i can't even begin to debug the error. the stack trace tells me whats going wrong but I can't write code to handle a case I can't reproduce at least a little.

You said you were able to reproduce the issue locally. Is this not the case?

I'm able to reproduce it with local files, without network so it's not an issue with online files, it's a general Jimp.read issue.

Your statement here would lead me to believe you can at lest reproduce it 1 in 10 times.

I'm not sure how to go forward. Could you just try/catch?

@Allaphy can you point me towards some code that produces the issue?

Allaphy commented 5 years ago

send the buffer to cosole.log i know how to do it, but when i do this, it wont send the message on discord.

should i do something with this buffer ?

Allaphy commented 5 years ago

var p1 = hguData[sort1].user; // String var pa1 = hguData[sort1].avatar; // avatar URL (image) var p2 = hguData[sort2].user; // String var pa2 = hguData[sort2].avatar; // avatar URL (image) var data; var imgs = [pa1, pa2]; var jimps = [];

for (var i = 0; i < imgs.length; i++) { jimps.push(await jimp.read(imgs[i])); }

Promise.all(jimps).then(function() { return Promise.all(jimps); }).then(function(d) { data = d; return jimp.loadFont(jimp.FONT_SANS_16_WHITE); }).then(async function(font) { msg.react("⏳"); data[0].resize(75, 75); data[1].resize(75, 75);

let nova = new jimp(255, 125, '#000000', (err, image) => {});
nova.composite(data[0], 5, 25);
nova.composite(data[1], 130, 25);
nova.print(font, 32, 2, "P1");
nova.print(font, 155, 2, "P2");
nova.print(font, 5, 103, p1);
nova.print(font, 130, 103, p2);
nova.write(`./images/hg/hgtest.png`);
await msg.channel.send({files: [`./images/hg/hgtest.png`]});
hipstersmoothie commented 5 years ago

@Allaphy That code wasn't really doing what you wanted to accomplish. The following does what you intended. I also changed:

@DavidOndrus if you provide me an example like this I can try to fix it as well.

const Jimp = require('jimp');

const hguData = [
  {
    user: 'Dan',
    avatar: 'https://avatars0.githubusercontent.com/u/1192452?s=88&v=4'
  },
  {
    user: 'Mary',
    avatar: 'https://avatars3.githubusercontent.com/u/179534?s=460&v=4'
  }
];

const sort1 = 0;
const sort2 = 1;

test();

async function test() {
  const PICTURE_WIDTH = 75;

  const { user: user1, avatar: avatar1 } = hguData[sort1];
  const { user: user2, avatar: avatar2 } = hguData[sort2];

  const images = [avatar1, avatar2];
  const jimps = await Promise.all(images.map(img => Jimp.read(img)));

  jimps.map(img => img.resize(PICTURE_WIDTH, PICTURE_WIDTH));

  console.log('⏳');

  const nova = await Jimp.create(250, 125, '#000000');
  const AVATAR1_START = 25;
  const AVATAR2_START = 150;

  nova.composite(jimps[0], AVATAR1_START, 25);
  nova.composite(jimps[1], AVATAR2_START, 25);

  const font = await Jimp.loadFont(Jimp.FONT_SANS_16_WHITE);

  nova.print(font, AVATAR1_START, 2, { text: 'P1', alignmentX: Jimp.HORIZONTAL_ALIGN_CENTER }, PICTURE_WIDTH);
  nova.print(font, AVATAR2_START, 2, { text: 'P2', alignmentX: Jimp.HORIZONTAL_ALIGN_CENTER }, PICTURE_WIDTH);
  nova.print(font, AVATAR1_START, 103, { text: user1, alignmentX: Jimp.HORIZONTAL_ALIGN_CENTER }, PICTURE_WIDTH);
  nova.print(font, AVATAR2_START, 103, { text: user2, alignmentX: Jimp.HORIZONTAL_ALIGN_CENTER }, PICTURE_WIDTH);

  await nova.writeAsync(`./hgtest.png`);

  console.log('all done!');
  // await msg.channel.send({ files: [`./images/hg/hgtest.png`] });
}

hgtest

DavidOndrus commented 5 years ago

My code is really simple in this case - it crashes on the first Jimp line (Jimp.read) - in the case where it happened, I also had sails.log.debug before Jimp.read printing uploadedFiles[0] object a "fd" field was properly there:

file.upload(function (err, uploadedFiles) {
      if (err) {
        sails.log.debug(err);
        return res.negotiate(err);
      }

      if (uploadedFiles.length === 0) {
        sails.log.debug('No file was uploaded');
        return res.serverError({error: 'No file was uploaded'})
      }

      const fd = uploadedFiles[0].fd;
      Jimp.read(fd)
        .then(image => {
          const mimeType = image.getMIME();
          return image
            .resize(Jimp.AUTO, 500, Jimp.RESIZE_NEAREST_NEIGHBOR)
            .getBufferAsync(mimeType)
            .then(buffer => {
              //do stuff with buffer
            })
        })
    });
hipstersmoothie commented 5 years ago

Have you been able to reproduce the error outside of sails? As in a just a simple script no server. I still have not been able to reproduce your issue. Currently running this following with your file with no errors so far:

const Jimp = require('jimp');

test();

async function test() {
  while (true) {
    try {
      const image = await Jimp.read('./exampleImages/636629e3-dcee-4a34-9c07-99f7aadfbd99.jpg');
      await image.resize(Jimp.AUTO, 500, Jimp.RESIZE_NEAREST_NEIGHBOR).writeAsync('test.png');
      console.log('wrote image');
    } catch (error) {
      console.log('CAUGHT ERROR!');
      console.log(error);
    }
  }
}
DavidOndrus commented 5 years ago

I didn't try outside of sails because Sails is a must for me in the project, it must work everywhere. You said "I still have now been able to reproduce your issue" - so were you ? Or is it a mistake ?

hipstersmoothie commented 5 years ago

Oops just a typo. Still cannot reproduce. If you cannot reproduce outside of sails then this must be a problem with the interaction between the two.

I have never used sails so i really need you to put together a repo with as little code as possible that showcases the problem. Otherwise I will not be able to fix this at all.

hipstersmoothie commented 5 years ago

To me it seems like this has to be async issues with your code. You've said:

This leads me to believe that you are not properly awaiting a promise at some point resulting in the image sometimes being there. jimp is just using fs.readFile to read the file on your local machine. If readFile returns a null buffer that generally means that the file is not yet on your system.

Again without a repo showcasing the issue I cannot debug this issue only theorize.

hipstersmoothie commented 5 years ago

@DavidOndrus do you happen to be using webpack?

DavidOndrus commented 5 years ago

No, I don't use webpack. I agree that's a strange issue, I will probably need to resolve this by myself.

Allaphy commented 5 years ago

thank you man, the code is really better now. It still doing the MIME's error (not so mutch like before), but for my case, i could unless send a message to repeat the process so its fine now.

If, eventually this problem get a solution, i'll find out, but for now its not a big problem any more.

hipstersmoothie commented 5 years ago

I'm gonna close this. If someone can put together an example that errors consistently and share it with me feel free to open an issue!

darrensw777 commented 5 years ago

Same problem here guys, this fails every time (is it because there is no file extension? If so is there a workaround?):

const image_url = 'https://images.asos-media.com/products/river-island-maxi-dress-with-tie-neck-in-snake-print/10985965-1-snakeprint';
Jimp.read(image_url)
                    .then(image => image.write(`/path/to/images/${lastInsertId}.jpg`))
                    .catch(err => {
                        console.error(err);
                    });

This is the error I get:

{ Error: Could not find MIME for Buffer <null>
18:31:36 0|server   |     at Jimp.parseBitmap (/home/myAccount/node_modules/@jimp/core/dist/utils/image-bitmap.js:108:15)
18:31:36 0|server   |     at Jimp.parseBitmap (/home/myAccount/node_modules/@jimp/core/dist/index.js:498:32)
18:31:36 0|server   |     at /home/myAccount/node_modules/@jimp/core/dist/index.js:440:15
18:31:36 0|server   |     at /home/myAccount/node_modules/@jimp/core/dist/index.js:168:14
18:31:36 0|server   |     at /home/myAccount/node_modules/@jimp/core/dist/request.js:56:9
18:31:36 0|server   |     at IncomingMessage.<anonymous> (/home/myAccount/node_modules/phin/lib/phin.compiled.js:1:2162)
18:31:36 0|server   |     at emitNone (events.js:91:20)
18:31:36 0|server   |     at IncomingMessage.emit (events.js:185:7)
18:31:36 0|server   |     at endReadableNT (_stream_readable.js:974:12)
18:31:36 0|server   |     at _combinedTickCallback (internal/process/next_tick.js:74:11) methodName: 'constructor' }
hipstersmoothie commented 5 years ago

Write requires a mime type in pretty sure. Nowhere near a computer so u can’t link

hipstersmoothie commented 5 years ago

Jimp.auto as the second param

darrensw777 commented 5 years ago

No dice, exactly same error, tried with Jimp.auto and Jimp.AUTO

hipstersmoothie commented 5 years ago

@darrensw777 The following code works for me.

const Jimp = require("jimp");

test();

async function test() {
  const image = await Jimp.read(
    "https://images.asos-media.com/products/river-island-maxi-dress-with-tie-neck-in-snake-print/10985965-1-snakeprint"
  );

  await image.writeAsync("test.jpg", Jimp.AUTO);
}

but your code also just worked when i ran it....

const Jimp = require("jimp");

const image_url =
  "https://images.asos-media.com/products/river-island-maxi-dress-with-tie-neck-in-snake-print/10985965-1-snakeprint";

Jimp.read(image_url)
  .then(image => image.write(`test.jpg`))
  .catch(err => {
    console.error(err);
  });
hipstersmoothie commented 5 years ago

write doesn't return a promise so you should actually use writeAsync

const Jimp = require("jimp");

const image_url =
  "https://images.asos-media.com/products/river-island-maxi-dress-with-tie-neck-in-snake-print/10985965-1-snakeprint";

Jimp.read(image_url)
  .then(image => image.writeAsync(`test.jpg`))
  .catch(err => {
    console.error(err);
  });

what version are you using?

darrensw777 commented 5 years ago

Definitely not working still for me with that image url.

Tried:

const Jimp = require('jimp');

const image_url =
    'https://images.asos-media.com/products/river-island-maxi-dress-with-tie-neck-in-snake-print/10985965-1-snakeprint';

Jimp.read(image_url)
    .then(image => image.writeAsync(`test.jpg`))
    .catch(err => {
        console.error(err);
    });

Get the following error:

0|server   | { Error: Could not find MIME for Buffer <null>
0|server   |     at Jimp.parseBitmap (/home/myAccount/node_modules/@jimp/core/dist/utils/image-bitmap.js:108:15)
0|server   |     at Jimp.parseBitmap (/home/myAccount/node_modules/@jimp/core/dist/index.js:498:32)
0|server   |     at /home/myAccount/node_modules/@jimp/core/dist/index.js:440:15
0|server   |     at /home/myAccount/node_modules/@jimp/core/dist/index.js:168:14
0|server   |     at /home/myAccount/node_modules/@jimp/core/dist/request.js:56:9
0|server   |     at IncomingMessage.<anonymous> (/home/myAccount/node_modules/phin/lib/phin.compiled.js:1:2162)
0|server   |     at emitNone (events.js:91:20)
0|server   |     at IncomingMessage.emit (events.js:185:7)
0|server   |     at endReadableNT (_stream_readable.js:974:12)
0|server   |     at _combinedTickCallback (internal/process/next_tick.js:74:11) methodName: 'constructor' }

I only installed this yesterday so presumably the latest version. Installed using NPM

Using with NodeJS v. 11.4.0

OS: Linux 3.10.0-862.14.4.el7.x86_64 x86_64

How can I use and have it not throw a script blocking error? I have a fallback if this fails but it will not fail silently.

hipstersmoothie commented 5 years ago

Since i cannot reproduce this locally an example repo is gonna be needed. Also this has nothing to do with the issue we are using to talk about it so another issue would be cool as well.

How can I use and have it not throw a script blocking error? I have a fallback if this fails but it will not fail silently.

aren't you catching and printing the error? the program should continue right?

as I said without a reproducible example I will be little help other than just speculation

RMCoder198 commented 5 years ago

I solved this issue by using promise method as suggested in docs instead of using callback method. Another reason i am getting this error is because of having too many images on destination directory. @hipstersmoothie you might try to reproduce it by having 50+ images inside destination directory.

mayankbytes commented 5 years ago

I am also facing same issue, when I gave image path, like following let image = await Jimp.read("./assets/not-button/1.jpg");

But when import my image and then pass it to JIMP.read(), it works, not sure what the issue is

import notButton1 from './assets/not-button/1.jpg'; let image = await Jimp.read("notButton1");

What is difference in both approach and why it is not working in first one? Any leads are appreciated.

@hipstersmoothie

FridayJew commented 5 years ago

The same error, the files are received from the Internet and saved on the disk, but are not read by the jimp module. There are no visible problems with them.

codan84 commented 5 years ago

I get this error without fail every time I read certain images from a URL.

My app does a bing image search and then tries to read a random image. some of them read fine, some of them throw an error as described in this issue.

The ones that throw an error, throw it always.

One of the offending images: https://s-media-cache-ak0.pinimg.com/736x/c9/8f/e1/c98fe17dc7de72bb29c34a0c79ef5762.jpg

Here is a repo that reproduces the issue (I am running this on node v10.16.2).

I can provide many more image URLs that always fail.

Example output:

{ Error: Could not find MIME for Buffer <null>
    at Jimp.parseBitmap (/Users/gruszd01/workspace/jimp-bug/node_modules/@jimp/core/dist/utils/image-bitmap.js:108:15)
    at Jimp.parseBitmap (/Users/gruszd01/workspace/jimp-bug/node_modules/@jimp/core/dist/index.js:498:32)
    at /Users/gruszd01/workspace/jimp-bug/node_modules/@jimp/core/dist/index.js:440:15
    at /Users/gruszd01/workspace/jimp-bug/node_modules/@jimp/core/dist/index.js:168:14
    at /Users/gruszd01/workspace/jimp-bug/node_modules/@jimp/core/dist/request.js:56:9
    at IncomingMessage.<anonymous> (/Users/gruszd01/workspace/jimp-bug/node_modules/phin/lib/phin.compiled.js:1:2100)
    at IncomingMessage.emit (events.js:203:15)
    at endReadableNT (_stream_readable.js:1145:12)
    at process._tickCallback (internal/process/next_tick.js:63:19) methodName: 'constructor' }

=========================================
Called jimp.read() 1000 times.
  Failed: 1000.
  Succeeded: 0.
=========================================
codan84 commented 5 years ago

Since i cannot reproduce this locally an example repo is gonna be needed. Also this has nothing to do with the issue we are using to talk about it so another issue would be cool as well.

How can I use and have it not throw a script blocking error? I have a fallback if this fails but it will not fail silently.

aren't you catching and printing the error? the program should continue right?

as I said without a reproducible example I will be little help other than just speculation

@hipstersmoothie I have provided an example repo to reproduce the issue above ⬆️

FridayJew commented 5 years ago

Since i cannot reproduce this locally an example repo is gonna be needed. Also this has nothing to do with the issue we are using to talk about it so another issue would be cool as well.

How can I use and have it not throw a script blocking error? I have a fallback if this fails but it will not fail silently.

aren't you catching and printing the error? the program should continue right? as I said without a reproducible example I will be little help other than just speculation

@hipstersmoothie I have provided an example repo to reproduce the issue above ⬆️

I think no one will answer. Topic is closed.

codan84 commented 5 years ago

Since i cannot reproduce this locally an example repo is gonna be needed. Also this has nothing to do with the issue we are using to talk about it so another issue would be cool as well.

How can I use and have it not throw a script blocking error? I have a fallback if this fails but it will not fail silently.

aren't you catching and printing the error? the program should continue right? as I said without a reproducible example I will be little help other than just speculation

@hipstersmoothie I have provided an example repo to reproduce the issue above ⬆️

I think no one will answer. Topic is closed.

Submitted a new, open one ;)

AriaFantom commented 4 years ago

const Discord = require('discord.js'); const db = require("quick.db") var jimp = require('jimp');

module.exports = async (client, member) =>{ let wChan = db.fetch(${member.guild.id})

if(wChan == null) return;

if(!wChan) return;

let font = await jimp.loadFont(jimp.FONT_SANS_32_BLACK) //We declare a 32px font let font64 = await jimp.loadFont(jimp.FONT_SANS_64_WHITE) //We declare a 64px font let bfont64 = await jimp.loadFont(jimp.FONT_SANS_64_BLACK) let mask = await jimp.read('https://i.imgur.com/552kzaW.png') //We load a mask for the avatar, so we can make it a circle instead of a shape let welcome = await jimp.read('http://rovettidesign.com/wp-content/uploads/2011/07/clouds2.jpg') //We load the base image

jimp.read(member.user.displayAvatarURL).then(avatar => { //We take the user's avatar avatar.resize(200, 200) //Resize it mask.resize(200, 200) //Resize the mask avatar.mask(mask) //Make the avatar circle welcome.resize(1000, 300)

welcome.print(font64, 265, 55, Welcome ${member.user.username}) //We print the new user's name with the 64px font welcome.print(bfont64, 265, 125, To ${member.guild.name}) welcome.print(font64, 265, 195, There are now ${member.guild.memberCount} users) welcome.composite(avatar, 40, 55).write('Welcome2.png') //Put the avatar on the image and create the Welcome2.png bot try{ member.guild.channels.get(wChan).send(``, { files: ["Welcome2.png"] }) //Send the image to the channel }catch(e){ // dont do anything if error occurs // if this occurs bot probably can't send images or messages } })

} ○This is a event guildmemberaddup up○ Pls help me and tell me how to fix

tomisinteazer commented 3 years ago

this issue for me was caused by the .DS_store i removed it from the array and all worked fine

seiyria commented 3 years ago

@tomisinteazer thanks, this helped me. Erroneously assuming fs.readDirSync would only return the png files I was expecting was a mistake; I had to filter the array but files that had png in the name.

AsuScholar commented 2 years ago

this issue for me was caused by the .DS_store i removed it from the array and all worked fine

Thank you! This fixed my issue as well.