Buffer support [assistance]

[lateiskate]

Hello, I need to fetch images and pass them to .classify(). The images i'll be fetching have .png format.

const tf = require('@tensorflow/tfjs-node');
const fetch = require("node-fetch");
const nsfw = require('nsfwjs');
const model = await nsfw.load();
let r = await fetch(IMAGE)
let buffer = await r.buffer();
const readImage = tf.node.decodeImage(buffer)
const predictions = await model.classify(readImage)
return predictions

I used https://js.tensorflow.org/api_node/1.7.0/#node.decodePng Because it's stated that // Image must be in tf.tensor3d format.

readImage does return:

Tensor {
  kept: false,
  isDisposedInternal: false,
  shape: [ 128, 128, 4 ],
  dtype: 'int32',
  size: 65536,
  strides: [ 512, 4 ],
  dataId: {},
  id: 4402,
  rankType: '3',
  scopeId: 2430
}

but model.classify(readImage) returns: Error: Size(200704) must match the product of shape 1,224,224,3

What am I doing wrong?

[GantMan]

Hiya!

The problem might be this hardcoded 3 https://github.com/infinitered/nsfwjs/blob/master/src/index.ts#L129

But to be sure I'd need to run your code to be 100 sure.

Can you try modifying this 3 to a 4 locally and see if your code passes. If so, we can see about fixing a dynamic alpha channel in the code.

If this ISN'T the problem, I'd like you to make a small public repo so I can hunt down the bug.

[lateiskate]

@GantMan

I did try to change it to 4 but I recieved Error when checking : expected input_1 to have shape [null,224,224,3] but got array with shape [1,224,224,4]. Changed it in https://github.com/infinitered/nsfwjs/blob/master/src/index.ts#L74 as well and recieved the same error

Uhm, do I just upload the code I posted above into a repo?

[GantMan]

You don't have to. I didn't know if you had a nice example setup. Like specific with URL so I could just pull down the code and try to debug. If not, I could create one.

[lateiskate]

@GantMan This is literally what I'm using :D

const nsfw = require('nsfwjs'),
    tf = require('@tensorflow/tfjs-node'),
    fetch = require("node-fetch");

async function predict(imageURL) {
    const model = await nsfw.load();
    let imageFetched = await fetch(imageURL)
    let imageBuffer = await imageFetched.buffer();
    const readImage = tf.node.decodeImage(imageBuffer)
    const predictions = await model.classify(readImage)
    return predictions
}

await predict('https://4al52k24l8r51wpym5i46ltd-wpengine.netdna-ssl.com/wp-content/uploads/sites/2/2020/02/GettyImages-1199242002-1-1024x576.jpg')

Node: 12 nsfwjs: 2.1.0 @tensorflow/tfjs-node: 1.7.0

[ElvinGomez]

Hi @lateiskate and @GantMan, I know is a little bit late response but I had the same problem so the way that solved (I don't know if is the best solution) is using a lib sharp to convert the png buffer to jpg. I think the problem is when use tf.node.decoImage(buffer) that will return tf.Tensor3D or tf.Tensor4D but the method infer() use tf.Tensor3D so I tried to cast decodeImage() as Tensor3D but didn't work.

[cdanwards]

@lateiskate I'm going to close this issue for now as it's been a while since a possible solution was provided. We'll gladly reopen this issue if you'd like us to. Thanks!