how to generate IBufferImage in pure javascript ?

[liuxinqiang]

As in your readme, there is an example like this:

const fs = require('fs');
const cv = require('opencv4nodejs');
const { Darknet } = require('darknet');

const darknet = new Darknet({
  weights: 'yolov3.weights',
  config: 'cfg/yolov3.cfg',
  namefile: 'data/coco.names'
});
const cap = new cv.VideoCapture('video.mp4');
let frame;
let index = 0;
do {
  frame = cap.read().cvtColor(cv.COLOR_BGR2RGB);
  console.log('frame', index++); 
  console.log(darknet.detect({
    b: frame.getData(),
    w: frame.cols,
    h: frame.rows,
    c: frame.channels
  }));
} while(!frame.empty);

Now, I want to detect a single image ,but for some reason , I don't have opencv4nodejs installed, I just want to pass a IBufferImage to Darknet, the question is, how can I generate IBufferImage in pure javascript ?

[liuxinqiang]

For some detail information , my data flow is like this: I use electron to build a desktop app, darknet.js for detection calc in node, in chromium , I add a video element to handle camera input, and a canvas element to get some picture data, then send the picture data as base64 to send to node throw rpc, and the node send the base64 data to darknet, then darknet send the result throw rpc to chromium, chromium draw the result over video element.

This chain was broken when I pass base64 data to daknent, I use new Buffer.form(base64imgData, 'base64') to generate a new Buffer and sent it to daknent, but the result is an empty array.

[bennetthardwick]

Hi @liuxinqiang! You can get pixel data for a base64 url using the pngjs library. Here's an example:

const fs = require('fs');
const { Darknet } = require('../darknet');
const { PNG } = require('pngjs');

const darknet = new Darknet({
    weights: 'yolov3-tiny.weights',
    config: 'yolov3-tiny.cfg',
    namefile: 'coco.names'
});

// Include your method of creating a buffer from base64 here
const dataURL = fs.readFileSync('./dog-base64-dataURL.txt', { encoding: 'utf8' });
const dataString = dataURL.replace(/^data:image\/(png|jpg);base64,/, "")
const buffer = Buffer.from(dataString, 'base64');

// Read the PNG in the buffer. Get the width, height and pixel data.
const png = PNG.sync.read(buffer);

// Run the original detection
console.log("Dog:", darknet.detect('./dog.jpg'));

// Run the new detection
console.log("Dog base64:", darknet.detect({
  w: png.width,
  h: png.height,
  c: 4,
  b: png.data
}));

Let me know if this helps!

[bennetthardwick]

Feel free to "@" me on this issue if you need help, but I'm going to close it now.

[celevra]

@bennetthardwick tried your example with jpeg-js

                                      request({url: url, method: 'get', encoding: null}, function(error, res, body) {
                                            img = jpeg.decode(body);
                                            console.log(darknet.detect({
                                              w: img.width,
                                              h: img.height,
                                              c: 4,
                                              b: img.data
                                            }));
                                      });

but i get

/root/tmp/node/node_modules/darknet/darknet.js:97
                throw new Error('Could not get valid image from input!');
                ^

[celevra]

here is a more complete example:

process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = 0
var request = require('request');

const { Darknet } = require('darknet');
var jpeg = require('jpeg-js');

const darknet = new Darknet({
    weights: './yolov3.weights',
    config: './yolov3.cfg',
    namefile: './coco.names'
});
var url = 'https://wiki.robotika.sk/robowiki/images/f/f4/Testimage.jpg';
request({ url: url, method: 'get', encoding: null }, function(error, res, body) {

    console.log('Response: StatusCode:', res && res.statusCode);
    console.log('Response: Body: Length: %d. Is buffer: %s', body.length, (body instanceof Buffer));

    img = jpeg.decode(body);
    console.log(darknet.detect({
        w: img.width,
        h: img.height,
        c: 4,
        b: img.data
    }));
});

[bennetthardwick]

Hey @celevra, thanks for finding that bug! It's now been fixed in darknet@2.0.17.