Example for using direct buffer

[vajonam]

Hi, I am trying to directly use a buffer, but I am not sure how I call the .detect method. I am using multer with in memory storage

my question is about predictions = darknet.detect(req.file.buffer);, I can see that detect(input: string | IBufferImage | IOpenCVFrame, config: IConfig = {}):, How can I send it IBufferImage, I have the code I am using below, which results in a throw new Error('Could not get valid image from input!');

Writing it to a file and read it back works, but I was hoping to not need that step.

app.post('/yolo', upload.single('photo'), function (req, res, next) {
  if (req.file.buffer.length < 100)  {
    console.log(`-- invalid file`);
    res.json({});
    return;      
  }

  var predictions = "";
  try {
    console.time("Detection");
    predictions = darknet.detect(req.file.buffer);
    console.timeEnd("Detection");

[vajonam]

any thoughts on this?

[bennetthardwick]

Hey sorry, thought I replied. An IBufferImage is the following type: https://github.com/bennetthardwick/darknet.js/blob/d750f140b96a63c2a49afe8b2a83cf85e0499909/lib/darknet.ts#L5-L10 Where w is the width of the image in pixels, h is the height in pixels, c is the number of channels per pixel and b is the raw RGB data.

If you're uploading image like pngs or jpgs you'll need to use a library to convert them into an rgb buffer. At the moment Darknet does this in this function:

https://github.com/AlexeyAB/darknet/blob/14b196d4f2f73fb2f6f8c4019de9a0de00c5a27e/src/image.c#L1487-L1522

I'll try and put up an example of how to do this later tonight.

[vajonam]

Thanks for this. I understand now. I found some dodgy code that will do it.. I will test it out, but maybe a convinence method to be able to send a image buffer (png/jpg) would be nice to add to darknet.js

let image = msg.payload;
for(let y = 0; y < image.bitmap.height; y+=2){
    for(let x = image.bitmap.width; x > 0; x--){
        color = image.getPixelColor(x-1, y) ;
        r = (color >> 24) & 255; 
        g = (color >> 16) & 255; 
        b = (color >> 8) & 255; 
        rgb.push(r);
        rgb.push(g);
        rgb.push(b);
    }
    for(let x = 0; x < image.bitmap.width; x++){
        color = image.getPixelColor(x, y+1) ;
        r = (color >> 24) & 255; 
        g = (color >> 16) & 255; 
        b = (color >> 8) & 255; 
        rgb.push(r);
        rgb.push(g);
        rgb.push(b);
    }
}
msg.rgb = rgb;
msg.rgbBuffer = Buffer.from(rgb);

[bennetthardwick]

Nice!

but maybe a convinence method to be able to send a image buffer (png/jpg) would be nice to add to darknet.js

I've had a look and it seems like the only way with darknet to load a png/jpg is to go by filename. Maybe the convenience function would write to a tmp folder and then clean up afterwards? I've opened up #36 for this.