output image with boxes

[vajonam]

will it be possible to output an image, which I can return to call (I am calling this API via an express based web service) along with the predictions json?

[bennetthardwick]

I'm currently rewriting darknet an will definitely consider adding in this functionality.

What would the interface look like? Darknet saves an image with boxes to disk and return it's path?

[vajonam]

Yes that will work. It can return it’s path and I can return that response

I was thinking of stuffing the predictions into a response header and the image stream back to the caller, but open to other more graceful suggestions

Or Maybe base64 encoded object as part of JSON response.

You can peek at the docker image I have built to do this using express.

https://github.com/vajonam/darknet-express

On Tue, Jul 16, 2019 at 7:14 PM Bennett Hardwick notifications@github.com wrote:

I'm currently rewriting darknet an will definitely consider adding in this functionality.

What would the interface look like? Darknet saves an image with boxes to disk and return it's path?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bennetthardwick/darknet.js/issues/21?email_source=notifications&email_token=AABFHNKO2NF3K65DYNXFTBLP7ZI5JA5CNFSM4IEDKKJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2CRBIA#issuecomment-512037024, or mute the thread https://github.com/notifications/unsubscribe-auth/AABFHNPYZ4ABAUGRCLTGV6TP7ZI5JANCNFSM4IEDKKJQ .

[vajonam]

any chance you are thinking to base your rewrite on the https://github.com/AlexeyAB/darknet fork? seems that it has number of performance enhancements.

[bennetthardwick]

I'll look into it, thanks! It seems like it might be a drop-in replacement, which should be easy. I'll have to look into things like compatability with Mac - which darknet.js has at the moment.

[vajonam]

Yes just tried it seems to work if I just pull from the AlexyAB repo instead, the Makefile needs some defaults set like LIBSO=1 etc. but otherwise works.

On Wed, Jul 17, 2019 at 11:18 PM Bennett Hardwick notifications@github.com wrote:

I'll look into it, thanks! It seems like it might be a drop-in replacement, which should be easy. I'll have to look into things like compatability with Mac - which darknet.js has at the moment.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bennetthardwick/darknet.js/issues/21?email_source=notifications&email_token=AABFHNJQB2AERUTVJNVSQ6LP77OIBA5CNFSM4IEDKKJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2HGM6A#issuecomment-512648824, or mute the thread https://github.com/notifications/unsubscribe-auth/AABFHNNMR7TO3B4QVETIPO3P77OIBANCNFSM4IEDKKJQ .

[vajonam]

Also just curious how far out do you think this rewrite project is?

I might do some of the box drawing on the image on the node JS for now. Just curious if should spend the time just wait for you.

On Thu, Jul 18, 2019 at 10:39 AM Manojav Sridhar manojav@manojav.com wrote:

Yes just tried it seems to work if I just pull from the AlexyAB repo instead, the Makefile needs some defaults set like LIBSO=1 etc. but otherwise works.

On Wed, Jul 17, 2019 at 11:18 PM Bennett Hardwick < notifications@github.com> wrote:

I'll look into it, thanks! It seems like it might be a drop-in replacement, which should be easy. I'll have to look into things like compatability with Mac - which darknet.js has at the moment.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bennetthardwick/darknet.js/issues/21?email_source=notifications&email_token=AABFHNJQB2AERUTVJNVSQ6LP77OIBA5CNFSM4IEDKKJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2HGM6A#issuecomment-512648824, or mute the thread https://github.com/notifications/unsubscribe-auth/AABFHNNMR7TO3B4QVETIPO3P77OIBANCNFSM4IEDKKJQ .

[bennetthardwick]

I've pretty much completed the synchronous API - you can test it out by installing darknet@beta - but haven't started working on the async API just yet. It's at the top of my priorities before working on new features.

If you're interested in looking into it, darknet has a method called draw_detections, but you have to pass stuff like a character map to it, so it's a bit tedious.

[vajonam]

cool, which branch on github is that? I only see a tag for 2.0.1 which looks quite new? but npm version says 2.0.6

[bennetthardwick]

I just pushed the tags. It's also here

[vajonam]

I did try the alexyAB fork, it is about 4 seconds (11s vs 14s) for the regular mode, but it consistently misses a bunch of objects, so am not going to push that thread for now..

[vajonam]

Regarding the Mask I found a neat solution:

I found a neat solution. I use ZoneMinder for my cameras. I can define the a zones in there as an Inactive Zone, read this via an API via Node-RED and then I get back the predictions from darknet.js if they lie within that defined polygon exclude object from the result. I have the code to do all of this if incase you are interested to provide it as a feature.

Now when you get around to building the box drawing API's will be nice to be able to pass it only the predictions I want to draw on there. or to pass it point array [[x1,y1],[x2,y2]] etc. to check if the bounds of the box are in that polygon if they are ignore. will leave that up to you.

check if inside a polygon https://github.com/substack/point-in-polygon

[bennetthardwick]

Nice! I think it's probably best to draw the boxes on the images using node canvas since this library looks pretty good. It probably doesn't make sense to push the data back and forth.

[viper86it]

Yes just tried it seems to work if I just pull from the AlexyAB repo instead, the Makefile needs some defaults set like LIBSO=1 etc. but otherwise works. … On Wed, Jul 17, 2019 at 11:18 PM Bennett Hardwick @.**> wrote: I'll look into it, thanks! It seems like it might* be a drop-in replacement, which should be easy. I'll have to look into things like compatability with Mac - which darknet.js has at the moment. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#21?email_source=notifications&email_token=AABFHNJQB2AERUTVJNVSQ6LP77OIBA5CNFSM4IEDKKJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2HGM6A#issuecomment-512648824>, or mute the thread https://github.com/notifications/unsubscribe-auth/AABFHNNMR7TO3B4QVETIPO3P77OIBANCNFSM4IEDKKJQ .

Could you please explain me how to use Darknet (AlexeyAB) fork? I tryed different ways but without success :(

[vajonam]

Dont remeber how I did, but firstly can you build it? if so you need to enable LIB in the config file so it will output a darknet.so so something like that. This was using the previous FFI implementation. I haven't tried it with latest implementation that doesn't use FFI

On Fri, Sep 6, 2019 at 3:33 AM viper86it notifications@github.com wrote:

Yes just tried it seems to work if I just pull from the AlexyAB repo instead, the Makefile needs some defaults set like LIBSO=1 etc. but otherwise works. … <#m_1442341700097876691m-2475680855178156105_> On Wed, Jul 17, 2019 at 11:18 PM Bennett Hardwick @.**> wrote: I'll look into it, thanks! It seems like it might* be a drop-in replacement, which should be easy. I'll have to look into things like compatability with Mac - which darknet.js has at the moment. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <

21 https://github.com/bennetthardwick/darknet.js/issues/21?email_source=notifications&email_token=AABFHNJQB2AERUTVJNVSQ6LP77OIBA5CNFSM4IEDKKJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2HGM6A#issuecomment-512648824>,

or mute the thread https://github.com/notifications/unsubscribe-auth/AABFHNNMR7TO3B4QVETIPO3P77OIBANCNFSM4IEDKKJQ .

Could you please explain me how to use Darknet (AlexeyAB) fork? I tryed different ways but without success :(

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bennetthardwick/darknet.js/issues/21?email_source=notifications&email_token=AABFHNPPNLC3AUCWEWLQZF3QIIBU7A5CNFSM4IEDKKJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6CAQTA#issuecomment-528746572, or mute the thread https://github.com/notifications/unsubscribe-auth/AABFHNJLJYSOVZVO6IDVJGLQIIBU7ANCNFSM4IEDKKJQ .

[celevra]

this works, needs a for loop and width & height detection

var fs = require('fs');
const { Darknet } = require('darknet');

const darknet = new Darknet({
    weights: './yolov3.weights',
    config: './yolov3.cfg',
    namefile: './coco.names'
});

var cords = darknet.detect('/tmp/testcam-20191001-1233481751.jpg')
cords.x = cords.x-cords.w/2
cords.y = cords.y-cords.h/2

const { createCanvas, loadImage } = require('canvas')
const canvas = createCanvas(3840, 2160)
const ctx = canvas.getContext('2d')

loadImage('/tmp/testcam-20190930-1758380917.jpg').then((image) => {
    ctx.drawImage(image, 0, 0)
    ctx.strokeStyle = 'rgba(255,0,0,1)'
    ctx.lineWidth = 10;
    ctx.strokeRect(cords.x, cords.y, cords.w, cords.h);

    fs.writeFileSync('/tmp/test.png', canvas.toBuffer())
})

[vajonam]

@celevra thanks for the pointers

was able to get this working, iterates through the predictions and then draws boxes and returns the image as base64 string along with the array of predictions.

app.post('/yolo', upload.single('photo'), function (req, res, next) {
  var filename = `./${req.file.destination}${req.file.filename}`
  console.log(`-- yolo received --: ${filename}`)
  if (req.file.size < 100) {
    fs.unlink(filename, d => { })
    console.log(`-- invalid file`);
    res.json({});
    return;

  }

  var predictions = "";
  try {
    console.time("Detection");
    predictions = darknet.detect(filename);
    console.timeEnd("Detection");

  } catch (err) {
    console.log(`-- yolo: --:${err.message}`);
  }
  var result = "";

  const { createCanvas, loadImage } = require('canvas')
  const canvas = createCanvas(1280, 720)
  const ctx = canvas.getContext('2d')

  loadImage(filename).then((image) => {
    ctx.drawImage(image, 0, 0)
    ctx.strokeStyle = 'rgba(255,0,0,1)';
    ctx.lineWidth = 2;
    ctx.fillStyle = 'rgba(255,0,0,1)';
    predictions.forEach(function (prediction) {
      var result = prediction.name + " : " + prediction.prob.toFixed(2)*100 + "%";
      ctx.lineWidth = 2;
      ctx.strokeRect(prediction.box.x-prediction.box.w/2, prediction.box.y-prediction.box.h/2, prediction.box.w, prediction.box.h);
      ctx.lineWidth = 1;
      ctx.font = '20px';
      ctx.fillText(result, (prediction.box.x-prediction.box.w/2)+10,  (prediction.box.y-prediction.box.h/2)+20);
    });
    fs.writeFileSync('/tmp/test.png', canvas.toBuffer());
    var retrunValue = {};

    retrunValue.predictions = predictions;
    retrunValue.image = canvas.toDataURL("image/jpeg");
    console.log(retrunValue.predictions);
    console.log(`-- yolo: predictions --: ${result}`);
    res.json(retrunValue);
    fs.unlink(filename, d => { })
    console.log(`-- yolo processed  --: ${filename}`)

  });

  })

[vajonam]

@bennetthardwick I think all the code is there there do draw the boxes, do you want pull this into examples? I dont think we should do it within in the API, it would need canvas and other stuff that is a bit of a bloat.. we can setup it up as an example.

[bennetthardwick]

Added a drawing box example to the examples folder in 34afadd14430f59fa08c84a42e071646a93ce983