feature request: mask

[vajonam]

can we provide mask in terms of set points for a polygon to be able to mask of part of the image that can be ignored when doing the detection.

For example I have an image of my driveway and walkway, I would like it ignore the road, neighbours house etc for object detection. what are you thoughts.

[bennetthardwick]

There's no ability to set mask regions in upstream darknet, at least from what I've seen, so if you wanted to exclude regions to improve performance that wouldn't be possible with darknet.js.

It would be possible to check whether boxes are in a certain region and not return them (or not render them #21)

[vajonam]

understood, i was thinking we can mask out the image before passing it to darknet, black it out or white out the polygons are passed in as mask, so we don't waste cycles on those parts of the image. I can put some code together that uses canvas to do this masking if you like.

[bennetthardwick]

That'd be cool! I'd be interested to see the performance difference between leaving the image as is and masking out certain areas.

[vajonam]

You raise a good point.. :-)

So did a test with two images.

[2019-07-23T13:24:54.294Z] -- yolo received --: ./uploads/4d0427966fe42a6e3b2d0b0b79fabcf6
[2019-07-23T13:25:08.614Z] Detection: 14319.532ms
[2019-07-23T13:25:08.614Z] -- yolo: predictions --: car:1.00,
[2019-07-23T13:25:08.615Z] -- yolo processed  --: ./uploads/4d0427966fe42a6e3b2d0b0b79fabcf6
[2019-07-23T13:25:18.775Z] -- yolo received --: ./uploads/f9b9e20e1c2b1deefc2d4528f8f48331
[2019-07-23T13:25:32.549Z] Detection: 13774.059ms
[2019-07-23T13:25:32.550Z] -- yolo: predictions --: car:1.00,car:0.65,car:0.73,
[2019-07-23T13:25:32.551Z] -- yolo processed  --: ./uploads/f9b9e20e1c2b1deefc2d4528f8f48331

really the masked image took longer! this could be load that was on the box at the time as well. so the efficiency argument doesn't hold.

However it removes unwanted clutter in the image, for example don't really care about neighbours cars, just care about my driveway

[bennetthardwick]

I got a similar performance degradation for the mask, which is weird.

Masked: 40.72807399998419
Masked: 37.42474299995229
Masked: 40.294895000057295
Masked: 39.88511399994604
Masked: 40.7130350000225

Unmasked: 30.5615969998762
Unmasked: 26.3722429999616
Unmasked: 26.48802399984561
Unmasked: 25.96524800010957
Unmasked: 26.827330000000075

Because of this I don't feel it really make sense to add masking into darknet at this time. If the original darknet starts to support it I'll definitely add it in.