Nice to have feature: Image cropping

[jksemple]

In my project I need square images so I need to crop a section off the left and right of each capture. It would be nice if this was done within the library as part of capture().

[eloquentarduino]

Added to my to-do list.

[nicholasareed]

It would be ideal if this supported multiple crops, and potentially allowed a few different shapes to be drawn and manipulated (oval, rectangle) that would be ignored in detection. #37 might also benefit from the ability to have arbitrary shapes for the masking (ie the whole image is blacked out, but the masked areas are all allowed to "see through"). I suppose it only makes sense to do either masking or cropping, we wouldn't let a user do both in an image.

The use case is when you have a few areas of the image you want to ignore or highlight, for example: a few windows in an image, where they don't neatly fall into a rectangle (so being able to drag the corners to create non-parallel sides is super important).

Maybe usage something like:

// cropping/ignoring shapes 
detection.crop.addCircle([100,100], 50); // [x,y] center, radius 
detection.crop.addQuad([10,10], [40,8], [10,36], [38,34]); // [x,y] of: top-left, top-right, bottom-left, bottom-right

// masking (ie "blackout image, then punch holes through shapes") 
detection.mask.enable();
detection.mask.addCircle(...);
detection.mask.addQuad(...);

// and an ability to reset the detection of shapes
detection.crop.clear();
detection.mask.clear();
detection.mask.disable();

[nicholasareed]

Actually, polygons and the required raycasting probably adds alot of overhead, so it might make more sense to build and precompute the mask pixels somewhere else (like a javascript canvas), then store that in memory and apply the mask to each frame. This makes it much easier to create complex masks/crops.

[eloquentarduino]

The cropping @jksemple was talking about is very different from the masking you're talking about. Regarding masking, I had a rectangle/circle mask implementation in the first version of this software, so I can consider re-introducing it. But this would throw away the use of the built-in dl::image::get_moving_point_number function (which is currently used). Not too bad IMO, will have to test.

I envision a few different implementations:

1. simple primitives built-in (rectangle, circle): I like the API you proposed
2. binary mask: I like the idea of drawing a mask in an external tool and export a mask array to be applied
3. custom mask function: you can pass a function which accepts (x, y) coordinates and returns true if the pixel should be counted and false if it should be skipped

Work on this library have been really slow lately, but I'm collecting all your positive feedback to integrate when I start working back.

Feel free to suggest more features or comment on my ideas on this.

[jksemple]

My thinking on this has changed a bit recently. I think it's more flexible to expose the whole image comparison process in a lambda so that each pair of pixels to compare are presented to the consumer's comparison function which can decide whether to ignore pixels based on x, y position or arbitrary masks and/or apply arbitrary comparisons such as darker or lighter. I've implemented a version of this in my library ESP-Image. I'll be publishing a new version of it when I return from holiday. Ah. I see Simone is considering the arbitrary comparison function too.

[nicholasareed]

I'm leaning towards using a binary mask (compressed with Row Length Encoding), because it is so flexible to create the masks (compared to defining points); I already threw together a simple FabricJS canvas editor that lets me freehand draw shapes for masking, and it could easily export that to be usable by the ESP32. It could also trivially be extended to include circles and other polygons.

(https://codesandbox.io/p/sandbox/laughing-vaughan-jnqr4t).

The cropping @jksemple was talking about is very different from the masking you're talking about.

Agreed, I was kinda smushing a few concepts together :) All about modifying the frame before motion detection processing imo

[jksemple]

For my purposes the built-in get_moving_point_number() function is not sufficiently flexible. I want to look for darker pixels in the new image vs the old image and I can imagine people might want to look for lighter or redder or whatever. So I combined the flexibility I needed for comparison with masking. I only want to look at the central circular area of the images so I can test whether each point is inside or not before doing the comparison. Then return true or false from the lambda and hence calculate the same moving_point ratio in the parent library function. You could apply a more complex mask if you want without bloating the library

[jksemple]

The cropping I wanted is much less of an issue now (though I think it should still be available) because you can request any shape of image directly from the camera sensor using sensor->set_res_raw