back2raw save png file provide buffer option

[youngdjn]

I would like to request an option for ROI.back2raw(..., save_folder="...") to save cropped images without masking them to the projected polygon; just crop the images to the bounding box of the projected polygon (ideally with a buffer option too). For example, for one image, the projected polygon looks like this:

but the image saved when using the save_folder option gets masked to this:

I would prefer for the saved image file to not be masked; just save the unmasked image for the full crop extent. It would be also nice if an optional bounding box buffer width (in pixels) could be applied before cropping the image. The buffer that is applied in the preview image (first image above) is nice.

Thanks for considering this enhancement.

[HowcanoeWang]

The saved image is the alpha layer masked png. It using the fourth layer to describe transparency. The RGB layer has the full information.

imarray = imread('saved_alpha.png')
imarray_full_rgb = imarray[0:3]

plt.imshow(imarray_full_rgb)

The imarray_full_rgb may what you want.

Saving with buffer is not hard to writing code, I will give you an examples soon.

[HowcanoeWang]

Actually, this request feature does not match the target of back2raw function (find the ROI on the raw image, something like image segmentation tasks, while the request is more like image detection task + buffer).

Instead of handling in that function directly, we would recommend to wrttting your own code with a few lines for this special need at current stage.

Dara prepare:

import numpy as np
import matplotlib.pyplot as plt
from skimage.io import imread, imsave
import easyidp as idp

lotus = idp.data.Lotus()
roi = idp.ROI(lotus.shp, name_field='plot_id')
roi.get_z_from_dsm(lotus.metashape.dsm)
ms = idp.Metashape(lotus.metashape.project, chunk_id=0)
img_dict_ms = roi.back2raw(ms)

For one example:

>>> pos = img_dict_ms['N1W1']["DJI_0479"]
>>> pos
array([[  43.91987231, 1247.04066843],
       [  69.02210471,  972.89937995],
       [ 353.25370817,  993.30409335],
       [ 328.10701395, 1267.40353337],
       [  43.91987231, 1247.04066843]])

Read the raw image array, and calculate the boundary plus buffer, and then save the image

img_array = imread(ms.photos["DJI_0479"].path)

xmin, ymin = pos.min(axis=0)
xmax, ymax = pos.max(axis=0)

buffer = 40
xmin -= buffer
ymin -= buffer
xmax += buffer
ymax += buffer

img_pos = np.array([[xmin, ymin], [xmin, ymax], [xmax, ymax], [xmax, ymin], [xmin, ymin]])

cropped_png, offset = idp.cvtools.imarray_crop(img_array, img_pos)

plt.imshow(cropped_png)

Where the offset is the left-top corner of the cropped section in the orginal image.

This does match the output preview:

ms.show_roi_on_img(img_dict_ms, "N1W1", "DJI_0479", show=True)

Then using the skimage imsave function to save images, and write your own forloops for all.

---

Caution

1. When adding buffer, please be careful when the buffered bounding box is out of image area
2. When buffer is added, the ROI position should also changes with buffer offsets
3. When writting for loops, it is not recommended to loop by roi_name -> image_name, instead, you should try to revert the dictionary to image_name -> roi_name, in order to avoid repeatly read one image for many times.

According to previous caution points, the following way may be a better option?

1. Only save the back2raw dict as json file (instead of many small png files, by idp.json.write_json()) and no need to do the backward calculation for each time.
2. Before training, read that json to dict (idp.json.read_json()) to get the raw image coordinate
3. Read the raw imarray, and crop the imarray, then doing the training.

This may save a lot of image / coordinate transform and I/O steps.