Unable to run model$predict_tile if I set return_plot=TRUE

[priyapatel1]

I'm trying to create a plot for my tif image with bounding boxes overlaid. I can run model$predict_tile and view the plot perfectly fine with I use the built in tif, like this:

bounding_boxes_reference = model$predict_tile('OSBS_029.tif', return_plot = TRUE)

reference_raster = terra::rast(bounding_boxes_reference[,,3:1]/255)

plot(reference_raster)

This gives me a raster with bounding boxes overlaid. However, when I use my OWN raster (attached) - I run into issues. First, I tried running it with return_plot = FALSE.

raster_path = get_data("/Users/priyapatel/Library/CloudStorage/OneDrive-UniversityofToronto/CANUE/3_TCC/sample2.tif") # Gets a path to an example raster tile

bounding_boxes = model$predict_tile(raster_path, return_plot=FALSE)

From that code chunk I get the following output:

/Users/priyapatel/.virtualenvs/r-reticulate/lib/python3.9/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:419: Consider setting `persistent_workers=True` in 'predict_dataloader' to speed up the dataloader worker initialization.
Predicting DataLoader 0: 100%|██████████| 24/24 [00:04<00:00,  4.85it/s]
683 predictions in overlapping windows, applying non-max supression
444 predictions kept after non-max suppression

I also get a variable of bounding boxes, with xmin, xmax, etc. This is all fine, but I would like to see the raster with boxes overlaid. When I change the return_plot argument to TRUE, I get the following:

bounding_boxes = model$predict_tile(raster_path, return_plot=TRUE)

# output 
/Users/priyapatel/.virtualenvs/r-reticulate/lib/python3.9/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:419: Consider setting `persistent_workers=True` in 'predict_dataloader' to speed up the dataloader worker initialization.
Predicting DataLoader 0: 100%|██████████| 24/24 [00:04<00:00,  5.03it/s]
683 predictions in overlapping windows, applying non-max supression
444 predictions kept after non-max suppression
/Users/priyapatel/.virtualenvs/r-reticulate/lib/python3.9/site-packages/deepforest/visualize.py:118: UserWarning: Input images must be channels last format [h, w, 3] not channels first [3, h, w], using np.rollaxis(image, 0, 3) to invert!
  warnings.warn("Input images must be channels last format [h, w, 3] not channels "
/Users/priyapatel/.virtualenvs/r-reticulate/lib/python3.9/site-packages/deepforest/visualize.py:126: UserWarning: No color was provided and the label column is not numeric. Using a single default color.
  warnings.warn("No color was provided and the label column is not numeric. "
Error in py_call_impl(callable, call_args$unnamed, call_args$named) : 
  cv2.error: OpenCV(4.10.0) :-1: error: (-5:Bad argument) in function 'rectangle'
> Overload resolution failed:
>  - img data type = uint32 is not supported
>  - Expected Ptr<cv::UMat> for argument 'img'
>  - argument for rectangle() given by name ('color') and position (3)
>  - argument for rectangle() given by name ('color') and position (3)

Run `reticulate::py_last_error()` for details.

What am I doing wrong here? Why can't I see the plot, but I can get the bounding box extents?

[bw4sz]

Again, lets post in the other repo, these are definitely R related issues and not the underlying API. DeepForestR integration is a whole other challenge and there have 10x more python use. Repost there and we can help.

[ethanwhite]

@priyapatel1 - I transferred this issue into the deepforestr repository since @bw4sz thought it was an R issue (@bw4sz - this is a good way to handle this in the future).

That said, it looks to me like it's actually an issue with the formatting of your raster not being supported by opencv (the core Python library that we use for visualization). For some reason your raster values are stored as uint32 (32 bit unsigned integers) when normally rasters are stored as byte (8 bit unsigned integers). If I convert your raster to byte then everything works fine.

I did this using gdal:

gdal_translate -ot Byte sample2.tif sample2_byte.tif

but there are certainly other ways to do it.

[ethanwhite]

Closing due to inactivity. Feel free to reopen if you come back to this @priyapatel1