Predicted_raster doesn't load

[matismatis]

Hello everyone,

I am a forestry student, and trying to learn python for remote sensing/GIS. This is one of my first projects with python, so I am sorry if this is a a simple issue.

I am trying to make ''deepforest'' run with Anaconda Powershell Prompt (miniconda3) and Jupyter Notebook, using Windows 11 Home version 22h2. I installed ''deepforest'' and associated packages on the powershell prompt, and tried running the code in Jupyter. My first TIFF file consists of 4 TIF rasters that I merged in QGIS. It could not be read/recognized by the code, so I tried running an individual TIF file. The file could be read, and it gave me a number of detections. However, I could not open the predicted_raster, so I can't verify the output.

I have uploaded the code and the response in 2 separate text files. Sorry if this is not how you're supposed to attach them, kind of new to coding and such. I tried uploading the tif that I use, but I am unable to do so, my apologies. deepforest_code_jupyter.txt response_deepforest_code.txt

Later on I tried different ways to open the predicted_raster. First I verified it using: print(predicted_raster)

The response that I got was:

[[ 67 56 44] [ 66 56 44] [ 64 54 43] ... [126 145 157] [119 140 153] [120 141 155]]

Then I tried to open it with: plt.imshow(predicted_raster, vmin=0, vmax=255) plt.show()

The response I got was:

I also tried: plt.imshow(predicted_raster, aspect='auto') plt.show()

And got:

Finally I tried: plt.plot(predicted_raster[:, 0], label='Intensity') plt.legend() plt.show()

And got:

I hope that my message contains enough info on how I ran the code. If something is missing/unclear, please let me know what I should share.

[bw4sz]

Happy to help. Can you start by using rasterio to load your raster so we can see it.

import rasterio
from matplotlib import pyplot
raster_path = "<path to your raster here>"
image = rasterio.open(raster_path).read()
image.shape

pyplot.imshow(image)
pyplot.show()

Check out https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#quoting-code for how to format code and paste it in this issue. That's the easiest way to help.

[matismatis]

Thank you for your response. I was able to see the image when using your code, so I am not sure why I am unable to see the file after deepforest detected trees.

Hopefully this is okay. Below is the output after running this code.

I also uploaded the .tif to google drive, if that would help. https://drive.google.com/file/d/1ymAY0VEZ2CvRuEXteMRBv__OW08ht8kn/view?usp=sharing

Again, thank you so much for your help!

import rasterio
from matplotlib import pyplot

# Replace the path below with the path to your raster file
raster_path = "D:\\GISPROJECT\\deepforest\\training\\dsm\\rgb\\2022_194000_449000_RGB_hrl.tif"

with rasterio.open(raster_path) as src:
    image = src.read()

print(image.shape)

pyplot.imshow(image.transpose((1, 2, 0)))  # Transpose the dimensions for plotting
pyplot.show()

[bw4sz]

Okay, pasting your code here for future ease of reference.

from deepforest import main
import os
import matplotlib.pyplot as plt

# Initialize the model
model = main.deepforest()
model.use_release()

# Path to the raster image
raster_path = r"D:\GISPROJECT\deepforest\training\dsm\rgb\2022_194000_449000_RGB_hrl.tif"

# Specify the patch size and overlap
patch_size = 300
patch_overlap = 0.25

# Perform prediction on the tile
result = model.predict_tile(raster_path, return_plot=True, patch_size=patch_size, patch_overlap=patch_overlap)

# Extract the predicted raster and boxes from the result
predicted_raster = result[0]
predicted_boxes = result[1]

# Load the original image for visualization
original_image = plt.imread(raster_path)

# Display the original image
plt.imshow(original_image)

# Plot the predicted bounding boxes on the original image
for _, box in predicted_boxes.iterrows():
    xmin, ymin, xmax, ymax = box["xmin"], box["ymin"], box["xmax"], box["ymax"]
    rect = plt.Rectangle((xmin, ymin), xmax - xmin, ymax - ymin, linewidth=1, edgecolor="r", facecolor="none")
    plt.gca().add_patch(rect)

# Save the plot with bounding boxes to the "boompjes" directory
output_dir = "boompjes"
os.makedirs(output_dir, exist_ok=True)
output_path = os.path.join(output_dir, "predicted_boompjes_tile.png")
plt.savefig(output_path)

# Show the plot (optional)
plt.show()

# Print the path where the visualization is saved
print(f"Visualization saved to: {output_path}")

Couple things jump out at me.

# Extract the predicted raster and boxes from the result
predicted_raster = result[0]
predicted_boxes = result[1]

When you return plot = True with predict tile, you just get the raster image, you don't get the boxes. Its either boxes or image, based on return plot argument. See

https://deepforest.readthedocs.io/en/latest/getting_started.html#predict-a-tile

Also, we already have a plot_predictions function if you wanted https://deepforest.readthedocs.io/en/latest/_modules/deepforest/visualize.html#plot_predictions

but more importantly, the when you specify return plot, you get the raster with boxes already overlayed using cv2 draw, so you just need to call

raster_path = get_data("OSBS_029.tif")
# Window size of 300px with an overlap of 25% among windows for this small tile.
predicted_raster = model.predict_tile(raster_path, return_plot = True, patch_size=300, patch_overlap=0.25)
plt.imshow(predicted_raster)
plt.show()

Most importantly, was there something about the docs or examples that could have helped you get to this answer more quickly? Its really important to me that we continue to make the API as easy as possible. I'll add the plotting in the get started, perhaps commented out, so that users can see it.

[ethanwhite]

@jyo94 - it looks like you have an issue with loading the model from a checkpoint, which is unrelated to this issue. If this is already covered in your other issues we'll get to them as soon as we can. If not then please confirm that the error is just from the first 2-3 lines of code and then open a new issue noting just that code and the associated error.

[matismatis]

Hello everyone,

Thanks for the replies! I have been messing with trying to make it work the last days, but so far I haven't been able to. Since I am quite new to Python, I feel as if this might be a step too high for me unfortunately.

However, I also feel like I might have another issue that caused me to be unable to use ''DeepForest''. I have a AMD Ryzen 7 3700U CPU with Radeon Vega Mobile Gfx GPU. I read that they are not compatible with CUDA. I also get the message that no GPU, TPU, IPU or HPU is available when I try to use the model to make predictions.

[ethanwhite]

@matismatis - OK, let's start by making sure that things are working as expected on your system with known data. To do this I'm going to have you run the relevant code from the Getting Started page.

Open a Jupyter Notebook and run this code:

from deepforest import main
from deepforest import get_data
import os
import matplotlib.pyplot as plt

model = main.deepforest()
model.use_release()

raster_path = get_data("OSBS_029.tif")
predicted_raster = model.predict_tile(raster_path, return_plot = True, patch_size=300,patch_overlap=0.25)

plt.imshow(predicted_raster)
plt.show()

Do you see a picture with trees and bounding boxes?

[matismatis]

Hi Ethan, Thanks for your help. I do see a picture with trees and bounding boxes after running the code. I also got some warning messages, not sure if they are relevant.

GPU available: False, used: False

TPU available: False, using: 0 TPU cores

IPU available: False, using: 0 IPUs

HPU available: False, using: 0 HPUs

C:\Users\Matis\miniconda3\envs\deepfor\Lib\site-packages\pytorch_lightning\trainer\connectors\logger_connector\logger_connector.py:67: Starting from v1.9.0, `tensorboardX` has been removed as a dependency of the `pytorch_lightning` package, due to potential conflicts with other packages in the ML ecosystem. For this reason, `logger=True` will use `CSVLogger` as the default logger, unless the `tensorboard` or `tensorboardX` packages are found. Please `pip install lightning[extra]` or one of them to enable TensorBoard support by default

Model from DeepForest release https://github.com/weecology/DeepForest/releases/tag/1.0.0 was already downloaded. Loading model from file.

C:\Users\Matis\miniconda3\envs\deepfor\Lib\site-packages\deepforest\main.py:111: UserWarning: The config file specifies architecture retinanet, but the release model is torchvision retinanet. Reloading with deepforest.main with a retinanet model
  warnings.warn(
Loading pre-built model: https://github.com/weecology/DeepForest/releases/tag/1.0.0
C:\Users\Matis\miniconda3\envs\deepfor\Lib\site-packages\pytorch_lightning\trainer\connectors\data_connector.py:436: Consider setting `persistent_workers=True` in 'predict_dataloader' to speed up the dataloader worker initialization.
Predicting DataLoader 0: 100%
4/4 [00:14<00:00, 0.28it/s]

197 predictions in overlapping windows, applying non-max supression
94 predictions kept after non-max suppression

C:\Users\Matis\miniconda3\envs\deepfor\Lib\site-packages\deepforest\visualize.py:118: UserWarning: Input images must be channels last format [h, w, 3] not channels first [3, h, w], using np.rollaxis(image, 0, 3) to invert!
  warnings.warn("Input images must be channels last format [h, w, 3] not channels "

C:\Users\Matis\miniconda3\envs\deepfor\Lib\site-packages\deepforest\visualize.py:126: UserWarning: No color was provided and the label column is not numeric. Using a single default color.
  warnings.warn("No color was provided and the label column is not numeric.

Could these warnings cause issues when I try to run the code with my own pictures/using tiles?

EDIT: I would like to thank everyone for their help. Instead of using aerial photographs, I now used orthophoto.tif I made with my drone. I somehow got it to work, not sure how. This tif has a higher resolution, perhaps that has been helpful?

[ethanwhite]

Congrats @matismatis! I'm very glad to hear that it's working for you. The resolution difference could definitely help. We do most of our work with orthomosaic tifs as well.

None of those warnings should be causing any issues and we're working on trying to cleanup as many of them as possible.

If you have anymore questions or issues as you continue your work just let us know!