weecology / DeepForest

Python Package for Airborne RGB machine learning
https://deepforest.readthedocs.io/
MIT License
465 stars 170 forks source link

Orthomosaic image #598

Closed jyo94 closed 6 months ago

jyo94 commented 6 months ago

I have drone images of a plot taken from different angles. Now I want to stitch or create a orthomosaic image of the entire plot using these images(say 15-17 images approx or even more. Each image has a resolution of 5k-6k px). There is no meta information for the images say lat long details. Is there any feature in deepforest to create ortho images like creating rasters?

I tried WebODM, but the resultant image is of 4 channels and I couldn't run the deepforest model to it. Also the image is not clear.

I'm attaching the screenshot of the orthophoto (not able to upload since it is a tiff file):

image
ethanwhite commented 6 months ago

deepforest doesn't handle the creation of orthomosaics. That's a complex task that belongs to software specialized in those methods (like ODM) and as you've seen can be a bit tricky if you don't have positional information associated with the images. The parameters used for stitching can definitely influence the resulting image quality to you may want to keep experimenting with webODM to see if you can get a clearer result. If you need help with creating good orthomosaics https://gis.stackexchange.com/ can be a good place to ask questions.

You can fix the 4 channel issue by removing the alpha channel (that's the 4th channel that often gets included by orthomosaic creating software). You can typically do this by changing an export option (we use agisoft not ODM so I'm not sure of the setting off the top of my head). You can also remove it after creating the ortho using gdal (see instructions here, after installing gdal) or using Python (both OpenCV and PIL should be able to do this).

jyo94 commented 6 months ago

Thank you so much @ethanwhite for the super quick response. I'm still experimenting with WebODM for the orthomosaic task. Just one question, when a orthomosaic image is generated (say like tiff file or jpg), can we still use our existing deepforest model that was trained on raster images of raw images (by raw images I meant the input for orthophoto creation).

I will close the issue after your response. Thanks once again

ethanwhite commented 6 months ago

when a orthomosaic image is generated (say like tiff file or jpg), can we still use our existing deepforest model that was trained on raster images of raw images (by raw images I meant the input for orthophoto creation).

Short answer - Probably. The best way to know is to apply your model to the orthomosaic and see how it works.

Long answer - Model transfer from raw to orthomosaics and the relative performance of models trained on raw vs orthomosaic data are open questions. It probably depends a lot on the specific orthomosaics and how much distortion occurs during the orthomosaic processing (which depends to some degree on the range of canopy heights, with larger ranges in canopy height introducing more distortion). The good news is that the deepforest release model is trained on orthomosaics, so if you fine tuned the release model then there should still be a lot of information in the network about orthomosaic distortions and I would expect it to handle the transfer pretty well. Since this is a prediction task all that really matters is does the model perform well on the orthomosaic, so once you have an orthomosaic you're happy with then take a look and see how the model works.

jyo94 commented 6 months ago

Great that model is trained on orthomosaics, I will definitely try out the model performance on orthomosaic (once my orthomosaic task is successful). Thank you so much @ethanwhite

ethanwhite commented 6 months ago

You're welcome!