Thanks for making this available.

bw4sz commented 2 years ago

Really well organized. I'm going to give it a try on deepforest and see what kind of results we get. Long term I'd like to add it to our baseline model -> https://deepforest.readthedocs.io/en/latest/, since this is a common use case. All thoughts welcome.

jonathanventura commented 2 years ago

Thanks, and glad to hear it might be of use!

That is a good idea to test deepforest on it. Here is a quick script:

import deepforest
import deepforest.main

import rasterio
import os

model = deepforest.main.deepforest()
model.use_release()

os.makedirs('deepforest',exist_ok=True)

names = [name.rstrip() for name in open('test.txt','r')]
for name in names:  
    image_path = os.path.join('images',f'{name}.tif')
    output_path = os.path.join('deepforest',f'{name}.json')
    with rasterio.open(image_path,'r') as src:
        transform = src.transform   
        crs = src.crs
    pred = model.predict_image(path=image_path)
    gdf = deepforest.utilities.annotations_to_shapefile(pred, transform, crs)
    gdf.to_file(output_path,driver='GeoJSON')

Qualitatively, the precision looks good, but it seems to miss the smaller trees. I will work on writing some code to evaluate it quantitatively. Do you have any suggestions about how to set the parameters?

That would be great to incorporate this data into the deepforest baseline. But would we need to add bounding box annotations?

bw4sz commented 2 years ago

Linking this issue for tracking. https://github.com/weecology/DeepForest/issues/340

We have noticed that deepforest is not great at predicting outside of the 10cm resolution it was trained on. Its something I think is solvable and want to re-visit. At the very least different crop + zoom data augmentations. One somewhat hacky workaround is to change the size of the input image, especially when using the predict_tile function.

https://deepforest.readthedocs.io/en/latest/better.html#check-patch-size

I'm not comfortable with this solution long term.

More generally there are some really interesting computer vision for ecology questions orbiting this work. I'm very interested in 'generalized' models. As humans we 'trees' as a single semantic concept, I am able to identify trees from locations and resolutions I have no personal experience with. Yet the models appear to be very sensitive to these changes in background and resolution. I'd like to try co-training datasets of different resolution, forest/urban, to understand the tradeoffs (or joint improvements) in a single global model.

In terms of using your data in the deepforest baseline, we have had good experience with weak supervision in training. Use deepforest to predict the images and use heuristic logic to try to infer bounding boxes. We've mostly done this with birds where the objects are rare in the image and the ground truth points are sparse. It will be more challenging here, but I still think it is worth it. Obviously this wouldn't be reasonable to get a test score, this would just be for encouraging deepforest to learn features in more urban NAIP imagery backgrounds. All of this is completely hypothetical, i'm supposed to be working on a full NEON level (100+) tree species classification project, but if you are interested in contributing i'm always happy to discuss. Thanks again for making the data available.

jonathanventura / urban-tree-detection-data

Thanks for making this available. #1