search

weecology / DeepForest

Python Package for Airborne RGB machine learning
https://deepforest.readthedocs.io/
MIT License
485 stars 172 forks source link

DeepForest on Earth Engine #611

Closed aeiche01 closed 6 months ago

aeiche01 commented 6 months ago

I tend to work with remote sensing images on earth engine, since I don't have a lot of memory available to me on my personal computer. Unfortunately, as far as I can tell in order to use DeepForest on images I have to pull them off of earth engine first.

It would be great if we could translate DeepForest to work on Earth Engine itself.

ethanwhite commented 6 months ago

Thanks for the suggestion @aeiche01 - it's a cool idea, but I don't know if it is feasible for us.

If you do you processing directly in Earth Engine Code Editor using Javascript then that's not something we can add our functionality too (we'd have to start by rewriting the entire package in a different language).

There is a Python API for Earth Engine that you/we might be able to use to do processing on the data hosted on Earth Engine (https://developers.google.com/earth-engine/tutorials/community/intro-to-python-api). If you have a strong need here I'd recommend seeing if you can figure out how to run DeepForest on the imagery you need to use with that API.

Even if it is possible to build this API use directly into DeepForest it would be a big time commitment. So, we'd need to know that this is a common use case. I suggest we leave this issue open and if a number of other folks have similar needs and want to comment about their use cases then if there is sufficient interest we can explore the possibility. @aeiche01 - we'd be interested in hearing more about your specific use case if you don't mind sharing.

aeiche01 commented 6 months ago

Thanks, Ethan! I've spoken to some of the Earth Engine developers about using the API, and unfortunately we can't use an external python package to analyze images on their platform through the API. Looks like DeepForest would have to be rebuilt on their platform using Javascript, which is definitely not an easy task.

For me, I have a large area that I'd like to ID tree crowns for, but I don't have high resolution data outside of what I can get off Earth Engine (NAIP data).

ethanwhite commented 6 months ago

Thanks for additional info @aeiche01!

I think there are a couple of ways to think about this:

  1. Pull down small chunks of the NAIP imagery using the Python API, run DeepForest, store the results, delete the imagery, pull down the next bit. This would be free with an Earth Engine key and relatively easy. If you go this route we'd love to add it as an example to our docs if you're interested.
  2. I did do some more looking around and the following might work to run the model on Earth Engine https://developers.google.com/earth-engine/guides/machine-learning#getting-predictions-from-a-model-outside-earth-engine. This would require us to deploy the DeepForest model on Google's VertexAI (which can apparently be done https://cloud.google.com/blog/topics/developers-practitioners/pytorch-google-cloud-how-deploy-pytorch-models-vertex-ai). VertexAI is typically not free, but I don't know how their pricing model interacts with running on Earth Engine. This is something we could potentially try, but it wouldn't be immediately.
aeiche01 commented 6 months ago

I think (1) is the best way to move forward. The only problem I thought about (which is why I was wondering about using DeepForest directly on Earth Engine) was how to deal with tree canopies that get bisected on the edge of each individual image and how to combine them at the end. Any suggestions?

(2) is probably not useful for DeepForest, since there's already a VertexAI crown segmentation option available in Earth Engine. If DeepForest gets hosted on VertexAI it loses its biggest advantage of being a free option for Earth Engine

ethanwhite commented 6 months ago

Thanks @aeiche01

The only problem I thought about (which is why I was wondering about using DeepForest directly on Earth Engine) was how to deal with tree canopies that get bisected on the edge of each individual image and how to combine them at the end. Any suggestions?

This is actually an issue for all imagery, not specific to fetching chunks from Earth Engine. When DeepForest (and most computer vision algorithms) processes a large tile it crops that tile into smaller pieces, makes predictions, and then reassembles them.

@bw4sz - are we doing anything special during that reassembly to deal with trees at the edges right now?

bw4sz commented 6 months ago

Yes, on predict_tile, when we mosiac, we use a second round of non-max suppression to choose among overlapping boxes at tile edges. Here:

https://github.com/weecology/DeepForest/blob/1f3b980adda821b39df741bac81063410cda1435/deepforest/predict.py#L86

We need to do a better job evaluating this code, since most evaluation occurs over the predict_image function.

ethanwhite commented 6 months ago

Going to close this for now. @aeiche01 - see @bw4sz's comment above for what we currently do to address this issue in predict_tile and let us know if you have any more questions as you go. Good luck with your project!