MarconiS
commented
1 year ago I am off the grid the rest of the week but I am definitively giving a crack at it next week. I've got the function for it up and running on the GeotreeHealth repo. Just need to refactor the function to work directly with deepForest and allow for either SAM or fastSAM (fastSAM seems it is already integrated with transformers library, which is a plus: https://huggingface.co/An-619/FastSAM)
It is slow, but it is not overwhelming slow: the good news is that
https://segment-anything.com/ was a new transformer object detection model. @MarconiS showed good results for bounding box to polygon workflows. The main problem was it was too slow. There is a new CNN module that claims similar performance at 50x speed. This needs to be investigated.
https://arxiv.org/pdf/2306.12156.pdf
Next steps.
[ ] @MarconiS can you give us links and a couple code snippets to assess next steps for getting this into main branch. [ ] Determine how to evaluate results given bounding box test data we have on hand [ ] Identify potential workflow: after prediction (predict_image or predict_tile), take bounding box coordinates as prompts for segment anything. Do we use the 4 vertexes of the bounding box, or the centroids of the boxes? [ ] Add fastSA model weights to git download. Use for either bird or tree release.