By Zhuoyi Zhao and Chengyan Fan from Cryosphere Lab, ESSC, CUHK.
Geo-SAM is a QGIS plugin that aims to help people segment, delineate or label landforms efficiently when using large-size geospatial raster images. Segment Anything Model (SAM) is a foundation AI model with the superpower, but the model size is huge, and using it to process images can take a long time, even with a modern GPU. Our tool uses the strategies of encoding image features in advance and trimming the SAM model. The interactive segmentation algorithm can perform real-time inference (at millisecond
speeds) on a laptop CPU, making it a convenient and efficient tool for dealing with remote sensing images.
The Geo-SAM plugin includes two separate parts: the Image Encoding Part, and the Interactive Segmentation Part. The image encoding part is designed to generate and save the image features using the SAM image encoder, and the encoding process only needs to run once per image. The segmentation part is for interactively segmenting landforms, and it can only be used to segment preprocessed images (whose features have been generated in advance using the encoding tool, as the included demo image).
Comparison of the workflow between Geo-SAM and the original SAM. original SAM package encodes prompts and image simultaneously, while the Geo-SAM model encodes image into feature files at once and queries prompts in real-time (at millisecond speeds) by loading those saved features. |
Read the documentation for more details: https://geo-sam.readthedocs.io/en/latest/.
Preview Mode
Demo for Interactive Segmentation (Execute SAM following the mouse cursor)
instantly after giving prompts
, and can even display results in real-time following the mouse cursor
(Preview mode, currently only available in the dev version, will be added to the stable version after being rigorously tested). Users can have a smooth, interactive experience.This can greatly improve the efficiency and user experience of segmentation.Zhao, Zhuoyi, Fan, Chengyan, & Liu, Lin. (2023). Geo SAM: A QGIS plugin using Segment Anything Model (SAM) to accelerate geospatial image segmentation (1.1.0). Zenodo. https://doi.org/10.5281/zenodo.8191039
@software{zhao_zhuoyi_2023_8191039,
author = {Zhao, Zhuoyi and Fan, Chengyan and Liu, Lin},
title = {{Geo SAM: A QGIS plugin using Segment Anything Model (SAM) to accelerate geospatial image segmentation}},
month = jul,
year = 2023,
publisher = {Zenodo},
version = {1.1.0},
doi = {10.5281/zenodo.8191039},
url = {https://doi.org/10.5281/zenodo.8191039}
}
This repo benefits from Segment Anything and TorchGeo. Thanks for their wonderful work.