Seg2Map :mag_right: :milky_way:
An interactive web map app for applying Doodleverse/Zoo models to geospatial imagery
Overview:
-
Seg2Map facilitates application of Deep Learning-based image segmentation models and apply them to high-resolution (~1m or less spatial footprint) geospatial imagery, in order to make high-resolution label maps. Please see our wiki for more information.
-
The principle aim is to generate time-series of label maps from a time-series of imagery, in order to detect and assess land use/cover change. This project also demonstrates how to apply generic models for land-use/cover on publicly available high-resolution imagery at arbitrary locations.
-
Imagery comes from Google Earth Engine via s2m_engine. Initially, we focus on NAIP time-series, available for the conterminious United States since 2003. In the future, Planetscope imagery may also be made available (for those with access, such as federal researchers).
-
We offer a set of Segmentation Zoo models, especially created and curated for this project based on a publicly available datasets. These datasets have been selected because they are public, large (several hundred to several thousand labeled images), and provide broad class labels for generic land use/cover mapping needs.
Installation Instructions
In order to use seg2map you need to install Python packages in an environment. We recommend you use Anaconda to install the python packages in an environment for seg2map. After you install Anaconda on your PC, open the Anaconda prompt or Terminal in Mac and Linux and use the cd
command (change directory) to go the folder where you have downloaded the seg2map repository.
- Create an Anaconda environment
-
Activate your conda environment
conda activate seg2map
- If you have successfully activated seg2map you should see that your terminal's command line prompt should now start with
(seg2map)
.
- Install Conda Dependencies
- seg2map requires
jupyterlab
and geopandas
to function properly so they will be installed in the seg2map
environment.
- Geopandas has GDAL as a dependency so its best to install it with conda.
- Make sure to install geopandas from the
conda-forge
channel to ensure you get the latest version.
- Make sure to install both jupyterlab and geopandas from the conda forge channel to avoid dependency conflicts
conda install -c conda-forge geopandas jupyterlab -y
- Install the seg2map from PyPi
pip install seg2map
- Uninstall the h5py installed by pip and reinstall with conda-forge
pip uninstall h5py -y
conda install -c conda-forge h5py -y
Having Installation Errors?
Use the command conda clean --all
to clean old packages from your anaconda base environment. Ensure you are not in your seg2map environment or any other environment by running conda deactivate
, to deactivate any environment you're in before running conda clean --all
. It is recommended that you have Anaconda prompt (terminal for Mac and Linux) open as an administrator before you attempt to install seg2map
again.
Conda Clean Steps
conda deactivate
conda clean --all
How to Use Seg2Map
- Sign up to use Google Earth Engine Python API
First, you need to request access to Google Earth Engine at https://signup.earthengine.google.com/. It takes about 1 day for Google to approve requests.
-
Activate your conda environment
conda activate seg2map
- If you have successfully activated seg2map you should see that your terminal's command line prompt should now start with
(seg2map)
.
- Install the seg2map from PyPi
cd <location you downloaded seg2map>
ex: cd C:\1_repos\seg2map
- Launch Jupyter Lab
- make you run this command in the seg2map directory so you can choose a notebook to use.
jupyter lab
Features
1. Download Imagery from Google Earth Engine
Use google earth engine to download multiple years worth of imagery.
You can download multiple ROIs and years of data at lighting speeds 🌩️
2. Apply Models to Imagery
3. Load Segmented Imagery onto the Map
Generic workflow:
- Provide a web map for navigation to a location, and draw a bounding box
- Provide an interface for controls (set time period, etc)
- Download geospatial imagery (for now, just NAIP)
- Provide tools to select and apply a Zoo model to create a label image
- Provide tools to interact with those label images (download, mosaic, merge classes, etc)
Authors
Contributions:
We welcome collaboration! Please use our Discussions tab if you're interested in this project. We welcome user-contributed models! They must be trained using Segmentation Gym, and then served and documented through Segmentation Zoo - get in touch and we'll walk you through the process!
Roadmap / progress
V1
- [X] Develop codes to create a web map for navigation to a location, and draw a bounding box
- [X] Develop codes interface for controls (time period, etc)
- [X] Develop codes for downloading NAIP imagery using GEE
- [X] Put together a prototype jupyter notebook for web map, bounding box, and image downloads
- [ ] Create Seg2Map models
- [X] Coast Train / aerial / high-res. sat
- [X] 2 class dataset (water, other)
- [X] zenodo release for 768x768 imagery zenodo page
- [X] Coast Train / NAIP
- [X] 5 class dataset (water, whitewater, sediment, bare terrain, other terrain)
- [X] 8 class dataset (water, whitewater, sediment, bare terrain, marsh veg, terrestrial veg, ag., dev.)
- [X] zenodo release of 5-class ResUNet models for 768x768 imagery zenodo page
- [X] zenodo release of 8-class ResUNet models for 768x768 imagery zenodo page
- [X] zenodo release of 5-class Segformer models for 768x768 imagery zenodo page
- [X] zenodo release of 8-class Segformer models for 768x768 imagery zenodo page
- [X] Chesapeake Landcover (CCLC) / NAIP
- [X] 7 class dataset (water, tree canopy / forest, low vegetation / field, barren land, impervious (other), impervious (road), no data)
- [X] zenodo release of 7-class ResUNet models for 512x512 imagery page
- [X] zenodo release of 7-class SegFormer models for 512x512 imagery page
- [X] EnviroAtlas / NAIP
- [X] 6 class dataset (water, impervious, barren, trees, herbaceous, shrubland)
- [X] zenodo release of 6-class ResUNet models for 1024 x 1024 models zenodo page
- [X] OpenEarthMap / aerial / high-res. sat
- [X] 9 class dataset (bareland, rangeland, dev., road, tree, water, ag., building, nodata)
- [X] zenodo release of 9-class ResUNet models for 512x512 models zenodo page
- [X] DeepGlobe / aerial / high-res. sat
- [X] 7 class dataset (urban, ag., rangeland, forest, water, bare, unknown)
- [X] zenodo release for of 7-class ResUNet models 512x512 imagery zenodo page
- [ ] Barrier Islands / orthomosaic / coastlines
- [ ] Substrate data 6 class (dev, sand, mixed, coarse, unknown, water)
- [ ] zenodo release of substrate models for 768x768 imagery
- [ ] Vegetation type data 7 class (shrub/forest, shrub, none/herb., none, herb., herb./shrub, dev)
- [ ] zenodo release of Vegetation type models for 768x768 imagery
- [ ] Vegetation density data 7 class (dense, dev., moderate, moderate/dense, none, none/sparse, sparse)
- [ ] zenodo release of Vegetation density models for 768x768 imagery
- [X] Geomorphic setting data 7 class (beach, backshore, dune, washover, barrier interior, marsh, ridge/swale)
- [ ] zenodo release of Geomorphic setting models for 768x768 imagery
- [X] Supervised classification data 9 class (water, sand, herbaceous veg./low shrub, sparse/moderate, herbaceous veg/low shrub, moderate/dense, high shrub/forest, marsh/sediment, marsh/veg, marsh, high shrub/forest, development)
- [ ] zenodo release of Supervised classification models for 768x768 imagery
- [X] AAAI / aerial / high-res. sat
- [X] 2 class dataset (other, building)
- [X] zenodo release for 1024x1024 imagery zenodo page
- [X] 2 class dataset (other, flooded building)
- [X] zenodo release for 1024x1024 imagery zenodo page
- [X] xBD-hurricanes / aerial / high-res. sat, a subset of the XView2 dataset
- [X] 4 class building dataset (other, no damage, minor damage, major damage)
- [X] zenodo release for 768x768 imagery zenodo page
- [X] 2 class building dataset (other, building)
- [X] zenodo release for 768x768 imagery zenodo page
- [ ] Superclass models
- [X] 8 merged datasets for 8 separate superclass models (water, sediment, veg, herb. veg., woody veg., impervious, building, agriculture)
- [ ] zenodo release for 768x768 imagery / water
- [ ] zenodo release for 768x768 imagery / sediment
- [ ] zenodo release for 768x768 imagery / veg
- [ ] zenodo release for 768x768 imagery / herb. veg.
- [ ] zenodo release for 768x768 imagery / woody veg.
- [ ] zenodo release for 768x768 imagery / impervious
- [ ] zenodo release for 768x768 imagery / building
- [ ] zenodo release for 768x768 imagery / agriculture
- [ ] Develop codes/docs for selecting model
- [ ] Develop codes/docs for applying model to make label imagery
- [ ] Tool for mosaicing labels
- [ ] Tool for downloading labels in geotiff format
V2
- [ ] Tool for post-processing/editing labels
- [ ] Tool for detecting change
- [ ] Make Planetscope 3m imagery available via Planet API (federal researchers only)
- [ ] Include additional models/datasets (TBD)
Datasets
General Landcover
DeepGlobe
EnviroAtlas
OpenEarthMap
Coastal Landcover
Chesapeake Landcover
- webpage
- Zenodo model release (512x512): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Res-UNet models for Chesapeake/7-class segmentation of RGB 512x512 high-res. images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7576904
- Zenodo SegFormer model release (512x512): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Segformer models for Chesapeake/7-class segmentation of RGB 512x512 high-res. images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7677506
Coast Train
- paper
- website
- data
- preprint
- Zenodo model release, 2-class (768x768): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Res-UNet models for CoastTrain water/other segmentation of RGB 768x768 orthomosaic images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7574784
- Zenodo model release, 5-class (768x768): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Res-UNet models for CoastTrain/5-class segmentation of RGB 768x768 NAIP images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7566992
- Zenodo model release, 8-class (768x768): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Res-UNet models for CoastTrain/8-class segmentation of RGB 768x768 NAIP images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7570583
- Zenodo SegFormer model release, 5-class (768x768): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map SegFormer models for CoastTrain/5-class segmentation of RGB 768x768 NAIP images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7641708
- Zenodo SegFormer model release, 8-class (768x768): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map SegFormer models for CoastTrain/8-class segmentation of RGB 768x768 NAIP images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7641724
AAAI / Buildings / Flooded Buildings
- data
- data
- paper
- Zenodo model release (1024x1024) building / no building: Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Res-UNet models for segmentation of buildings of RGB 1024x1024 high-res. images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7607895
- Zenodo model release (1024x1024) flooded building / no flooded building: Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Res-UNet models for segmentation of AAAI/flooded buildings in RGB 1024x1024 high-res. images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7622733
XBD-hurricanes
- Xview2 challenge
- XBD-hurricanes code
- Zenodo SegFormer model release (768x768) building damage: Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map SegFormer models for segmentation of xBD/damaged buildings in RGB 768x768 high-res. images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7613175
- Zenodo SegFormer model release (768x768) building presence/absence: Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map SegFormer models for segmentation of xBD/buildings in RGB 768x768 high-res. images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7613212
Barrier Islands
- webpage
- paper
- Zenodo Substrate model release (768x768):
- Zenodo Vegetation type model release (768x768):
- Zenodo Vegetation density model release (768x768):
- Zenodo Geomorphic model release (768x768):
- Zenodo Supervised classification model release (768x768):
Superclasses
A. Water:
- Coast Train
- Chesapeake
- EnviroAtlas
- OpenEarthMap
- DeepGlobe
- Barrier Substrate
- NOAA
- [v2: Barrier Substrate]
- [v2: Elwha]
B. Sediment:
- Coast Train
- NOAA
- [v2: Barrier Substrate (sand, mixed, coarse)]
- [v2: Elwha]
C. Bare:
- Chesapeake (barren land)
- EnviroAtlas (barren)
- OpenEarthMap (bareland)
D. Vegetated:
- Coast Train (marsh veg, terrestrial veg, ag)
- FloodNet (tree, grass)
- Chesapeake (tree canopy / forest, low vegetation / field)
- EnviroAtlas (trees, herbaceous, shrubland)
- OpenEarthMap (rangeland, tree, ag)
- DeepGlobe (ag., rangeland, forest)
- NOAA (veg)
- [v2: Elwha]
- [v2: Barrier Substrate]
E. Impervious:
- FloodNet (Building-flooded, Building-non-flooded, Road-flooded, Road-non-flooded, vehicle)
- Chesapeake (impervious (other), impervious (road))
- EnviroAtlas (impervious)
- OpenEarthMap (dev, road, building)
- DeepGlobe (urban)
- NOAA (dev)
- [v2: Elwha]
F. Building:
- OpenEarthMap (building)
- AAAI (building)
G. Agriculture:
- OpenEarthMap (ag)
- DeepGlobe (ag)
H. Woody Veg:
- FloodNet (tree)
- Chesapeake (tree canopy / forest)
- EnviroAtlas (trees)
- OpenEarthMap (tree)
- DeepGlobe (forest)
- [v2: Elwha]
- [v2: Barrier Substrate]
References
Notes
Classes:
|
Coast Train 1 |
Coast Train 2 |
Coast Train 3 |
FloodNet |
Chesapeake |
EnviroAtlas |
OpenEarthMap |
DeepGlobe |
AAAI |
NOAA |
Barrier Substrate |
A. Water |
X |
X |
X |
X |
X |
X |
X |
X |
|
X |
X |
a. whitewater |
|
X |
X |
|
|
|
|
|
|
|
|
a. pool |
|
|
|
X |
|
|
|
|
|
|
|
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
B. Sediment |
|
X |
X |
|
|
|
|
|
|
X |
|
b. sand |
|
|
|
|
|
|
|
|
|
|
X |
b. mixed |
|
|
|
|
|
|
|
|
|
|
X |
b. coarse |
|
|
|
|
|
|
|
|
|
|
X |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
C. Bare/barren |
|
X |
X |
|
X |
X |
X |
X |
|
|
|
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
d. marsh |
|
|
X |
|
|
|
|
|
|
|
|
d. terrestrial veg |
|
|
X |
|
|
|
|
|
|
X |
|
d. agriculture |
|
|
X |
|
|
|
X |
X |
|
|
|
d. grass |
|
|
|
X |
|
|
|
|
|
|
|
d. herbaceous / low vegetation / field |
|
|
|
|
X |
X |
|
|
|
|
|
d. tree/forest |
|
|
|
X |
X |
X |
X |
X |
|
|
|
d. shrubland |
|
|
|
|
|
X |
|
|
|
|
|
d. rangeland |
|
|
|
|
|
X |
X |
X |
|
|
|
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
E. Impervious/urban/developed |
|
|
X |
|
|
X |
X |
X |
|
X |
X |
e. impervious (other) |
|
|
|
|
X |
|
|
|
|
|
|
e. impervious (road) |
|
|
|
|
X |
|
X |
|
|
|
|
e. Building-flooded |
|
|
|
X |
|
|
|
|
|
|
|
e. Building-non-flooded |
|
|
|
X |
|
|
X |
|
X |
|
|
e. Road-flooded |
|
|
|
X |
|
|
|
|
|
|
|
e. Road-non-flooded |
|
|
|
X |
|
|
|
|
|
|
|
e. Vehicle |
|
|
|
X |
|
|
|
|
|
|
|
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
--- |
X. Other |
X |
X |
|
|
|
|
|
|
X |
|
|