coolzhao / Geo-SAM

A QGIS plugin tool using Segment Anything Model (SAM) to accelerate segmenting or delineating landforms in geospatial raster images.
MIT License
199 stars 26 forks source link

Changing the resolution results in a high batch/sample size - dev #13

Open 4del-Yousefi opened 11 months ago

4del-Yousefi commented 11 months ago

Hello, Thanks for keeping us updated and fixing the issues, I'm using the Copilot tool with the following settings – I copied and pasted them into the encoder.

image

It resulted in a very high batch/sample size:

SAM model initialized. SAM model type: vit_b Device type: cuda:0 on NVIDIA GeForce GTX 1070 Ti Patch size: (1024, 1024) Batch size: 1 -> Patch sample num: 27559879364 -> Total batch num: 27559879364

The selected area is this: image

Fanchengyan commented 11 months ago

Hi @4del-Yousefi . Can you please share all the logs with us? We are unable to pinpoint the cause of the error based on the description you provided. Additionally, we have released a Python package called GeoSAM-Image-Encoder that does not rely on QGIS. If you're interested, you can also try it out in Colab. https://github.com/coolzhao/Geo-SAM/blob/dev/GeoSAM-Image-Encoder/examples/geosam-image-encoder.ipynb

4del-Yousefi commented 11 months ago

Hey @Fanchengyan, I have the logs for that run, I also I will try and use the package, thank you!:

QGIS version: 3.30.0-'s-Hertogenbosch QGIS code revision: f186b8efe0e Qt version: 5.15.3 Python version: 3.9.5 GDAL version: 3.6.2 GEOS version: 3.11.1-CAPI-1.17.1 PROJ version: Rel. 9.1.1, December 1st, 2022 PDAL version: 2.4.3 (git-version: f8d673) Algorithm started at: 2023-08-07T15:16:56 Algorithm 'Geo-SAM Image Encoder' starting… Input parameters: { 'BANDS' : [1,2,3], 'BATCH_SIZE' : 1, 'CKPT' : 'C:\Users\Adel\Desktop\SAMTOOL\Models\sam_vit_b_01ec64.pth', 'CRS' : QgsCoordinateReferenceSystem('EPSG:4326'), 'CUDA' : True, 'CUDA_ID' : 0, 'EXTENT' : '55.203142664,55.204045383,25.109473253,25.110170752 [EPSG:4326]', 'INPUT' : 'C:/Users/Adel/Downloads/P1/P1/testArea.tif', 'LOAD' : True, 'MODEL_TYPE' : 2, 'OUTPUT' : 'TEMPORARY_OUTPUT', 'RANGE' : [0,255], 'RESOLUTION' : 1e-06, 'STRIDE' : 512 }

Input data value range to be rescaled: [0.0, 255.0] (set by user) Layer path: C:/Users/Adel/Downloads/P1/P1/testArea.tif Layer name: testArea Layer CRS: EPSG:4326 Layer pixel size: 1.5328381745825362e-07, 1.5328383422480063e-07 degrees Bands selected: [1, 2, 3] Target CRS: EPSG:32640 Target resolution: 1e-06 meters Processing extent: minx:318821.500314, maxx:318913.569375,miny:2778274.272565, maxy:2778352.742931 Processing image size: (width 92069062, height 78470367)

RasterDataset info: filename_glob: testArea.tif, all bands: ['Band 1', 'Band 2', 'Band 3', 'Band 4'], input bands: ['Band 1', 'Band 2', 'Band 3'], resolution: 1e-06, bounds: [318366.0378347719, 319624.7036324028, 2777874.8307066844, 2778773.8526859754, 0.0, 9.223372036854776e+18], num: 1

SAM model initialized. SAM model type: vit_b Device type: cuda:0 on NVIDIA GeForce GTX 1070 Ti Patch size: (1024, 1024) Batch size: 1 Patch sample num: 27559879364 Total batch num: 27559879364

Batch no. 1 loaded img_shape: (3, 1024, 1024) patch_size: torch.Size([1, 3, 1024, 1024])

// After that it processes them one by one...

Fanchengyan commented 11 months ago

Hi @4del-Yousefi . We have found the reason. The resolution exported by Encoder Copilot is in the units of the image projection (in degrees, as in your example). However, Image Encoder currently only supports resolution in meters. We suggest that you estimate the resolution of the image in meters yourself and multiply it by the resolution scale provided in Encoder Copilot. Then manually enter the calculated resolution into Image Encoder. In the future, we will consider how to address this issue.

image

4del-Yousefi commented 11 months ago

Thanks, I am not sure how to do the calculation, but will try to figure it out! also small note, when i use the night theme in QGIS the Copilot menu doesnt show currectly,

image

coolzhao commented 11 months ago

Hi @4del-Yousefi, thanks for your feedback. Based on your circumstances, I suggest reprojecting your image into UTM CRS (in meter units). We will continue to improve our tool to handle images with the degree units better.

BTW, by using the default settings of the Encoding Tool, you can find the native resolution of your image in meters from the log information. This can help you decide the target resolution.

Fanchengyan commented 11 months ago

Hi @4del-Yousefi ,If you are unsure about the resolution of your image in meters, you can first reproject the image into a coordinate system with units in meters, such as UTM. After the reprojection, Encoder Copilot will provide the resolution in meters.

4del-Yousefi commented 11 months ago

image

changing the coordinate system fixed the issue, however there is a bug with the "Draw on canvas" tool in the copilot, when I reuse it to redraw the area it leaves an artifact of the old selection.

Fanchengyan commented 11 months ago

Hi @4del-Yousefi, it is just due to the bugs for not clearing the canvas properly in the dev version. This will disappear when you restart QGIS. We will fix this in the stable version. Thanks for your feedback.