Open emilyselwood opened 5 years ago
This appears to be something wrong with my environment. I've cleaned up a lot of old containers and it is now running better...
PS C:\git\grfn-s1tbx-rtc> docker run -it -v "/data/asf-test:/output" --rm asfdaac/s1tbx-rtc "--granule" "S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B"
Earthdata Login username: wil.selwood@sa.catapult.org.uk
Earthdata Login password:
Fetching granule information
Downloading granule from https://datapool.asf.alaska.edu/GRD_HD/SB/S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B.zip
Preparing digital elevation model
Snapping to grid at posting of 30 meters
Total ned13 4.251283331218865 5.153048980215999 0.8250034780458587
Total srtmgl1 4.244142131781835 5.153048980215999 0.8236176578325353
Total srtmau1 0 5.153048980215999 0.0
Total ned1 4.270356053122839 5.153048980215999 0.828704727922815
Total ned2 0 5.153048980215999 0.0
Total srtmgl3 4.24890249734856 5.153048980215999 0.8245414537415205
NED13
['n19w156', 'n20w155', 'n20w156', 'n20w157', 'n21w156', 'n21w157', 'n21w158']
0...10...20...30...40...50...60...70...80...90...100 - done.
Creating initial raster file
Converting to WGS84
Converting to pixel as point
Translating raster file to UTM coordinates
Snapping file to grid at 30 meters
New coordinates: 359280.000000 76350.000000 2288460.000000 2069790.000000
gdal_translate -ot Int16 temp_dem NED13
Input file size is 9431, 7289
0...10...20...30...40...50...60...70...80...90...100 - done.
Apply-Orbit-File
gpt Apply-Orbit-File -Ssource=S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B.zip -t Apply-Orbit-File
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.hsqldb.persist.Logger: dataFileCache open start
INFO: org.esa.snap.engine_utilities.download.downloadablecontent.DownloadableContentImpl: http retrieving http://step.esa.int/auxdata/orbits/Sentinel-1/POEORB/S1B/2019/04/S1B_OPER_AUX_POEORB_OPOD_20190520T110553_V20190429T225942_20190501T005942.EOF.zip
INFO: org.esa.snap.core.gpf.common.WriteOp: Start writing product S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B_Orb to Apply-Orbit-File
Writing...
Appears to have run but I get no outputs in the folder defined...
PS C:\git\grfn-s1tbx-rtc> docker run -it -v "/data/asf-test:/output" --rm asfdaac/s1tbx-rtc "--granule" "S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B"
Earthdata Login username: wil.selwood@sa.catapult.org.uk
Earthdata Login password:
Fetching granule information
Downloading granule from https://datapool.asf.alaska.edu/GRD_HD/SB/S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B.zip
Preparing digital elevation model
Snapping to grid at posting of 30 meters
Total ned13 4.251283331218865 5.153048980215999 0.8250034780458587
Total srtmgl1 4.244142131781835 5.153048980215999 0.8236176578325353
Total srtmau1 0 5.153048980215999 0.0
Total ned1 4.270356053122839 5.153048980215999 0.828704727922815
Total ned2 0 5.153048980215999 0.0
Total srtmgl3 4.24890249734856 5.153048980215999 0.8245414537415205
NED13
['n19w156', 'n20w155', 'n20w156', 'n20w157', 'n21w156', 'n21w157', 'n21w158']
0...10...20...30...40...50...60...70...80...90...100 - done.
Creating initial raster file
Converting to WGS84
Converting to pixel as point
Translating raster file to UTM coordinates
Snapping file to grid at 30 meters
New coordinates: 359280.000000 76350.000000 2288460.000000 2069790.000000
gdal_translate -ot Int16 temp_dem NED13
Input file size is 9431, 7289
0...10...20...30...40...50...60...70...80...90...100 - done.
Apply-Orbit-File
gpt Apply-Orbit-File -Ssource=S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B.zip -t Apply-Orbit-File
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.hsqldb.persist.Logger: dataFileCache open start
INFO: org.esa.snap.engine_utilities.download.downloadablecontent.DownloadableContentImpl: http retrieving http://step.esa.int/auxdata/orbits/Sentinel-1/POEORB/S1B/2019/04/S1B_OPER_AUX_POEORB_OPOD_20190520T110553_V20190429T225942_20190501T005942.EOF.zip
INFO: org.esa.snap.core.gpf.common.WriteOp: Start writing product S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B_Orb to Apply-Orbit-File
Writing...
....10%....20%....30%....40%....50%....60%....70%....80%....90%....100% done.
INFO: org.esa.snap.core.gpf.common.WriteOp: End writing product Apply-Orbit-File to Apply-Orbit-File
INFO: org.esa.snap.core.gpf.common.WriteOp: Time: 66.520 s total, 3.965 ms per line, 0.000157 ms per pixel
Calibration
gpt Calibration -Ssource=Apply-Orbit-File.dim -t Calibration -PoutputBetaBand=true -PoutputSigmaBand=false
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.hsqldb.persist.Logger: dataFileCache open start
INFO: org.esa.snap.core.gpf.common.WriteOp: Start writing product Apply-Orbit-File_Cal to Calibration
Writing...
....10%....20%....30%....40%....50%....60%....70%....80%....90%....100% done.
INFO: org.esa.snap.core.gpf.common.WriteOp: End writing product Calibration to Calibration
INFO: org.esa.snap.core.gpf.common.WriteOp: Time: 154.548 s total, 9.212 ms per line, 0.000366 ms per pixel
Speckle-Filter
gpt Speckle-Filter -Ssource=Calibration.dim -t Speckle-Filter -Pfilter=Lee Sigma
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.hsqldb.persist.Logger: dataFileCache open start
INFO: org.esa.snap.core.gpf.common.WriteOp: Start writing product Calibration_Spk to Speckle-Filter
Writing...
....10%....20%....30%....40%....50%....60%....70%....80%....90%....100% done.
INFO: org.esa.snap.core.gpf.common.WriteOp: End writing product Speckle-Filter to Speckle-Filter
INFO: org.esa.snap.core.gpf.common.WriteOp: Time: 1120.068 s total, 66.766 ms per line, 0.002650 ms per pixel
Multilook
gpt Multilook -Ssource=Speckle-Filter.dim -t Multilook -PnRgLooks=3 -PnAzLooks=3
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.hsqldb.persist.Logger: dataFileCache open start
INFO: org.esa.snap.core.gpf.common.WriteOp: Start writing product Speckle-Filter_ML to Multilook
Writing...
....10%....20%....30%....40%....50%....60%....70%....80%....90%....100% done.
INFO: org.esa.snap.core.gpf.common.WriteOp: End writing product Multilook to Multilook
INFO: org.esa.snap.core.gpf.common.WriteOp: Time: 65.240 s total, 11.667 ms per line, 0.001389 ms per pixel
Terrain-Flattening
gpt Terrain-Flattening -Ssource=Multilook.dim -t Terrain-Flattening -PreGridMethod=False -PdemName='External DEM' -PexternalDEMFile=NED13 -PexternalDEMNoDataValue=-32767
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.hsqldb.persist.Logger: dataFileCache open start
INFO: org.esa.snap.core.gpf.common.WriteOp: Start writing product Multilook_TF to Terrain-Flattening
Writing...
....10%....20%....30%....40%....50%....60%....70%....80%....90%....100% done.
INFO: org.esa.snap.core.gpf.common.WriteOp: End writing product Terrain-Flattening to Terrain-Flattening
INFO: org.esa.snap.core.gpf.common.WriteOp: Time: 210.516 s total, 37.646 ms per line, 0.004483 ms per pixel
Terrain-Correction
gpt Terrain-Correction -Ssource=Terrain-Flattening.dim -t Terrain-Correction -PpixelSpacingInMeter=30.0 -PmapProjection=AUTO:42001 -PsaveProjectedLocalIncidenceAngle=False -PdemName='External DEM' -PexternalDEMFile=NED13 -PexternalDEMNoDataValue=-32767
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.hsqldb.persist.Logger: dataFileCache open start
INFO: org.esa.snap.core.gpf.common.WriteOp: Start writing product Terrain-Flattening_TC to Terrain-Correction
Writing...
....10%....20%....30%....40%....50%....60%....70%....80%....90%....100% done.
INFO: org.esa.snap.core.gpf.common.WriteOp: End writing product Terrain-Correction to Terrain-Correction
INFO: org.esa.snap.core.gpf.common.WriteOp: Time: 168.219 s total, 23.091 ms per line, 0.002449 ms per pixel
Creating output file
gdal_translate -of GTiff -a_nodata 0 Terrain-Correction.data/Gamma0_VH.img temp.tif
Input file size is 9427, 7285
0...10...20...30...40...50...60...70...80...90...100 - done.
gdaladdo -r average temp.tif 2 4 8 16
0...10...20...30...40...50...60...70...80...90...100 - done.
gdal_translate -co TILED=YES -co COMPRESS=DEFLATE -co COPY_SRC_OVERVIEWS=YES temp.tif /output/S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B_VH_RTC.tif
Input file size is 9427, 7285
0...10...20...30...40...50...60...70...80...90...100 - done.
Creating output file
gdal_translate -of GTiff -a_nodata 0 Terrain-Correction.data/Gamma0_VV.img temp.tif
Input file size is 9427, 7285
0...10...20...30...40...50...60...70...80...90...100 - done.
gdaladdo -r average temp.tif 2 4 8 16
0...10...20...30...40...50...60...70...80...90...100 - done.
gdal_translate -co TILED=YES -co COMPRESS=DEFLATE -co COPY_SRC_OVERVIEWS=YES temp.tif /output/S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B_VV_RTC.tif
Input file size is 9427, 7285
0...10...20...30...40...50...60...70...80...90...100 - done.
Preparing arcgis xml file /output/S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B_VV_RTC.tif.xml.
Preparing arcgis xml file /output/S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B_VH_RTC.tif.xml.
PS C:\git\grfn-s1tbx-rtc>
We don't officially support running on Windows, but docker's -v
flag controls where your output files will end up. Can you verify that /data/asf-test
is a valid directory on your Windows system, and that Docker has access to it?
Thanks @asjohnston-asf that pointed to the problem... need the "c:" on the front of the volume path. and with docker only having 8Gb of ram.
PS C:\data\asf-test> docker run -itv "c:/data/asf-test:/output" --rm asfdaac/s1tbx-rtc "--granule" "S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B"
Earthdata Login username: wil.selwood@sa.catapult.org.uk
Earthdata Login password:
Fetching granule information
Downloading granule from https://datapool.asf.alaska.edu/GRD_HD/SB/S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B.zip
Preparing digital elevation model
Snapping to grid at posting of 30 meters
Total ned13 4.251283331218865 5.153048980215999 0.8250034780458587
Total srtmgl1 4.244142131781835 5.153048980215999 0.8236176578325353
Total srtmau1 0 5.153048980215999 0.0
Total ned1 4.270356053122839 5.153048980215999 0.828704727922815
Total ned2 0 5.153048980215999 0.0
Total srtmgl3 4.24890249734856 5.153048980215999 0.8245414537415205
NED13
['n19w156', 'n20w155', 'n20w156', 'n20w157', 'n21w156', 'n21w157', 'n21w158']
0...10...20...30...40...50...60...70...80...90...100 - done.
Creating initial raster file
Converting to WGS84
Converting to pixel as point
Translating raster file to UTM coordinates
Snapping file to grid at 30 meters
New coordinates: 359280.000000 76350.000000 2288460.000000 2069790.000000
gdal_translate -ot Int16 temp_dem NED13
Input file size is 9431, 7289
0...10...20...30...40...50...60...70...80...90...100 - done.
Apply-Orbit-File
gpt Apply-Orbit-File -Ssource=S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B.zip -t Apply-Orbit-File
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.hsqldb.persist.Logger: dataFileCache open start
INFO: org.esa.snap.engine_utilities.download.downloadablecontent.DownloadableContentImpl: http retrieving http://step.esa.int/auxdata/orbits/Sentinel-1/POEORB/S1B/2019/04/S1B_OPER_AUX_POEORB_OPOD_20190520T110553_V20190429T225942_20190501T005942.EOF.zip
INFO: org.esa.snap.core.gpf.common.WriteOp: Start writing product S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B_Orb to Apply-Orbit-File
Writing...
....10%....20%....30%....40%....50%....60%....70%....80%....90%....100% done.
INFO: org.esa.snap.core.gpf.common.WriteOp: End writing product Apply-Orbit-File to Apply-Orbit-File
INFO: org.esa.snap.core.gpf.common.WriteOp: Time: 66.417 s total, 3.959 ms per line, 0.000157 ms per pixel
Calibration
gpt Calibration -Ssource=Apply-Orbit-File.dim -t Calibration -PoutputBetaBand=true -PoutputSigmaBand=false
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.hsqldb.persist.Logger: dataFileCache open start
INFO: org.esa.snap.core.gpf.common.WriteOp: Start writing product Apply-Orbit-File_Cal to Calibration
Writing...
....10%....20%....30%....40%....50%....60%....70%....80%....90%.... done.
INFO: org.esa.snap.core.gpf.common.WriteOp: End writing product Calibration to Calibration
INFO: org.esa.snap.core.gpf.common.WriteOp: Time: 169.836 s total, 10.124 ms per line, 0.000402 ms per pixel
Speckle-Filter
gpt Speckle-Filter -Ssource=Calibration.dim -t Speckle-Filter -Pfilter=Lee Sigma
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.hsqldb.persist.Logger: dataFileCache open start
INFO: org.esa.snap.core.gpf.common.WriteOp: Start writing product Calibration_Spk to Speckle-Filter
Writing...
....10%....20%....30%....40%....50%....60%....70%....80%....90%....100% done.
INFO: org.esa.snap.core.gpf.common.WriteOp: End writing product Speckle-Filter to Speckle-Filter
INFO: org.esa.snap.core.gpf.common.WriteOp: Time: 1047.085 s total, 62.416 ms per line, 0.002478 ms per pixel
Multilook
gpt Multilook -Ssource=Speckle-Filter.dim -t Multilook -PnRgLooks=3 -PnAzLooks=3
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.hsqldb.persist.Logger: dataFileCache open start
INFO: org.esa.snap.core.gpf.common.WriteOp: Start writing product Speckle-Filter_ML to Multilook
Writing...
....10%....20%....30%....40%....50%....60%....70%....80%....90%....100% done.
INFO: org.esa.snap.core.gpf.common.WriteOp: End writing product Multilook to Multilook
INFO: org.esa.snap.core.gpf.common.WriteOp: Time: 61.078 s total, 10.922 ms per line, 0.001301 ms per pixel
Terrain-Flattening
gpt Terrain-Flattening -Ssource=Multilook.dim -t Terrain-Flattening -PreGridMethod=False -PdemName='External DEM' -PexternalDEMFile=NED13 -PexternalDEMNoDataValue=-32767
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.hsqldb.persist.Logger: dataFileCache open start
INFO: org.esa.snap.core.gpf.common.WriteOp: Start writing product Multilook_TF to Terrain-Flattening
Writing...
....10%....20%....30%....40%....50%....60%....70%....80%....90%....100% done.
INFO: org.esa.snap.core.gpf.common.WriteOp: End writing product Terrain-Flattening to Terrain-Flattening
INFO: org.esa.snap.core.gpf.common.WriteOp: Time: 210.324 s total, 37.612 ms per line, 0.004479 ms per pixel
Terrain-Correction
gpt Terrain-Correction -Ssource=Terrain-Flattening.dim -t Terrain-Correction -PpixelSpacingInMeter=30.0 -PmapProjection=AUTO:42001 -PsaveProjectedLocalIncidenceAngle=False -PdemName='External DEM' -PexternalDEMFile=NED13 -PexternalDEMNoDataValue=-32767
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.hsqldb.persist.Logger: dataFileCache open start
INFO: org.esa.snap.core.gpf.common.WriteOp: Start writing product Terrain-Flattening_TC to Terrain-Correction
Writing...
....10%....20%....30%....40%....50%....60%....70%....80%....90%....100% done.
INFO: org.esa.snap.core.gpf.common.WriteOp: End writing product Terrain-Correction to Terrain-Correction
INFO: org.esa.snap.core.gpf.common.WriteOp: Time: 155.585 s total, 21.357 ms per line, 0.002265 ms per pixel
Creating output file
gdal_translate -of GTiff -a_nodata 0 Terrain-Correction.data/Gamma0_VH.img temp.tif
Input file size is 9427, 7285
0...10...20...30...40...50...60...70...80...90...100 - done.
gdaladdo -r average temp.tif 2 4 8 16
0...10...20...30...40...50...60...70...80...90...100 - done.
gdal_translate -co TILED=YES -co COMPRESS=DEFLATE -co COPY_SRC_OVERVIEWS=YES temp.tif /output/S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B_VH_RTC.tif
Input file size is 9427, 7285
0...10...20...30...40...50...60...70...80...90...100 - done.
Creating output file
gdal_translate -of GTiff -a_nodata 0 Terrain-Correction.data/Gamma0_VV.img temp.tif
Input file size is 9427, 7285
0...10...20...30...40...50...60...70...80...90...100 - done.
gdaladdo -r average temp.tif 2 4 8 16
0...10...20...30...40...50...60...70...80...90...100 - done.
gdal_translate -co TILED=YES -co COMPRESS=DEFLATE -co COPY_SRC_OVERVIEWS=YES temp.tif /output/S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B_VV_RTC.tif
Input file size is 9427, 7285
0...10...20...30...40...50...60...70...80...90...100 - done.
Preparing arcgis xml file /output/S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B_VH_RTC.tif.xml.
Preparing arcgis xml file /output/S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B_VV_RTC.tif.xml.
PS C:\data\asf-test> ls
Directory: C:\data\asf-test
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 27/06/2019 13:49 78562182 S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B_VH_RTC.tif
-a---- 27/06/2019 13:49 10736 S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B_VH_RTC.tif.xml
-a---- 27/06/2019 13:49 78748096 S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B_VV_RTC.tif
-a---- 27/06/2019 13:49 10736 S1B_IW_GRDH_1SDV_20190430T161529_20190430T161554_016038_01E295_771B_VV_RTC.tif.xml
-a---- 24/06/2019 10:44 68914 s1tbx-rtc.sh
Hopefully this will be helpful when/if you are going to support windows. Any questions yell...
Using Docker for windows. Translated the run script into commands
run result:
I've got the disk image max size set to 64Gb, though my laptop doesn't have enough ram to give it more memory. I'll try later on my desktop with more ram and update this issue with the result of that run.