Open mdsumner opened 4 months ago
trickle job was set too short, but had these issues
ERROR 4: vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210220090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25' does not exist in the file system, and is not recognized as a supported dataset name. input source not readable: vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210220090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25 Error : [1] "failed to read/write vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210220090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25\n/scratch/pawsey0973/mdsumner/data/idea/archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/2021/02/20210220090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.tif" ERROR 4:
vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210221090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25' does not exist in the file system, and is not recognized as a supported dataset name.
input source not readable: vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210221090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25
Error :
[1] "failed to read/write vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210221090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25\n/scratch/pawsey0973/mdsumner/data/idea/archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/2021/02/20210221090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.tif"
[1] 6900
[1] 7000
[1] 7100
[1] 7200
[1] 7300
[1] 7400
ERROR 4: `vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20221109090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25' does not exist in the file system, and is not recognized as a supported dataset name.
input source not readable: vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20221109090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25
Error :
[1] "failed to read/write vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20221109090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25\n/scratch/pawsey0973/mdsumner/data/idea/archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/2022/11/20221109090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.tif"
[1] 7500
here's the final trickle report
ERROR 4: vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210220090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25' does not exist in the file system, and is not recognized as a supported dataset name. input source not readable: vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210220090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25 Error : [1] "failed to read/write vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210220090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25\n/scratch/pawsey0973/mdsumner/data/idea/archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/2021/02/20210220090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.tif" ERROR 4:
vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210221090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25' does not exist in the file system, and is not recognized as a supported dataset name.
input source not readable: vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210221090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25
Error :
[1] "failed to read/write vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210221090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25\n/scratch/pawsey0973/mdsumner/data/idea/archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/2021/02/20210221090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.tif"
[1] 6900
[1] 7000
[1] 7100
[1] 7200
[1] 7300
[1] 7400
ERROR 4: vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20221109090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25' does not exist in the file system, and is not recognized as a supported dataset name. input source not readable: vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20221109090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25 Error : [1] "failed to read/write vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20221109090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25\n/scratch/pawsey0973/mdsumner/data/idea/archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/2022/11/20221109090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.tif" [1] 7500 [1] 7600 [1] 7700 [1] 7800 [1] 7900 ERROR 4:
vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20240225090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25' does not exist in the file system, and is not recognized as a supported dataset name.
input source not readable: vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20240225090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25
Error :
[1] "failed to read/write vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20240225090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25\n/scratch/pawsey0973/mdsumner/data/idea/archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/2024/02/20240225090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.tif"
ERROR 4: `vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20240226090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25' does not exist in the file system, and is not recognized as a supported dataset name.
input source not readable: vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20240226090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25
Error :
[1] "failed to read/write vrt:///vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20240226090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc?sd_name=analysed_sst&a_srs=EPSG:4326&a_scale=0.001&a_offset=25\n/scratch/pawsey0973/mdsumner/data/idea/archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/2024/02/20240226090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.tif"
there are 7936 files, but expecting 7938 (to 2024-02-24, note the viritual folder is listed as 2024-02-23 online)
find /scratch/pawsey0973/mdsumner/data/idea/archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/ -type f | grep tif | wc -w 7936