search

Applied-GeoSolutions / gips

Geospatial Image Processing System
GNU General Public License v3.0
17 stars 5 forks source link

PRISM fetches when data are already in the archive #252

Open bhbraswell opened 7 years ago

bhbraswell commented 7 years ago

PRISM fetches when data are already in the archive

(.venv) (screen)braswell@green:~/repo/gips$ gips_inventory prism -p tmin -d 2016-12-31 -t conus -v4 --fetch
GIPS Data Inventory (v0.9.1-dev)
Retrieving inventory for site tiles
_tmin: fetch tile conus for 2016-12-31 00:00:00
Downloading PRISM_tmin_provisional_4kmD1_20161231_bil.zip
/data/gips/prism/stage/prismDownloaderO0ohMy/PRISM_tmin_provisional_4kmD1_20161231_bil.zip already in archive
1 files not added to archive

Asset Holdings for tile conus
No matching files in inventory

SENSORS
prism: Daily Gridded Climate Data
bhbraswell commented 7 years ago

Also gips_process doesn't seem to do anything, and gips_project fails.

(.venv) (screen)braswell@green:~/repo/gips$ gips_project prism -p tmin -d 2016-12-31 -s gips/test/NHseacoast.shp -v4 --alltouch --fetch
GIPS Data Project (v0.9.1-dev)
Retrieving inventory for site NHseacoast-0
_tmin: fetch tile CONUS for 2016-12-31 00:00:00
Downloading PRISM_tmin_provisional_4kmD1_20161231_bil.zip
/data/gips/prism/stage/prismDownloaderDiUsaX/PRISM_tmin_provisional_4kmD1_20161231_bil.zip already in archive
1 files not added to archive
No data found for 2016-12-31 - 2016-12-31 (days 1-366) within temporal extent 2016-12-31 - 2016-12-31 (days 1-366)
ircwaves commented 7 years ago

Have you gips_inventory prism --rectify ed?

If yes, then try running after running:

export GIPS_ORM=False
ircwaves commented 7 years ago

note to self: add notes on this

ircwaves commented 7 years ago

Fixing #305 may squash this.

ra-tolson commented 7 years ago

Looks like post-#305 this problem persists (note v0.8.2 is due to my GIPS_OVERRIDE_VERSION). Here's a run with an asset showing in inventory, then attemping to fetch the same asset right afterwards; I confirmed it's downloading by watching the stage while the command was running:

(.venv) tolsonlt:~/src/gips$ gips_inventory prism -v4
GIPS Data Inventory (v0.8.2)
Retrieving inventory for site tiles

Asset Holdings for tile CONUS
    DATE       _ppt     _tmax     _tmin    Product  
2016        
    366                           100.0%   

1 files on 1 dates

SENSORS
prism: Daily Gridded Climate Data
(.venv) tolsonlt:~/src/gips$ time gips_inventory prism -p tmin -d 2016-12-31 -t conus -v4 --fetch
GIPS Data Inventory (v0.8.2)
Retrieving inventory for site tiles
_tmin: fetch tile conus for 2016-12-31 00:00:00
Downloading PRISM_tmin_stable_4kmD1_20161231_bil.zip
/home/tolson/src/gips/data-root/prism/stage/PRISM_tmin_stable_4kmD1_20161231_bil.zip already in archive
1 files not added to archive

Asset Holdings for tile conus
No matching files in inventory

SENSORS
prism: Daily Gridded Climate Data

real    0m4.842s
user    0m0.580s
sys     0m0.064s