Nowcast test error: relation "raster_resampled" does not exist

[kashingtonDC]

Hi,

I'm trying to run the nowcast test from the documentation [http://rheas.readthedocs.org/en/latest/nowcast.html]

Here are the contents of my conf file: [nowcast] model: vic, dssat startdate: 2003-1-1 enddate: 2003-3-31 basin: data/tests/basin.shp name: basin resolution: 0.25

[vic] exe: bin/vicNl precip: chirps temperature: ncep wind: ncep initialize: yes initial state: state/vic.state_20030101 save to: db save: net_short, soil_moist

[dssat] shapefile: data/tests/basin.shp exe: bin/dssat.exe ensemble size: 50

Here is the full error when running the command ./bin/rheas nowcast.conf : Traceback (most recent call last): File "./bin/rheas", line 15, in <module> sys.exit(rheas.run()) File "/Users/macbookpro/RHEAS/src/rheas.py", line 81, in run nowcast.execute(dbname, options) File "/Users/macbookpro/RHEAS/src/nowcast.py", line 187, in execute runVIC(dbname, options) File "/Users/macbookpro/RHEAS/src/nowcast.py", line 29, in runVIC runDeterministicVIC(dbname, options) File "/Users/macbookpro/RHEAS/src/nowcast.py", line 75, in runDeterministicVIC prec, tmax, tmin, wind = model.getForcings(options['vic']) File "/Users/macbookpro/RHEAS/src/vic/vic.py", line 353, in getForcings rtables[v] = self.createIndexTable("{0}.{1}".format(v, options[v])) File "/Users/macbookpro/RHEAS/src/vic/vic.py", line 294, in createIndexTable "select * from raster_resampled where sname='{0}' and tname like '{1}_%' and resolution={2}".format(sname, tname, self.res)) psycopg2.ProgrammingError: relation "raster_resampled" does not exist LINE 1: select * from raster_resampled where sname='precip' and tnam...

I also tried updating the database with ./bin/rheas -u data.conf

I'm fairly new to pgsql databases so apologies if I am missing something or this is a straightforward fix

[AdamJDuncan]

have you populated the db with chirps & ncep information for that shapefile area and daterange? need to do that first. (the data.conf step)

[kashingtonDC]

Ah, got it, thanks. I populated the database by adding parameters to data.conf and running ./bin/rheas -u data.conf. Then updated the nowcast.conf and nowcast.in files to reflect changes. This solved the above error.

However now when I run ./bin/rheas nowcast.conf I get:

File "/Users/macbookpro/RHEAS/bin/rheas", line 15, in <module> sys.exit(rheas.run()) File "/Users/macbookpro/RHEAS/src/rheas.py", line 81, in run nowcast.execute(dbname, options) File "/Users/macbookpro/RHEAS/src/nowcast.py", line 187, in execute runVIC(dbname, options) File "/Users/macbookpro/RHEAS/src/nowcast.py", line 29, in runVIC runDeterministicVIC(dbname, options) File "/Users/macbookpro/RHEAS/src/nowcast.py", line 69, in runDeterministicVIC startday, endyear, endmonth, endday, name) File "/Users/macbookpro/RHEAS/src/vic/vic.py", line 52, in __init__ "select resolution from vic.input order by abs(resolution - {0})".format(resolution)) psycopg2.ProgrammingError: relation "vic.input" does not exist

When I run ./bin/rheas -d postgres nowcast.confI get:

ERROR: relation "vic.input" does not exist at character 24 STATEMENT: select resolution from vic.input order by abs(resolution - 0.25) Traceback (most recent call last): File "/Users/macbookpro/RHEAS/bin/rheas", line 15, in <module> sys.exit(rheas.run()) File "/Users/macbookpro/RHEAS/src/rheas.py", line 81, in run nowcast.execute(dbname, options) File "/Users/macbookpro/RHEAS/src/nowcast.py", line 187, in execute runVIC(dbname, options) File "/Users/macbookpro/RHEAS/src/nowcast.py", line 29, in runVIC runDeterministicVIC(dbname, options) File "/Users/macbookpro/RHEAS/src/nowcast.py", line 69, in runDeterministicVIC startday, endyear, endmonth, endday, name) File "/Users/macbookpro/RHEAS/src/vic/vic.py", line 52, in __init__ "select resolution from vic.input order by abs(resolution - {0})".format(resolution)) psycopg2.ProgrammingError: relation "vic.input" does not exist

Looks like I am missing some of the VIC inputs. Is there a conf file for vic?

I'm happy to close this issue and open another one if that's preferable. Thanks!

[AdamJDuncan]

I think this may be caused by the initial state setting? Try vanilla, eg:

[nowcast]
model: vic
startdate: 2003-1-1
enddate: 2003-3-31
basin: data/tests/basin.shp
name: vanillatest
resolution: 0.25

[vic]
precip: chirps
temperature: ncep
wind: ncep
initialize: no
save to: db
save: net_short, soil_moist

[kashingtonDC]

When trying to populate the database with ncep (e.g. ./bin/rheas -u data.conf I get:

WARNING! Overwriting raster in tmax.ncep_4 table for 2003-01-01
Traceback (most recent call last):
  File "/Users/macbookpro/RHEAS/bin/rheas", line 15, in <module>
    sys.exit(rheas.run())
  File "/Users/macbookpro/RHEAS/src/rheas.py", line 76, in run
    update(dbname, config_filename)
  File "/Users/macbookpro/RHEAS/src/rheas.py", line 64, in update
    mod.download(dbname, dt, bbox)
  File "/Users/macbookpro/RHEAS/src/datasets/ncep.py", line 72, in download
    _downloadVariable(varname, dbname, dt, bbox)
  File "/Users/macbookpro/RHEAS/src/datasets/ncep.py", line 62, in _downloadVariable
    print("Imported {0} in {1}".format(t.strftime("%Y-%m-%d"), table))
  File "netCDF4/_netCDF4.pyx", line 3647, in netCDF4._netCDF4.Variable.__getattr__ (netCDF4/_netCDF4.c:35947)
  File "netCDF4/_netCDF4.pyx", line 3458, in netCDF4._netCDF4.Variable.getncattr (netCDF4/_netCDF4.c:33184)
  File "netCDF4/_netCDF4.pyx", line 1112, in netCDF4._netCDF4._get_att (netCDF4/_netCDF4.c:4200)
AttributeError: NetCDF: Attribute not found

Here is my data.conf file:

[domain]
minlat: 1.0
maxlat: 1.25
minlon: 35
maxlon: 35.25

[chirps]
startdate: 2003-1-1
enddate: 2003-3-31

[ncep]
startdate: 2003-1-1
enddate: 2003-3-31

[akssinghe]

Hi Kostas,

Even though it has been installed all necessary dependencies, it gives below error when downloading NCEP data. Kindly have a look.

Traceback (most recent call last): File "./bin/rheas", line 16, in sys.exit(rheas.run()) File "/home/susantha/RHEAS/src/rheas.py", line 76, in run update(dbname, config_filename) File "/home/susantha/RHEAS/src/rheas.py", line 64, in update mod.download(dbname, dt, bbox) File "/home/susantha/RHEAS/src/datasets/ncep.py", line 72, in download _downloadVariable(varname, dbname, dt, bbox) File "/home/susantha/RHEAS/src/datasets/ncep.py", line 62, in _downloadVariable print("Imported {0} in {1}".format(t.strftime("%Y-%m-%d"), table)) File "_netCDF4.pyx", line 3633, in netCDF4._netCDF4.Variable.getattr (netCDF4/_netCDF4.c:31607) File "_netCDF4.pyx", line 3448, in netCDF4._netCDF4.Variable.getncattr (netCDF4/_netCDF4.c:29324) File "_netCDF4.pyx", line 1099, in netCDF4._netCDF4._get_att (netCDF4/_netCDF4.c:4112)

AttributeError: NetCDF: Attribute not found

[kandread]

This is probably related to an issue with the upstream netCDF python package. Can you try updating to the newest package and see if that solves it? Also, the newest RHEAS version has a self-contained test suite so running ./bin/test after a correct installation should pass all 13 tests.

[akssinghe]

Hi K,

I am getting "psycopg2.ProgrammingError: relation "raster_resampled" does not exist" error since couple of days ago. I have installed psycopg2 and am with necessary input data sets. Pls advice to resolve this error.

---

Traceback (most recent call last): File "./bin/rheas", line 16, in sys.exit(rheas.run()) File "/home/rheas/RHEAS/src/rheas.py", line 81, in run nowcast.execute(dbname, options) File "/home/rheas/RHEAS/src/nowcast.py", line 191, in execute runVIC(dbname, options) File "/home/rheas/RHEAS/src/nowcast.py", line 29, in runVIC runDeterministicVIC(dbname, options) File "/home/rheas/RHEAS/src/nowcast.py", line 75, in runDeterministicVIC prec, tmax, tmin, wind = model.getForcings(options['vic']) File "/home/rheas/RHEAS/src/vic/vic.py", line 355, in getForcings rtables[v] = self.createIndexTable("{0}.{1}".format(v, options[v])) File "/home/rheas/RHEAS/src/vic/vic.py", line 296, in createIndexTable "select * from rasterresampled where sname='{0}' and tname like '{1}%' and resolution={2}".format(sname, tname, self.res)) psycopg2.ProgrammingError: relation "raster_resampled" does not exist

LINE 1: select * from raster_resampled where sname='precip' and tnam...

[kandread]

How are you ingesting the datasets, are you using rheas -d data.conf? That should trigger the creation of the raster catalog (see here) so it should always exist (unless the downloading was stopped externally). In any case, in order to fix this go inside a psql shell and paste create or replace function resampled(_s text, _t text, out result double precision) as $func$ begin execute format('select st_scalex(rast) from %s.%s limit 1',quote_ident(_s),quote_ident(_t)) into result; end $func$ language plpgsql; After that is finished paste this and you should have the catalog create or replace view raster_resampled as (select r_table_schema as sname,r_table_name as tname,resampled(r_table_schema,r_table_name) as resolution from raster_columns);

[akssinghe]

Hi K,

I am using rheas -u data.conf to ingest datasets.

Now it is working well. Thanks for guidance.

From: Kostas Andreadis [mailto:notifications@github.com] Sent: Monday, July 25, 2016 8:47 PM To: nasa/RHEAS RHEAS@noreply.github.com Cc: Susantha Jayasinghe susantha@adpc.net; Comment < comment@noreply.github.com> Subject: Re: [nasa/RHEAS] Nowcast test error: relation "raster_resampled" does not exist (#43)

How are you ingesting the datasets, are you using rheas -d data.conf? That should trigger the creation of the raster catalog (see here https://github.com/nasa/RHEAS/blob/master/src/dbio.py#L174) so it should always exist (unless the downloading was stopped externally). In any case, in order to fix this go inside a psql shell and paste create or replace function resampled(_s text, _t text, out result double precision) as $func$ begin execute format('select st_scalex(rast) from %s.%s limit 1',quote_ident(_s),quote_ident(_t)) into result; end $func$ language plpgsql; After that is finished paste this and you should have the catalog create or replace view raster_resampled as (select r_table_schema as sname,r_table_name as tname,resampled(r_table_schema,r_table_name) as resolution from raster_columns);

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nasa/RHEAS/issues/43#issuecomment-234957526, or mute the thread https://github.com/notifications/unsubscribe-auth/AROogGu4C-ifdipgIEm74go1VVcxM3feks5qZL5dgaJpZM4IQGzB .[image: Image removed by sender.]

[kandread]

Ok, closing the issue.