GIS4WRF / gis4wrf

QGIS toolkit 🧰 for pre- and post-processing 🔨, visualizing 🔍, and running simulations 💻 in the Weather Research and Forecasting (WRF) model 🌀
https://gis4wrf.github.io
MIT License
159 stars 36 forks source link

Error with RunGeogrib #163

Closed sbrockme closed 4 years ago

sbrockme commented 4 years ago

Hello,

I'm running through the tutorial for the first time, and am receiving a persistent error when attempting to run the WPS -> RunGeogrid AND RunMetgrid, Ungrib is fine. All files seem to be present and complete, however the Program Output throws the ERROR below:

Geogrid: Parsed 28 entries in GEOGRID.TBL Processing domain 1 of 2 ERROR: Could not open C:\Users\User\Documents\gis4wrf\datasets\geog/topo_gmted2010_5m/index Runtime: 0 s

Metgrid: Processing domain 1 of 2 ERROR: Couldn't open file ./geo_em.d01.nc for input. Runtime: 0 s

Unsure if this is a bug or I am missing something completely. Cheers.

dmey commented 4 years ago

Is C:\Users\User\Documents\gis4wrf\datasets\geog/topo_gmted2010_5m/index present? Can you tell us a bit more about your sys and QGIS/GIS4WRF versions you are running? Also are you following the tutorial as it is?

sbrockme commented 4 years ago

Yes, C:\Users\User\Documents\gis4wrf\datasets\geog/topo_gmted2010_5m/index is present, containing 32 files with a .index file. Running QGIS 3.8 Zanzibar, and the most recent version of GIS4WRF. I have followed the tutorial exactly step by step, with the exception of downloading geog data at hi resolution once, before starting it. After the initial download of data at hi res, I followed the tutorial exactly (which included downloading geog data again at low res).

dmey commented 4 years ago

can you open it? Is the content the same as:

type = continuous
signed = yes
projection = regular_ll
dx = 0.0833333
dy = 0.0833333
known_x = 1.0
known_y = 1.0
known_lat = -89.9583
known_lon = 0.04166667
wordsize = 2
tile_x = 600
tile_y = 600
tile_z = 1
tile_bdr=3
units="meters MSL"
description="GMTED2010 5-arc-minute topography height, subsampled by NCAR/MMM 2018-05-23 from 30-arc-second"

BTW, try to keep up-to-date with versions if there are no reasons for staying behind -- the latest QGIS version is 3.10.2 which is also LTR.

sbrockme commented 4 years ago

Respective index file

type = continuous signed = yes projection = regular_ll dx = 0.0833333 dy = 0.0833333 known_x = 1.0 known_y = 1.0 known_lat = -89.9583 known_lon = 0.04166667 wordsize = 2 tile_x = 600 tile_y = 600 tile_z = 1 tile_bdr=3 units="meters MSL" description="GMTED2010 5-arc-minute topography height, subsampled by NCAR/MMM 2018-05-23 from 30-arc-second"

dmey commented 4 years ago

can you send me the log again from when you click on run geogrid please as well as the namelist.wps -- drag and drop in comment.

sbrockme commented 4 years ago
Command: C:\Users\User\Documents\gis4wrf\dist\WPS-4.0-nompi\geogrid.exe
Working directory: C:/Users/User/Documents/gis4wrf/projects/2018_07_15_Amsterdam\run_wps
Parsed 28 entries in GEOGRID.TBL
Processing domain 1 of 2
ERROR: Could not open C:\Users\User\Documents\gis4wrf\datasets\geog/topo_gmted2010_5m/index
Runtime: 0 s
sbrockme commented 4 years ago
&share
nocolons = .true.
max_dom = 2
start_date = '2018-07-15_12:00:00', '2018-07-15_12:00:00'
end_date = '2018-07-15_18:00:00', '2018-07-15_18:00:00'
interval_seconds = 21600
/

&geogrid
parent_id = 1, 1
parent_grid_ratio = 1, 3
i_parent_start = 1, 11
j_parent_start = 1, 11
e_we = 31, 31
e_sn = 31, 31
map_proj = 'lambert'
dx = 9000.0
dy = 9000.0
ref_lon = 4.8952
ref_lat = 52.370199999992614
geog_data_res = 'lowres', 'lowres'
geog_data_path = '<ABS_PATH_TO_GEOG_DIR>'
truelat1 = 3.5
truelat2 = 7.0
stand_lon = 4.0
/

&metgrid
fg_name = 'FILE'
/
sbrockme commented 4 years ago

Apologies for the late response - above are the program output log and namelist.wps Also, I am running Windows 10, with the required RAM/disc etc. Keeping QGIS at 3.8 for dev purposes but will try upgrading if this fails

sbrockme commented 4 years ago
Command: C:\Users\User\Documents\gis4wrf\dist\WPS-4.0-nompi\ungrib.exe
Working directory: C:/Users/User/Documents/gis4wrf/projects/2018_07_15_Amsterdam\run_wps
 *** Starting program ungrib.exe ***
Start_date =  2018-07-15_12:00:00 ,      End_date = 2018-07-15_18:00:00
output format is WPS
Path to intermediate files is ./

###############################################################################

Inventory for date = 2018-07-15 12:00:00

PRES   TT       UU       VV       RH       HGT      PSFC     PMSL     SM000010 SM010040 SM040100 SM100200 SM010200 ST000010 ST010040 ST040100 ST100200 ST010200 SEAICE   LANDSEA  LANDN    SOILHGT  SKINTEMP SNOW     SNOWH    UMAXW    VMAXW    PMAXW    PMAXWNN  TMAXW    HGTMAXW  UTROP    VTROP    PTROP    PTROPNN  TTROP    HGTTROP
-------------------------------------------------------------------------------
2013.0  O        O        O        O        O        O        X        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O
2001.0  X        X        X        X        O        X        O        X        X        X        X        O        X        X        X        X        O        X        X        X        X        X        X        O        X        X        X        O        X        X        X        X        X        O        X        X
1000.0  X        X        X        X        X
 975.0  X        X        X        X        X
 950.0  X        X        X        X        X
 925.0  X        X        X        X        X
 900.0  X        X        X        X        X
 850.0  X        X        X        X        X
 800.0  X        X        X        X        X
 750.0  X        X        X        X        X
 700.0  X        X        X        X        X
 650.0  X        X        X        X        X
 600.0  X        X        X        X        X
 550.0  X        X        X        X        X
 500.0  X        X        X        X        X
 450.0  X        X        X        X        X
 400.0  X        X        X        X        X
 350.0  X        X        X        X        X
 300.0  X        X        X        X        X
 250.0  X        X        X        X        X
 200.0  X        X        X        X        X
 150.0  X        X        X        X        X
 100.0  X        X        X        X        X
  70.0  X        X        X        X        X
  50.0  X        X        X        X        X
  30.0  X        X        X        X        X
  20.0  X        X        X        X        X
  10.0  X        X        X        X        X
   7.0  X        X        X        X        X
   5.0  X        X        X        X        X
   3.0  X        X        X        X        X
   2.0  X        X        X        X        X
   1.0  X        X        X        X        X
-------------------------------------------------------------------------------

###############################################################################

Inventory for date = 2018-07-15 18:00:00

PRES   TT       UU       VV       RH       HGT      PSFC     PMSL     SM000010 SM010040 SM040100 SM100200 SM010200 ST000010 ST010040 ST040100 ST100200 ST010200 SEAICE   LANDSEA  LANDN    SOILHGT  SKINTEMP SNOW     SNOWH    UMAXW    VMAXW    PMAXW    PMAXWNN  TMAXW    HGTMAXW  UTROP    VTROP    PTROP    PTROPNN  TTROP    HGTTROP
-------------------------------------------------------------------------------
2013.0  O        O        O        O        O        O        X        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O
2001.0  X        X        X        X        O        X        O        X        X        X        X        O        X        X        X        X        O        X        X        X        X        X        X        O        X        X        X        O        X        X        X        X        X        O        X        X
1000.0  X        X        X        X        X
 975.0  X        X        X        X        X
 950.0  X        X        X        X        X
 925.0  X        X        X        X        X
 900.0  X        X        X        X        X
 850.0  X        X        X        X        X
 800.0  X        X        X        X        X
 750.0  X        X        X        X        X
 700.0  X        X        X        X        X
 650.0  X        X        X        X        X
 600.0  X        X        X        X        X
 550.0  X        X        X        X        X
 500.0  X        X        X        X        X
 450.0  X        X        X        X        X
 400.0  X        X        X        X        X
 350.0  X        X        X        X        X
 300.0  X        X        X        X        X
 250.0  X        X        X        X        X
 200.0  X        X        X        X        X
 150.0  X        X        X        X        X
 100.0  X        X        X        X        X
  70.0  X        X        X        X        X
  50.0  X        X        X        X        X
  30.0  X        X        X        X        X
  20.0  X        X        X        X        X
  10.0  X        X        X        X        X
   7.0  X        X        X        X        X
   5.0  X        X        X        X        X
   3.0  X        X        X        X        X
   2.0  X        X        X        X        X
   1.0  X        X        X        X        X
-------------------------------------------------------------------------------
 Name of source model =>NCEP GFS Analysis

###############################################################################

Inventory for date = 2018-07-15 12:00:00

PRES   TT       UU       VV       RH       HGT      PSFC     PMSL     SM000010 SM010040 SM040100 SM100200 SM010200 ST000010 ST010040 ST040100 ST100200 ST010200 SEAICE   LANDSEA  LANDN    SOILHGT  SKINTEMP SNOW     SNOWH    UMAXW    VMAXW    PMAXW    PMAXWNN  TMAXW    HGTMAXW  UTROP    VTROP    PTROP    PTROPNN  TTROP    HGTTROP
-------------------------------------------------------------------------------
2013.0  O        O        O        O        O        O        X        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O
2001.0  X        X        X        X        O        X        O        X        X        X        X        O        X        X        X        X        O        X        X        X        X        X        X        X        X        X        X        X        X        X        X        X        X        X        X        X
1000.0  X        X        X        X        X
 975.0  X        X        X        X        X
 950.0  X        X        X        X        X
 925.0  X        X        X        X        X
 900.0  X        X        X        X        X
 850.0  X        X        X        X        X
 800.0  X        X        X        X        X
 750.0  X        X        X        X        X
 700.0  X        X        X        X        X
 650.0  X        X        X        X        X
 600.0  X        X        X        X        X
 550.0  X        X        X        X        X
 500.0  X        X        X        X        X
 450.0  X        X        X        X        X
 400.0  X        X        X        X        X
 350.0  X        X        X        X        X
 300.0  X        X        X        X        X
 250.0  X        X        X        X        X
 200.0  X        X        X        X        X
 150.0  X        X        X        X        X
 100.0  X        X        X        X        X
  70.0  X        X        X        X        X
  50.0  X        X        X        X        X
  30.0  X        X        X        X        X
  20.0  X        X        X        X        X
  10.0  X        X        X        X        X
   7.0  X        X        X        X        X
   5.0  X        X        X        X        X
   3.0  X        X        X        X        X
   2.0  X        X        X        X        X
   1.0  X        X        X        X        X
-------------------------------------------------------------------------------
 Name of source model =>NCEP GFS Analysis

###############################################################################

Inventory for date = 2018-07-15 18:00:00

PRES   TT       UU       VV       RH       HGT      PSFC     PMSL     SM000010 SM010040 SM040100 SM100200 SM010200 ST000010 ST010040 ST040100 ST100200 ST010200 SEAICE   LANDSEA  LANDN    SOILHGT  SKINTEMP SNOW     SNOWH    UMAXW    VMAXW    PMAXW    PMAXWNN  TMAXW    HGTMAXW  UTROP    VTROP    PTROP    PTROPNN  TTROP    HGTTROP
-------------------------------------------------------------------------------
2013.0  O        O        O        O        O        O        X        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O        O
2001.0  X        X        X        X        O        X        O        X        X        X        X        O        X        X        X        X        O        X        X        X        X        X        X        X        X        X        X        X        X        X        X        X        X        X        X        X
1000.0  X        X        X        X        X
 975.0  X        X        X        X        X
 950.0  X        X        X        X        X
 925.0  X        X        X        X        X
 900.0  X        X        X        X        X
 850.0  X        X        X        X        X
 800.0  X        X        X        X        X
 750.0  X        X        X        X        X
 700.0  X        X        X        X        X
 650.0  X        X        X        X        X
 600.0  X        X        X        X        X
 550.0  X        X        X        X        X
 500.0  X        X        X        X        X
 450.0  X        X        X        X        X
 400.0  X        X        X        X        X
 350.0  X        X        X        X        X
 300.0  X        X        X        X        X
 250.0  X        X        X        X        X
 200.0  X        X        X        X        X
 150.0  X        X        X        X        X
 100.0  X        X        X        X        X
  70.0  X        X        X        X        X
  50.0  X        X        X        X        X
  30.0  X        X        X        X        X
  20.0  X        X        X        X        X
  10.0  X        X        X        X        X
   7.0  X        X        X        X        X
   5.0  X        X        X        X        X
   3.0  X        X        X        X        X
   2.0  X        X        X        X        X
   1.0  X        X        X        X        X
-------------------------------------------------------------------------------
Subroutine DATINT: Interpolating 3-d files to fill in any missing data...
Looking for data at time 2018-07-15_12
Found file:      FILE_2018-07-15_12
Looking for data at time 2018-07-15_18
Found file:      FILE_2018-07-15_18
End Subroutine DATINT.

**********
Deleting temporary files created by ungrib...
**********

Deleting file:  ./PFILE_2018-07-15_12
Deleting file:  ./PFILE_2018-07-15_18

**********
Done deleting temporary files.
**********

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!  Successful completion of ungrib.   !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Runtime: 0 s
sbrockme commented 4 years ago

Finally, the Run Metgrid trace

Command: C:\Users\User\Documents\gis4wrf\dist\WPS-4.0-nompi\metgrid.exe
Working directory: C:/Users/User/Documents/gis4wrf/projects/2018_07_15_Amsterdam\run_wps
Processing domain 1 of 2
ERROR: Couldn't open file ./geo_em.d01.nc for input.
Runtime: 0 s
dmey commented 4 years ago

From GIS4WRF > Simulation > Run, click on Open Configuration and edit the geog_data_path from

geog_data_path = '<ABS_PATH_TO_GEOG_DIR>'

to

geog_data_path = 'C:\Users\User\Documents\gis4wrf\datasets\geog/'

then save and run (assuming that 'User' is your username...)

image

sbrockme commented 4 years ago

It remains unchanged, throwing the same error...

&share nocolons = .true. max_dom = 2 start_date = '2018-07-15_12:00:00', '2018-07-15_12:00:00' end_date = '2018-07-15_18:00:00', '2018-07-15_18:00:00' interval_seconds = 21600 /

&geogrid parent_id = 1, 1 parent_grid_ratio = 1, 3 i_parent_start = 1, 11 j_parent_start = 1, 11 e_we = 31, 31 e_sn = 31, 31 map_proj = 'lambert' dx = 9000.0 dy = 9000.0 ref_lon = 4.8952 ref_lat = 52.370199999992614 geog_data_res = 'lowres', 'lowres' geog_data_path = 'C:\Users\User\Documents\gis4wrf\datasets\geog/' truelat1 = 3.5 truelat2 = 7.0 stand_lon = 4.0 /

&metgrid fg_name = 'FILE' /

dmey commented 4 years ago

Let's try to do it step by step from the command prompt. Open your command prompt, then type:

cd C:/Users/User/Documents/gis4wrf/projects/2018_07_15_Amsterdam\run_wps

then run ungrib.exe with

C:\Users\User\Documents\gis4wrf\dist\WPS-4.0-nompi\ungrib.exe

do you get the same? If you get the same error, please check you path carefully.

sbrockme commented 4 years ago

I did not, I believe the issue is that I have a space in my 'User' directory, the command fails at the space

dmey commented 4 years ago

I did not, I believe the issue is that I have a space in my 'User' directory, the command fails at the space

I do not understand what you mean. In any case, this looks to me just something to do with how you set up your paths and user account -- the errors are pretty clear to me.

sbrockme commented 4 years ago

Followed that exactly, ungrib.exe works, metgrid and geogrid both throwing the same errors, respectively, as before, when executed from the command line

sbrockme commented 4 years ago

Specifically, errors as follows:

metgrid.exe ERROR: Couldn't open file ./geo_em.d01.nc for input.

geogrid.exe ERROR: Could not open C:\Users\User\Documents\gis4wrf\datasets\geog/topo_gmted2010_5m/index

dmey commented 4 years ago

metgrid would only work after running geogrid and ungrib so no surprise there. With regards to metgrid, there is no magic, the error is telling you that the program cannot resolve the path to C:\Users\User\Documents\gis4wrf\datasets\geog/topo_gmted2010_5m/index so make sure that you can actually access that from the command prompt. Try with C:\Users\User\Documents\gis4wrf\datasets\geog/topo_gmted2010_5m? Can you cd into that from the command prompt. Then what happens if you do dir from there?

sbrockme commented 4 years ago

changing directly to the /topo_gmted2010_5m folder works just fine, dir returns a list of 33 files, which I assume are lowres raw files generated from the download.

Capture

Attempting to execute the index file directly is not a valid command

dmey commented 4 years ago

I'm afraid I am out of ideas -- assuming that you get the error

geogrid.exe ERROR: Could not open C:\Users\User\Documents\gis4wrf\datasets\geog/topo_gmted2010_5m/index

by running geogrid directly from the command prompt from C:/Users/User/Documents/gis4wrf/projects/2018_07_15_Amsterdam\run_wps I do no see what else you could be doing wrong. Given that we had never seen this before I would suggest checking the path again as well as that you have the correct permissions set when running the geogrid.

letmaik commented 4 years ago

@sbrockme You mentioned your 'User' directory has a space. Does that mean the real path is something like C:/Users/First Last/Documents/gis4wrf/..? If so, I don't think we tried running WPS/WRF when the path has spaces. It's probably not a gis4wrf issue but rather WRF-CMake.

To work-around this, in the settings, could you change your GIS4WRF working directory to some folder without spaces and try everything again? Don't try to copy the dataset folder yourself, please re-download, there is a known issue on Windows that will prevent you from doing that (related to a geog folder named "con").

sbrockme commented 4 years ago

Gentlemen! Thank you both very much for your insights....it's a great program and exactly what I'm looking for, so I want to make it work. I'll apply both suggestions/methods and get back. Oddly enough, I have also encountered the "con" loop, whereby it will not delete being a system variable. Anyway around this?

letmaik commented 4 years ago

In a command prompt, you can delete the "con" folder with rd /q /s \\?\c:\Users\User\Documents\gis4wrf\datasets\geog\xyz\con. Adjust the path accordingly. After that, you can delete the rest normally. Ideally, the WPS dataset should be changed by the WRF folks to not name folders "con".

sbrockme commented 4 years ago

It seems it was indeed a problem with a space character in the User directory C:/Users/First Last/Documents/gis4wrf/... Changing the folder to somewhere without a space worked immediately to solve the initial problems described above. I had to change MPI down to 6 in order to get a successful run.....Any quick way to increase resolution of the grid boxes further?

Additionally, regarding the "con" folder does not work in any conventional method I have tried, nor your suggestion above.

rd /q /s \?\c:\Users\User\Documents\gis4wrf\datasets\geog\xyz\con

from the command line is unable to find the path specified, although I have verified the path numerous times, copied and pasted, etc, to no avail.

dmey commented 4 years ago

Any quick way to increase resolution of the grid boxes further?

What does this mean?

Additionally, regarding the "con" folder does not work in any conventional method I have tried, nor your suggestion above.

rd /q /s \?\c:\Users\User\Documents\gis4wrf\datasets\geog\xyz\con

from the command line is unable to find the path specified, although I have verified the path numerous times, copied and pasted, etc, to no avail.

see https://github.com/GIS4WRF/gis4wrf/issues/104#issuecomment-443720429

letmaik commented 4 years ago

@dmey Nothing to do with additional backslashes. @sbrockme Try rd /q /s "\\?\c:\Users\User\Documents\gis4wrf\datasets\geog\xyz\con", that is, with quotes, since you have whitespaces.

sbrockme commented 4 years ago

Resolved with quotes. Thank you!

teerapolw commented 1 year ago

Excuse me, Can you already run geogrid and metgrid ? Can you tell me how to do that please ? Thanks