Cannot read urbfrac_nlcd2011 dataset due to "%" in description metadata field

[hoangnghiem72]

Describe the bug An error has occurred while executing Python code: configparser.InterpolationSyntaxError

When I open a existing GIS4WRF project by going to Simulation > General and clicking on Open a GIS4WRF Project and click on Select Folder, and see Error:

An error has occurred while executing Python code:

configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%, 23 = 90%, 24 = 95%)"' Traceback (most recent call last): File "/home/hoangnghiem/.local/share/QGIS/QGIS3/profiles/default/python/plugins/gis4wrf/plugin/ui/tab_simulation.py", line 115, in on_open_project self.set_project_in_tabs() File "/home/hoangnghiem/.local/share/QGIS/QGIS3/profiles/default/python/plugins/gis4wrf/plugin/ui/tab_simulation.py", line 74, in set_project_in_tabs self.datasets_tab.project = project File "/home/hoangnghiem/.local/share/QGIS/QGIS3/profiles/default/python/plugins/gis4wrf/plugin/ui/widget_datasets.py", line 70, in project self.populate_geog_data_tree() File "/home/hoangnghiem/.local/share/QGIS/QGIS3/profiles/default/python/plugins/gis4wrf/plugin/ui/widget_datasets.py", line 280, in populate_geog_data_tree add_derived_metadata_to_geogrid_tbl(tbl, self.options.geog_dir) File "/home/hoangnghiem/.local/share/QGIS/QGIS3/profiles/default/python/plugins/gis4wrf/core/readers/geogrid_tbl.py", line 112, in add_derived_metadata_to_geogrid_tbl meta = read_wps_binary_index_file(dataset_path) File "/home/hoangnghiem/.local/share/QGIS/QGIS3/profiles/default/python/plugins/gis4wrf/core/readers/wps_binary_index.py", line 172, in read_wps_binary_index_file m.description = clean_str(meta.get('description')) File "/usr/lib64/python3.7/configparser.py", line 1301, in get fallback=fallback, **kwargs) File "/usr/lib64/python3.7/configparser.py", line 799, in get d) File "/usr/lib64/python3.7/configparser.py", line 394, in before_get self._interpolate_some(parser, option, L, value, section, defaults, 1) File "/usr/lib64/python3.7/configparser.py", line 444, in _interpolate_some "found: %r" % (rest,)) configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%, 23 = 90%, 24 = 95%)"'

Python version: 3.7.6 (default, Jan 30 2020, 09:44:41) [GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] QGIS version: 3.12.1-București București, exported

Python Path: /usr/share/qgis/python /home/hoangnghiem/.local/share/QGIS/QGIS3/profiles/default/python /home/hoangnghiem/.local/share/QGIS/QGIS3/profiles/default/python/plugins /usr/share/qgis/python/plugins /usr/lib64/python37.zip /usr/lib64/python3.7 /usr/lib64/python3.7/lib-dynload /usr/lib64/python3.7/site-packages /usr/lib/python3.7/site-packages /home/hoangnghiem/.local/share/QGIS/QGIS3/profiles/default/python /home/hoangnghiem/.local/share/gis4wrf/python37/lib64/python3.7/site-packages /home/hoangnghiem/.local/share/gis4wrf/python37/lib/python3.7/site-packages

My OS is Linux Fedora 31 and run with QGIS 3.12 and GIS4WRF 0.14.2

Please help me to overcome the error. Thank you in advance.

[letmaik]

Looks like one of your geographic datasets has an index file that gis4wrf can't read because the description= line has some weird symbols. Could you figure out which one it is (by opening the index files in some text editor) and copy paste the relevant line here? This would help with debugging.

[hoangnghiem72]

Thank you very much Maik for your help. Based on your comment I found that in the last line of the index file in urbfrac_nlcd2011 dataset as follow: description="Urban fraction derived from 30 m NLCD 2011 (22 = 50%, 23 = 90%, 24 = 95%)" I correct it to: description="Urban fraction derived from 30 m NLCD 2011 (22 = 50%%, 23 = 90%%, 24 = 95%%)" GIS4WRF run with no error.

[letmaik]

Thanks, indeed we've never tested it using this optional dataset. It looks like the parser we use can't deal with the percent sign properly as you've discovered. I'm glad you found a work-around, it's definitely a bug in gis4wrf. For reference, this is the code from WPS that parses the description field:

                  else if (index('description',trim(buffer(1:i-1))) /= 0) then
                     ispace = i+1
                     iquoted = 0
                     do while (((ispace < eos) .and. (buffer(ispace:ispace) /= ' ')) .or. (iquoted == 1))
                        if (buffer(ispace:ispace) == '"' .or. buffer(ispace:ispace) == '''') iquoted = mod(iquoted+1,2)
                        ispace = ispace + 1
                     end do 
                     is_descr(idx) = .true.
                     source_descr(idx) = ' '
                     if (buffer(i+1:i+1) == '"' .or. buffer(i+1:i+1) == '''') i = i + 1
                     if (buffer(ispace-1:ispace-1) == '"' .or. buffer(ispace-1:ispace-1) == '''') ispace = ispace - 1
                     source_descr(idx)(1:ispace-i) = buffer(i+1:ispace-1)

I can't promise when we'll get around to fixing it, but for now the work-around you have seems to be fine, so I'll leave this issue open in case someone stumbles over the same.

EDIT: This seems to be a simple matter of disabling interpolation support in the parser via interpolation=None.

[letmaik]

The fix was easier than expected, will be part of the next release. Thanks again for reporting!