search

landam / grass-gis-git-migration-test

0 stars 0 forks source link

wxGUI v.in.ogr wrapper fails to find any layers if file path contains non-latin letters #143

Open landam opened 5 years ago

landam commented 5 years ago

Reported by marisn on 3 Jan 2015 11:05 UTC Steps to reproduce:

GRASS version: 7.0.0svn
GRASS SVN Revision: 63925
Build Date: 2015-01-02
Build Platform: i686-pc-mingw32
GDAL/OGR: 1.11.1
PROJ.4: 4.8.0
GEOS: 3.4.2
SQLite: 3.7.17
Python: 2.7.4
wxPython: 2.8.12.1
Platform: Windows-Vista-6.0.6002-SP2

Operating system

MS Windows

GRASS GIS version and provenance

svn-trunk

Migrated-From: https://trac.osgeo.org/grass/ticket/2524

landam commented 5 years ago

Comment by hellik on 3 Jan 2015 12:19 UTC Replying to [ticket:2524 marisn]:

Steps to reproduce:

  • mkdir "aurslieu dzelzce"
  • copy "myshapefile.*" "aurslieu dzelzce/"
  • start GRASS, open vector data import dialog
  • select "C:\balhblah\aurslieu dzelzce\myshapefile.shp" as a data source Observe how nothing happens. No error messages, no hangs, just nothing. No layers are listed and thus is not possible to continue with data import.

GRASS version: 7.0.0svn
GRASS SVN Revision: 63925
Build Date: 2015-01-02
Build Platform: i686-pc-mingw32
GDAL/OGR: 1.11.1
PROJ.4: 4.8.0
GEOS: 3.4.2
SQLite: 3.7.17
Python: 2.7.4
wxPython: 2.8.12.1
Platform: Windows-Vista-6.0.6002-SP2

Traceback (most recent call last):
  File "C:\OSGeo4W\apps\Python27\lib\site-packages\wx-2.8
-msw-unicode\wx\lib\filebrowsebutton.py", line 140, in
OnChanged

self.changeCallback(evt)
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\gui\wxpython\gui
_core\gselect.py", line 1920, in OnUpdate

self._reloadLayers()
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\gui\wxpython\gui
_core\gselect.py", line 1938, in _reloadLayers

input = dsn)
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\gui\wxpython\cor
e\gcmd.py", line 694, in RunCommand

ps = grass.start_command(prog, flags, overwrite, quiet,
verbose, **kwargs)
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\etc\python\grass
\script\core.py", line 349, in start_command

val = encode(val)
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\etc\python\grass
\script\utils.py", line 218, in encode

return string.encode(enc)
  File "C:\OSGeo4W\apps\Python27\lib\encodings\cp1252.py",
line 12, in encode

return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError
:
'charmap' codec can't encode character u'\u013c' in position
24: character maps to <undefined>
landam commented 5 years ago

Comment by annakrat on 3 Jan 2015 15:18 UTC The error is the same as in https://trac.osgeo.org/grass/ticket/2205. I have no idea about that one.

I don't get this error on Ubuntu. But when I have non ascii characters in the name of the file (not in the path to it), I got a different error coming from GetValidLayerName (this function tries to make the name sql compliant). Hopefully fixed in 63943.

landam commented 5 years ago

Modified by @landam on 12 May 2016 06:44 UTC

landam commented 5 years ago

Comment by @landam on 23 Aug 2016 10:04 UTC Still an issue?

landam commented 5 years ago

Comment by annakrat on 12 Sep 2016 21:39 UTC Tested on Ubuntu and Windows, no problems.

landam commented 5 years ago

Comment by marisn on 14 Mar 2017 08:30 UTC Tested import tool (File -> Import vector -> Common formats) - when choosing a Shapefile, no layers are listed and thus it is still impossible to import anything. This issue affects also v.import and v.in.ogr as all of them seem to share the same UI components.

GRASS versija: 7.3.svn                                                          
GRASS SVN revzija: https://trac.osgeo.org/grass/changeset/70508                                                      
Bvjuma datums: 2017-01-09                                                     
Bvanas platforma: x86_64-w64-mingw32                                         
GDAL: 2.1.2                                                                     
PROJ.4: 4.9.3                                                                   
GEOS: 3.5.0                                                                     
SQLite: 3.14.1                                                                  
Python: 2.7.5                                                                   
wxPython: 2.8.12.1                                                              
Platforma: Windows-8-6.2.9200 (OSGeo4W
landam commented 5 years ago

Comment by hellik on 14 Mar 2017 09:02 UTC Replying to [comment:6 marisn]:

Tested import tool (File -> Import vector -> Common formats) - when choosing a Shapefile, no layers are listed and thus it is still impossible to import anything. This issue affects also v.import and v.in.ogr as all of them seem to share the same UI components.

GRASS versija: 7.3.svn                                                          
GRASS SVN revzija: https://trac.osgeo.org/grass/changeset/70508                                                      
Bvjuma datums: 2017-01-09                                                     
Bvanas platforma: x86_64-w64-mingw32                                         
GDAL: 2.1.2                                                                     
PROJ.4: 4.9.3                                                                   
GEOS: 3.5.0                                                                     
SQLite: 3.14.1                                                                  
Python: 2.7.5                                                                   
wxPython: 2.8.12.1                                                              
Platforma: Windows-8-6.2.9200 (OSGeo4W

confirmed, tested here:

System Info                                                                     
GRASS version: 7.2.0                                                            
GRASS SVN revision: https://trac.osgeo.org/grass/changeset/70156                                                      
Build date: 2016-12-29                                                          
Build platform: x86_64-w64-mingw32                                              
GDAL: 2.1.2                                                                     
PROJ.4: 4.9.3                                                                   
GEOS: 3.5.0                                                                     
SQLite: 3.14.1                                                                  
Python: 2.7.5                                                                   
wxPython: 2.8.12.1                                                              
Platform: Windows-8-6.2.9200 (OSGeo4W) 
Traceback (most recent call last):
  File "C:\OSGEO4~1\apps\Python27\lib\site-packages\wx-2.8
-msw-unicode\wx\lib\filebrowsebutton.py", line 140, in
OnChanged

self.changeCallback(evt)
  File "C:\OSGEO4~1\apps\grass\grass-7.2.0\gui\wxpython\gui_
core\gselect.py", line 2055, in OnUpdate

self._reloadLayers()
  File "C:\OSGEO4~1\apps\grass\grass-7.2.0\gui\wxpython\gui_
core\gselect.py", line 2103, in _reloadLayers

input=dsn)
  File "C:\OSGEO4~1\apps\grass\grass-7.2.0\gui\wxpython\core
\gcmd.py", line 705, in RunCommand

quiet, verbose, **kwargs))
  File "C:\OSGEO4~1\apps\grass\grass-7.2.0\etc\python\grass\
script\core.py", line 311, in make_command

args.append(opt + b'=' + _make_val(val))
  File "C:\OSGEO4~1\apps\grass\grass-7.2.0\etc\python\grass\
script\core.py", line 98, in _make_val

return encode(val)
  File "C:\OSGEO4~1\apps\grass\grass-7.2.0\etc\python\grass\
script\utils.py", line 177, in encode

return string.encode(enc) if enc else string.encode()
  File "C:\OSGEO4~1\apps\Python27\lib\encodings\cp1252.py",
line 12, in encode

return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError
:
'charmap' codec can't encode character u'\u013c' in position
36: character maps to <undefined>
System Info                                                                     
GRASS version: 7.3.svn                                                          
GRASS SVN revision: https://trac.osgeo.org/grass/changeset/70750M                                                     
Build date: 2017-03-14                                                          
Build platform: x86_64-w64-mingw32                                              
GDAL: 2.1.2                                                                     
PROJ.4: 4.9.3                                                                   
GEOS: 3.5.0                                                                     
SQLite: 3.14.1                                                                  
Python: 2.7.5                                                                   
wxPython: 2.8.12.1                                                              
Platform: Windows-8-6.2.9200 (OSGeo4W) 
Traceback (most recent call last):
  File "C:\OSGEO4~1\apps\Python27\lib\site-packages\wx-2.8
-msw-unicode\wx\lib\filebrowsebutton.py", line 140, in
OnChanged

self.changeCallback(evt)
  File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\gui\wxpython\gu
i_core\gselect.py", line 2071, in OnUpdate

self._reloadLayers()
  File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\gui\wxpython\gu
i_core\gselect.py", line 2119, in _reloadLayers

input=dsn)
  File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\gui\wxpython\co
re\gcmd.py", line 705, in RunCommand

quiet, verbose, **kwargs))
  File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\etc\python\gras
s\script\core.py", line 312, in make_command

args.append(opt + b'=' + _make_val(val))
  File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\etc\python\gras
s\script\core.py", line 99, in _make_val

return encode(val)
  File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\etc\python\gras
s\script\utils.py", line 184, in encode

return string.encode(enc)
  File "C:\OSGEO4~1\apps\Python27\lib\encodings\cp1252.py",
line 12, in encode

return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError
:
'charmap' codec can't encode character u'\u013c' in position
36: character maps to <undefined>
landam commented 5 years ago

Comment by annakrat on 14 Apr 2017 14:44 UTC Is there any non-ascii character somewhere in the path?

landam commented 5 years ago

Comment by hellik on 14 Apr 2017 14:51 UTC Tested it then with the example in original report above.

landam commented 5 years ago

Comment by annakrat on 16 Apr 2017 14:27 UTC Some of the characters are not defined in cp1252, that's where the error comes from. There is nothing else we can do about it except of giving a better error message. When I switch my locale, I didn't get any error but it still doesn't work, just as Maris describes.

This is output I get in terminal when I switch to Czech locale (cp1250). Note that character **** is not defined in cp1252 but is defined in cp1250


C:\Users\anna>v.external in=C:\Users\anna\Desktop\d\aaaa.shp
D1/5: G_set_program_name(): v.external
D2/5: G_file_name(): path = C:\Users\anna\Documents\grassdata\nc_basic_spm_grass7\user1
D1/5: GRASS_VECTOR_OGR defined? no
D1/5: dsn: C:\Users\anna\Desktop\d\aaaa.shp
ERROR: Unable to open data source 'C:\Users\anna\Desktop\d\aaaa.shp'

When I remove ****, it works:

C:\Users\anna>v.external in=C:\Users\anna\Desktop\d\aaaa.shp --o
D1/5: G_set_program_name(): v.external
D2/5: G_file_name(): path = C:\Users\anna\Documents\grassdata\nc_basic_spm_grass7\user1
D1/5: GRASS_VECTOR_OGR defined? no
D1/5: dsn: C:\Users\anna\Desktop\d\aaaa.shp
D2/5: layer 'aaaa' was found
...

I think I am missing something here.

landam commented 5 years ago

Comment by neteler on 26 Jan 2018 11:40 UTC Ticket retargeted after milestone closed

landam commented 5 years ago

Modified by neteler on 12 Jun 2018 20:48 UTC

landam commented 5 years ago

Comment by mlennert on 2 Oct 2018 10:10 UTC Is this really an issue that can be handled within GRASS GIS ? It sounds more like a general encoding config issue.

landam commented 5 years ago

Comment by neteler on 14 Oct 2018 11:31 UTC Can you please try in trunk/Python3?

landam commented 5 years ago

Comment by neteler on 22 Oct 2018 20:50 UTC Ticket retargeted after milestone closed

landam commented 5 years ago

Comment by @landam on 11 Dec 2018 11:35 UTC Bump milestone to 7.4.4

landam commented 5 years ago

Comment by neteler on 4 Jan 2019 19:23 UTC Ticket retargeted after milestone closed