search

GoogleCodeArchives / gpicsync

Automatically exported from code.google.com/p/gpicsync
Other
0 stars 0 forks source link

Non-ASCII GeoNames cause an UnicodeDecodeError #91

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Get gpicsync-GUI.py from svn
2. Try to embed GeoNames in a photo taken in a place whose name has
non-ASCII characters

What is the expected output? What do you see instead?
Expected the geoname to be written into exif. (Presumably as UTF-8.)

What version of the product are you using? On what operating system?
svn r926, Mac OS X

Please provide any additional information below.
Here's what I see in terminal where I started gpicsync-GUI.py from.

=== geonameKeywords ===
 -iptc:city="Teekkarikylä" -iptc:province-state="Southern Finland"
-iptc:Country-PrimaryLocationName="Finland"
-iptc:Country-PrimaryLocationCode="FI" -iptc:Sub-location="1.02 Km East
Teekkarikylä" 
======
Unhandled exception in thread started by <function sync at 0x3ca0fb0>
Traceback (most recent call last):
  File "gpicsync-GUI.py", line 978, in sync
    os.popen('%s %s  -overwrite_original "-DateTimeOriginal>FileModifyDate"
"%s" '%(self.exifcmd,geonameKeywords,self.picDir+'/'+fileName))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 33:
ordinal not in range(128)

Original issue reported on code.google.com by hsivonen@iki.fi on 30 Jul 2009 at 11:39

GoogleCodeExporter commented 9 years ago 
I confirm this on Mac OS X

Original comment by pfhon...@gmail.com on 8 Sep 2009 at 8:15

GoogleCodeExporter commented 9 years ago 
I encountered the same issue on Karmic. Terminal output as follows:

Number of raw track points found:  36561
Number of valid track points found:  36561
Extracting data from valids track points ...
local UTC Offset (seconds)=  0
>>> self.utcOffest in kml for time stamps:  +0:00

Found fileName  _MG_3062.CR2  Processing now ...
/home/losheng/Pictures/2009-11-JPN/Card-01/originals-backup-Card-01/_MG_3062.CR2
readDateTimeSize answer Date/Time Original              : 2009:11:12 12:21:33
Image Size                      : 2496x1664

Writing best lat./long. match to pic. EXIF --> 35.758760731 N 139.689362571 E 
with
tpic-tgps= 7 seconds

timeStamp= <TimeStamp><when>2009-11-12T12:21:33+0:00</when> </TimeStamp>

width > height
latitude=  35.758760731   longitude=  139.689362571
url= 
http://ws.geonames.org/findNearbyPlaceName?lat=35.758760731&lng=139.689362571&st
yle=full
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<geonames>
<geoname>
<name>Azusawa</name>
<lat>35.7666667</lat>
<lng>139.7</lng>
<geonameId>1864834</geonameId>
<countryCode>JP</countryCode>
<countryName>Japan</countryName>
<fcl>P</fcl>
<fcode>PPLX</fcode>
<fclName>city, village,...</fclName>
<fcodeName>section of populated place</fcodeName>
<population/>
<alternateNames>Azukizawa, Azusawacho, Azukisawa</alternateNames>
<elevation>0</elevation>
<continentCode>AS</continentCode>
<adminCode1>40</adminCode1>
<adminName1>Tōkyō</adminName1>
<adminCode2/>
<adminName2/>
<timezone dstOffset="9.0" gmtOffset="9.0">Asia/Tokyo</timezone>
<distance>1.3015</distance>
</geoname>
</geonames>

Azusawa
1.30
Tōkyō
Japan
JP
nearbyPlaceLat, nearbyPlacelon 35.7666667 139.7
GPS lat,lon 35.758760731 139.689362571
deltaLat, deltaLon -0.007905969 -0.010637429
(tan(pi/8)*deltaLon) -0.00440616736057
(tan(3*pi/8)*deltaLon) -0.0256810253605
angle in degrees 53.3794643801
In (deltaLon <0) and (deltaLat <0)
South-West
gnInfos: Geonames: 1.30 Km South-West Azusawa Tōkyō Japan JP

=== gnIPTCsummary===   -iptc:caption-abstract="" ======
************* JP <type 'str'>

=== geonameKeywords ===
 -iptc:city="Azusawa" -iptc:province-state="Tōkyō"
-iptc:Country-PrimaryLocationName="Japan" -iptc:Country-PrimaryLocationCode="JP"
-iptc:Sub-location="1.30 Km South-West Azusawa" 
======
Unhandled exception in thread started by <function sync at 0x997c5dc>
Traceback (most recent call last):
  File "gpicsync-GUI.py", line 978, in sync
    os.popen('%s %s  -overwrite_original "-DateTimeOriginal>FileModifyDate" "%s"
'%(self.exifcmd,geonameKeywords,self.picDir+'/'+fileName))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 54: 
ordinal not
in range(128)

Original comment by losh...@gmail.com on 27 Nov 2009 at 2:37

GoogleCodeExporter commented 9 years ago 
Here's a fix.

Original comment by hsivonen@iki.fi on 6 Apr 2010 at 3:25

Attachments:

GoogleCodeExporter commented 9 years ago 
Thanks very much, this fix worked for me. It would be great if it could please 
be applied.

Original comment by aaronw...@gmail.com on 19 May 2013 at 5:14

GoogleCodeExporter commented 9 years ago 
Hello aaronw (and sorry hsivonen I've only seen the patch :/).

I've applied the patch on my Win7 box, unfortunately unicode characters are not 
well recognized on this picture from Israel (see tags):
http://flickr.com/gp/frenchy/90004j

aaronw are you on a win7 or win8 PC? If so could you just sent me a test sample 
which works now (and didn't work before) just for testing (gpx, utf offset 
value, one or two pictures).

Thanks!

Original comment by francois...@gmail.com on 19 May 2013 at 7:43

GoogleCodeExporter commented 9 years ago 
I am using Ubuntu 13.04.

Yes, now that you mention it, I realise that the tags have question marks 
instead of the special characters, sorry. That said, at least it runs, which 
was a marked improvement!

Original comment by aaronw...@gmail.com on 19 May 2013 at 9:07

GoogleCodeExporter commented 9 years ago 
"That said, at least it runs, which was a marked improvement!"

Is it a package on Ubuntu 13.04 or are using it from the SVN source here? I 
thought it should have worked from source (but unidecode lib must be installed 
first).
https://code.google.com/p/gpicsync/issues/detail?id=117

That said as you said previous patch is working it is better since there's no 
loss. I'm doing now a platform check in the commit I've just done.
If the OS is either Linux or OSX I'm using hsivonen command line otherwise I 
use unidecode lib on windows.

If you have a test sample I'm still interested for future tests. Thanks.

Original comment by francois...@gmail.com on 20 May 2013 at 3:13

GoogleCodeExporter commented 9 years ago 
The version that comes with Ubuntu 13.04 (Raring) is GPicSync  1.27 - 2008. 
I've tested it with the two attached and it crashes as described above:
=== geonameKeywords ===
 -iptc:city="Bran" -iptc:province-state="Braşov" -iptc:Country-PrimaryLocationName="Romania" -iptc:Country-PrimaryLocationCode="RO" -iptc:Sub-location="1.40 Km East Bran" -iptc:caption-abstract="" 
======
Unhandled exception in thread started by <function sync at 0x201ee60>
Traceback (most recent call last):
  File "gpicsync-GUI.py", line 911, in sync
    os.popen('%s %s  -overwrite_original "-DateTimeOriginal>FileModifyDate" "%s" '%(self.exifcmd,geonameKeywords,self.picDir+'/'+fileName))  
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 53: 
ordinal not in range(128)

Original comment by aaronw...@gmail.com on 27 May 2013 at 5:20

Attachments:

GoogleCodeExporter commented 9 years ago 
I have tried this with the svn version and receive a similar error:
$ ./gpicsync-GUI.py
(2, 8, 12, 1, '')
wxversion ['2.8-gtk2-unicode']
Python verion is 2.7.4 (default, Apr 19 2013, 18:28:01) 
[GCC 4.7.3]
couldn't import pytz
Searching configuration file ~/.gpicsync.conf
Couldn't find the configuration file.
Attempting to read the configuration file...
Number of raw track points found:  7242
Number of valid track points found:  7242
Extracting data from valids track points ...
local UTC Offset (seconds)=  10800.0
Couldn't create the backup folder, it maybe already exist
>>> self.utcOffest in kml for time stamps:  +3:00
Couldn't create the thumbs folder, it maybe already exist

Found fileName  IMG_0584.JPG  Processing now ...
/home/aaron/Desktop/Test/IMG_0584.JPG
Create a thumb now!
Writing best lat./long. match to pic. EXIF --> 45.51549512 N 25.36791644 E with 
tpic-tgps= 0 seconds

timeStamp= 
width > height
latitude=  45.51549512   longitude=  25.36791644
url=  
http://ws.geonames.org/findNearbyPlaceName?lat=45.51549512&lng=25.36791644&style
=full
1.40
Braşov
Romania
RO
East

Unhandled exception in thread started by <function sync at 0x173f848>
Traceback (most recent call last):
  File "./gpicsync-GUI.py", line 1051, in sync
    os.popen(('%s %s  -overwrite_original "-DateTimeOriginal>FileModifyDate" "%s" '%(self.exifcmd,geonameKeywords.decode("utf-8"),self.picDir+'/'+fileName)).encode("utf-8"))
  File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u015f' in position 
44: ordinal not in range(128)

Original comment by aaronw...@gmail.com on 27 May 2013 at 5:27