Application doesn't handle non-ASCII characters

CUTR-at-USF / gtfs-osm-sync

Synchronizes public transportation data in GTFS format with OpenStreetMap.org

Other

91 stars 18 forks source link

Application doesn't handle non-ASCII characters #25

Closed james2432 closed 8 years ago

james2432 commented 8 years ago

Characters like èÈéÉ etc are not currently handled when downloading from OSM Will create pull request

barbeau commented 8 years ago

@james2432 Thanks for filing this bug! Do you have a stack trace or any other information about what currently happens when it doesn't handle non-ASCII characters correctly?

It would be good to document this if possible.

james2432 commented 8 years ago

It makes the characters render incorrectly. Like in the image below, "DE LA CITÉ & DE LA GAPPE" appears as "DE LA CITÃ%...." and from osm it says "De la CitÃ©". This is due to invalid decoding(encoding on filewrite) as the old method was using default charsets defined by the operating system and may not be UTF8 (highly likely on windows machines) go-sync

barbeau commented 8 years ago

Perfect!