search

bertvandepoel / snelSLiM

A linguistic set of tools in Go and web interface in PHP to do quick Stable Lexical Marker Analysis
GNU Affero General Public License v3.0
3 stars 0 forks source link

Review encoding #16

Open bertvandepoel opened 5 years ago

bertvandepoel commented 5 years ago

Check how Go reacts to different encodings. It should process them correctly, but is that really the case? How about foliafolie and xmllint?

bertvandepoel commented 5 years ago

http://dominik.honnef.co/posts/2012/04/dealing_with_encodings_in_go/

bertvandepoel commented 5 years ago

https://godoc.org/github.com/rakyll/magicmime

bertvandepoel commented 5 years ago

Might be worth looking at EUC-JP and EUC-KR encodings. Probably also worth looking at encodings for Chinese and other non-ascii languages that have several decades of widespread digital use.