Japanese language detection problem

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Download attached Japanese text
2. Execute in CMD.exe: "java -jar langdetect.jar --detectlang -d profiles 
lang_detect.txt"

What is the expected output? [ja:0.7142823122662098]
What do you see instead? lang_detect.txt:[en:0.7142823122662098, 
pl:0.14285727552109861, tl:0.14285682309
334474]

What version of the product are you using? latest (langdetect-09-13-2011)
On what operating system? Windows 7

Original issue reported on code.google.com by vova.pri...@gmail.com on 20 Apr 2012 at 7:48

Attachments:

• lang_detect.txt

[GoogleCodeExporter]

Using attached text lang_detect_1.txt 

What is the expected output? [ja:0.9999952022259697]
What do you see instead? [en:0.9999952022259697]

Original comment by vova.pri...@gmail.com on 20 Apr 2012 at 7:57

Attachments:

• lang_detect_1.txt

[GoogleCodeExporter]

Solution: attached documents are in ANSI encoding, need to save them to UTF8

Original comment by vova.pri...@gmail.com on 21 Apr 2012 at 3:21