search

AnySoftKeyboard / AnySoftKeyboardTools

[DEPRECATED] AnySoftKeyboard various tools
27 stars 42 forks source link

Binary dictionary 4 MB limit #4

Open ivan-ushakov opened 9 years ago

ivan-ushakov commented 9 years ago

Hello. Dictionary tool has 4 MB limit (I guess because of 22 bit address). Is it possible to use it for large dictionaries like Turkish language with XML file size of 20 MB?

SvGeloven commented 8 years ago

Same question for 10 MB large file words.xml for the Dutch language with less than 400.000 words. The Dutch language uses a lot of compounds that need to be included explicitly in the dictionary.

SvGeloven commented 8 years ago

When I run LanguagePack-Dutch$ java -jar ../AnySoftKeyboardTools/makedict/makedict.jar I get the following error:

Reading words from input /home/***/workspace/LanguagePack-Dutch/dict/words.xml
Will store output files under /home/***/workspace/LanguagePack-Dutch/res/raw
Deleting previous versions...
Nodes = 1292442
Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 4194304
    at MakeBinaryDictionary.addNode(MakeBinaryDictionary.java:290)
    at MakeBinaryDictionary.writeWordsRec(MakeBinaryDictionary.java:343)
    at MakeBinaryDictionary.writeWordsRec(MakeBinaryDictionary.java:349)
    at MakeBinaryDictionary.writeWordsRec(MakeBinaryDictionary.java:349)
    at MakeBinaryDictionary.writeWordsRec(MakeBinaryDictionary.java:349)
    at MakeBinaryDictionary.writeWordsRec(MakeBinaryDictionary.java:349)
    at MakeBinaryDictionary.writeWordsRec(MakeBinaryDictionary.java:349)
    at MakeBinaryDictionary.writeWordsRec(MakeBinaryDictionary.java:349)
    at MakeBinaryDictionary.writeWordsRec(MakeBinaryDictionary.java:349)
    at MakeBinaryDictionary.writeWordsRec(MakeBinaryDictionary.java:349)
    at MakeBinaryDictionary.writeWordsRec(MakeBinaryDictionary.java:349)
    at MakeBinaryDictionary.writeWordsRec(MakeBinaryDictionary.java:349)
    at MakeBinaryDictionary.writeToDict(MakeBinaryDictionary.java:361)
    at MakeBinaryDictionary.<init>(MakeBinaryDictionary.java:161)
    at MakeBinaryDictionary.main(MakeBinaryDictionary.java:99)
    ... 5 more

Is this related to the 4 MB limit or another issue?

PanderMusubi commented 7 years ago

Is there any progress on this please?