search

realm / realm-java

Realm is a mobile database: a replacement for SQLite & ORMs
http://realm.io
Apache License 2.0
11.45k stars 1.75k forks source link

Sorting using ICU #1031

Open kneth opened 9 years ago

kneth commented 9 years ago

Sorting is currently done in "English locale". Using Android's ICU library could make it possible to specify the locale to sort in.

cmelchior commented 9 years ago

This would also fix #581

saket commented 8 years ago

Any updates on this? This drawback makes search kinda difficult for non-English users. I'm forced to manually iterate through RealmResult and compare text.

kneth commented 8 years ago

Sorry, we don't have updates on it. We fully understand searching and sorting for non-English character sets are painful. ICU is a very large dependency to add (our jar file could easily double in size), and we would like to find a best solution.

saket commented 8 years ago

Android already has ICU. Why add it again?

kneth commented 8 years ago

Different Android versions are using different ICU versions. And using non-NDK libraries in C++ code is often a bit cumbersome.

saket commented 8 years ago

Agreed with the NDK part, but choosing the version of ICU is your job. Android will do it automatically, right? Simply use the Locale APIs? Please correct if I'm wrong.

kneth commented 8 years ago

Using the Locale API implies that you will have to copy data (strings) between the JVM and our storage engine. It is definitely not acceptable from a performance perspective. Supporting ICU is important and we will get around the feature at some point.

saket commented 8 years ago

Ah, understood. Would you suggest any workaround in the meanwhile, other than iterating a table content manually?

kneth commented 8 years ago

If you are outside the range 0-591 (UTF-8), there is no easy workaround.

cmelchior commented 8 years ago

For reference: https://github.com/realm/realm-core/issues/1946

If we go the route where the bindings are responsible for sorting, we should look into building ICU specifically for a given app. Given the size of the default ICU (16 MB - http://userguide.icu-project.org/packaging) it will be unfeasible for most apps to package that.

If we automatically detected which languages are supported by an app, I think we could make a smart build of ICU with only the required languages. This is most likely not trivial though.

Right now ICU is only official supported in Android N through https://developer.android.com/preview/features/icu4j-framework.html, however there is no native support yet. The earliest that would happen is in O, and even then it will take years for it to become widely available.

Alternatively we should look into providing these tables in an alternative manner, e.g. we could create a special dependency with only the lookup table and distribute that as a normal dependency. Realm could then dynamically load these if found.

cmelchior commented 6 years ago

Maybe there will soon be light at the end of the tunnel: https://github.com/android-ndk/ndk/issues/548