search

realm / realm-core

Core database component for the Realm Mobile Database SDKs
https://realm.io
Apache License 2.0
1.02k stars 158 forks source link

Implement trimming of interned strings #7784

Closed sync-by-unito[bot] closed 1 month ago

sync-by-unito[bot] commented 4 months ago

See design document.

The success of interning and compressing needs to be monitored.

Trimming has to be done incrementally - no "stop the world" garbage collection.

Suggestion: When trimming is needed set up a second string-interner. Use the sign of StringIDs to distinguish between interners, allowing two interners to coexist.

Incrementally re-intern all properties of a column, then discard the old string interner.

Since trimming will take place after ingesting data, we can likely optimize compression by training the compressor on available data before interning.

sync-by-unito[bot] commented 4 months ago

➤ PM Bot commented:

Jira ticket: RCORE-2158