facebook / rocksdb

A library that provides an embeddable, persistent key-value store for fast storage.
http://rocksdb.org
GNU General Public License v2.0
27.83k stars 6.2k forks source link

How configure table_properties_collectors using Properties in Java? #12818

Closed ddupg closed 15 hours ago

ddupg commented 1 week ago

My project is developed using Java and uses org.rocksdb.ColumnFamilyOptions#getColumnFamilyOptionsFromProps to set options for CFs, following this document[1], I want to set table_properties_collector_factories to NewCompactOnDeletionCollectorFactory.

I have tried various ways but none have worked.

Properties props = new Properties();
// various ways I tried
props.put("table_properties_collectors", "id=CompactOnDeletionCollector:window_size=100:deletion_trigger=90:deletion_ratio=0.5");
props.put("table_properties_collectors", "CompactOnDeletionCollector:100:90:0.5");

return ColumnFamilyOptions.getColumnFamilyOptionsFromProps(props);

Then I see this in the LOG, and the parameters are all default values of 0

2024/06/27-17:53:19.343124 6163738624                   Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 0 Deletion trigger = 0 Deletion ratio = 0);

So my question is, does table_properties_collector_factories support being set using Properties or option string? If it does, how should I write the code?

[1] https://github.com/.../Implement-Queue-Service-Using...

Expected behavior

Use Properties to set table_properties_collector_factories of ColumnFamilyOptions in Java. Expected see LOG

Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 100 Deletion trigger = 90 Deletion ratio = 0.5);

Actual behavior

See this in LOG

Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 0 Deletion trigger = 0 Deletion ratio = 0);

Steps to reproduce the behavior

adamretter commented 6 days ago

Entires in Properties get converted into a string like key=value;key=value;... by a call to Options#getOptionStringFromProps.

As Options#setTablePropertiesCollectorFactory expects a Java Object, I don't think you will be able to set that via a string.

ddupg commented 3 days ago

@adamretter Thank you for your reply. I found this PR that supports customizing the TablePropertiesCollectorFactory with a string, and I saw an option called "table_properties_collectors" here, but I'm not sure how it can be applied to convert Properties into Options in Java. Can you help me with this?

adamretter commented 1 day ago

@ddupg I think what you are looking at is the CreateFromString interface that was added not long ago by @mrambacher to the C++ API. We don't yet support that in RocksJava, as it wasn't really clear to me initially how we could support that. We could re-examine if it might be possible to support this (or an equivalent) in RocksJava if that is helpful? However, it will take some time to achieve that.

ddupg commented 15 hours ago

@adamretter Thank you for your detailed response. Given this situation, I will attempt alternative methods. Looking forward to the day when this feature is available in RocksJava.