The class "com.alipay.sofa.jraft.util.StorageOptionsFactory" still invokes an old Constructor of RocksDB BloomFilter

samirvb commented 3 years ago

We are trying to upgrade our RocksDB version and found that we see the below exception in the com.alipay.sofa.jraft.util.StorageOptionsFactory method "getDefaultRocksDBTableConfig". The filter being set is a new org.rocksdb.BloomFilter where the constructor now accepts a Double and Boolean (https://javadoc.io/doc/org.rocksdb/rocksdbjni/6.6.4/org/rocksdb/BloomFilter.html) . We see an exception when we try to get a DefaultRocksDBTablesConfig object :

Caused by: java.lang.NoSuchMethodError: 'void org.rocksdb.BloomFilter.(int, boolean)' at com.alipay.sofa.jraft.util.StorageOptionsFactory.getDefaultRocksDBTableConfig(StorageOptionsFactory.java:303) at com.alipay.sofa.jraft.util.StorageOptionsFactory.getRocksDBTableFormatConfig(StorageOptionsFactory.java:289) at com.alipay.sofa.jraft.storage.impl.RocksDBLogStorage.createColumnFamilyOptions(RocksDBLogStorage.java:173) at com.alipay.sofa.jraft.storage.impl.RocksDBLogStorage.initAndLoad(RocksDBLogStorage.java:219) at com.alipay.sofa.jraft.storage.impl.RocksDBLogStorage.init(RocksDBLogStorage.java:205) at com.alipay.sofa.jraft.storage.impl.RocksDBLogStorage.init(RocksDBLogStorage.java:70) at com.alipay.sofa.jraft.storage.impl.LogManagerImpl.init(LogManagerImpl.java:184) at com.alipay.sofa.jraft.storage.impl.LogManagerImpl.init(LogManagerImpl.java:77) at com.alipay.sofa.jraft.core.NodeImpl.initLogStorage(NodeImpl.java:589) at com.alipay.sofa.jraft.core.NodeImpl.init(NodeImpl.java:999) at com.alipay.sofa.jraft.core.NodeImpl.init(NodeImpl.java:139) at com.alipay.sofa.jraft.RaftServiceFactory.createAndInitRaftNode(RaftServiceFactory.java:47) at com.alipay.sofa.jraft.RaftGroupService.start(RaftGroupService.java:129)

A clear and concise description of what the bug is.

Expected behavior

Actual behavior

Steps to reproduce

Minimal yet complete reproducer code (or GitHub URL to code)

Environment

SOFAJRaft version: 1.3.8
JVM version (e.g. java -version): OpenJDK Runtime Environment Corretto-11.0.12.7.1 (build 11.0.12+7-LTS)
OS version (e.g. uname -a): Linux a-3hrv73d76vs27 4.14.241-184.433.amzn2.x86_64 #1 SMP Wed Aug 4 14:35:15 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Maven version: 3.8.1
IDE version: IntelliJ community edition

killme2008 commented 3 years ago

Hi, you can override the default options by

BlockBasedTableConfig tableConfig = ...;
StorageOptionsFactory.registerRocksDBTableFormatConfig(RocksDBLogStorage.class,  tableConfig);

Note: register the table config before starting jraft node.

samirvb commented 3 years ago

Hi, you can override the default options by
BlockBasedTableConfig tableConfig = ...;
StorageOptionsFactory.registerRocksDBTableFormatConfig(RocksDBLogStorage.class,  tableConfig);
Note: register the table config before starting jraft node.

Hi ,

 I'm not sure how I can plug in the tableConfig into my RaftGroupService object. Can you help ? Below is my code :

BlockBasedTableConfig config = new BlockBasedTableConfig(). // setIndexType(IndexType.kHashSearch). // use hash search(btree) for prefix scan. setBlockSize(16 * SizeUnit.KB).// setFilterPolicy(new BloomFilter(16.0, false)). // setCacheIndexAndFilterBlocks(true). // setPinL0FilterAndIndexBlocksInCache(true); StorageOptionsFactory.registerRocksDBTableFormatConfig(RocksDBLogStorage.class,config);

        // Start the server
        node = (NodeImpl)this.raftGroupService.start(startRpcServer);

killme2008 commented 3 years ago

Right now the tableConfig is a global configuration, so you don't have to plug in it to your raft group service after registerRocksDBTableFormatConfig.

sofastack / sofa-jraft