bytedance / terarkdb

A RocksDB compatible KV storage engine with better performance
Apache License 2.0
2.04k stars 203 forks source link

customized terarkzip default options/options.cc modification. problem as mentioned #247

Open vipsland opened 2 years ago

vipsland commented 2 years ago

i've edited options/options.cc with error below. can anyone pls help? coz i got the code to make all request defaulting to use terarkzip coz my NON-C++ language binding for rocksdb doesn't have the terarkzip option binding.

pls help. thx

/*
ColumnFamilyOptions::ColumnFamilyOptions()
    : compression(Snappy_Supported() ? kSnappyCompression : kNoCompression),
      table_factory(
          std::shared_ptr<TableFactory>(new BlockBasedTableFactory())) {}
//
*/

ColumnFamilyOptions::ColumnFamilyOptions()
    : compression(Snappy_Supported() ? kSnappyCompression : kNoCompression),
      table_factory(
          std::shared_ptr<TableFactory>(new BlockBasedTableFactory())) {
  TerarkZipTableOptions tzt_options;
  tzt_options.localTempDir = "/tmp";
  tzt_options.indexNestLevel = 3;
  tzt_options.sampleRatio = 0.01;
  tzt_options.terarkZipMinLevel = 2; // Start using TerarkZipTable from level 2
  // reset table...
  table_factory.reset(NewTerarkZipTableFactory(tzt_options, table_factory));
}
[ 88%] Built target terarkdb
[ 88%] Building CXX object CMakeFiles/table_reader_bench.dir/table/table_reader_bench.cc.o
[ 89%] Building CXX object CMakeFiles/cache_bench.dir/cache/cache_bench.cc.o
[ 89%] Building CXX object CMakeFiles/range_del_aggregator_bench.dir/db/range_del_aggregator_bench.cc.o
[ 90%] Building CXX object CMakeFiles/memtablerep_bench.dir/memtable/memtablerep_bench.cc.o
[ 90%] Linking CXX executable cache_bench
[ 91%] Linking CXX executable range_del_aggregator_bench
/usr/bin/ld: [ 91%] Linking CXX executable memtablerep_bench
libterarkdb.a(options.cc.o): in function `terarkdb::ColumnFamilyOptions::ColumnFamilyOptions()':
/usr/local/src/src/terarkdb/options/options.cc:132: undefined reference to `terarkdb::NewTerarkZipTableFactory(terarkdb::TerarkZipTableOptions const&, std::shared_ptr<terarkdb::TableFactory>)'
/usr/bin/ld: [ 92%] Linking CXX executable table_reader_bench
libterarkdb.a(options.cc.o): in function `terarkdb::ColumnFamilyOptions::ColumnFamilyOptions()':
/usr/local/src/src/terarkdb/options/options.cc:132: undefined reference to `terarkdb::NewTerarkZipTableFactory(terarkdb::TerarkZipTableOptions const&, std::shared_ptr<terarkdb::TableFactory>)'
/usr/bin/ld: collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/cache_bench.dir/build.make:108: cache_bench] Error 1
make[1]: *** [CMakeFiles/Makefile2:293: CMakeFiles/cache_bench.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
libterarkdb.a(options.cc.o): in function `terarkdb::ColumnFamilyOptions::ColumnFamilyOptions()':
/usr/local/src/src/terarkdb/options/options.cc:132: undefined reference to `terarkdb::NewTerarkZipTableFactory(terarkdb::TerarkZipTableOptions const&, std::shared_ptr<terarkdb::TableFactory>)'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/range_del_aggregator_bench.dir/build.make:108: range_del_aggregator_bench] Error 1
make[1]: *** [CMakeFiles/Makefile2:363: CMakeFiles/range_del_aggregator_bench.dir/all] Error 2
/usr/bin/ld: libterarkdb.a(options.cc.o): in function `terarkdb::ColumnFamilyOptions::ColumnFamilyOptions()':
/usr/local/src/src/terarkdb/options/options.cc:132: undefined reference to `terarkdb::NewTerarkZipTableFactory(terarkdb::TerarkZipTableOptions const&, std::shared_ptr<terarkdb::TableFactory>)'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/memtablerep_bench.dir/build.make:108: memtablerep_bench] Error 1
make[1]: *** [CMakeFiles/Makefile2:328: CMakeFiles/memtablerep_bench.dir/all] Error 2
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/table_reader_bench.dir/build.make:108: table_reader_bench] Error 1
make[1]: *** [CMakeFiles/Makefile2:398: CMakeFiles/table_reader_bench.dir/all] Error 2
make: *** [Makefile:136: all] Error 2
yapple commented 2 years ago

add header try again

include "table/terark_zip_table.h"

vipsland commented 2 years ago

i did but i get...

[ 91%] Linking CXX executable range_del_aggregator_bench
[ 91%] Linking CXX executable memtablerep_bench
[ 92%] Linking CXX executable table_reader_bench
/usr/bin/ld: /usr/bin/ld: libterarkdb.a(options.cc.o): in function `terarkdb::ColumnFamilyOptions::ColumnFamilyOptions()':
/usr/local/src/src/terarkdb/options/options.cc:131: undefined reference to `terarkdb::NewTerarkZipTableFactory(terarkdb::TerarkZipTableOptions const&, std::shared_ptr<terarkdb::TableFactory>)'
libterarkdb.a(options.cc.o): in function `terarkdb::ColumnFamilyOptions::ColumnFamilyOptions()':
/usr/local/src/src/terarkdb/options/options.cc:131: undefined reference to `terarkdb::NewTerarkZipTableFactory(terarkdb::TerarkZipTableOptions const&, std::shared_ptr<terarkdb::TableFactory>)'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/cache_bench.dir/build.make:108: cache_bench] Error 1
make[1]: *** [CMakeFiles/Makefile2:293: CMakeFiles/cache_bench.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/range_del_aggregator_bench.dir/build.make:108: range_del_aggregator_bench] Error 1
make[1]: *** [CMakeFiles/Makefile2:363: CMakeFiles/range_del_aggregator_bench.dir/all] Error 2
/usr/bin/ld: collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/memtablerep_bench.dir/build.make:108: memtablerep_bench] Error 1
make[1]: *** [CMakeFiles/Makefile2:328: CMakeFiles/memtablerep_bench.dir/all] Error 2
libterarkdb.a(options.cc.o): in function `terarkdb::ColumnFamilyOptions::ColumnFamilyOptions()':
/usr/local/src/src/terarkdb/options/options.cc:131: undefined reference to `terarkdb::NewTerarkZipTableFactory(terarkdb::TerarkZipTableOptions const&, std::shared_ptr<terarkdb::TableFactory>)'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/table_reader_bench.dir/build.make:108: table_reader_bench] Error 1
make[1]: *** [CMakeFiles/Makefile2:398: CMakeFiles/table_reader_bench.dir/all] Error 2
make: *** [Makefile:136: all] Error 2
yapple commented 2 years ago

include

include "rocksdb/db.h"

include "rocksdb/options.h"

include "rocksdb/table.h"

include "table/terark_zip_table.h"

rocksdb::DB* db; rocksdb::Options options;

// Your db options here options.create_if_missing = true; options.wal_bytes_per_sync = 32768; options.bytes_per_sync = 32768;

// TerarkZipTable need a fallback options because you can indicate which LSM level you want to start using TerarkZipTable // For example, by setting tzt_options.terarkZipMinLevel = 2, TerarkDB will use your fallback Table on level 0 and 1. std::shared_ptr table_factory; rocksdb::BlockBasedTableOptions blockbased_options; blockbased_options.block_size = 8ULL << 10; table_factory.reset(NewBlockBasedTableFactory(blockbased_options));

rocksdb::TerarkZipTableOptions tzt_options; // TerarkZipTable requires a temp directory other than data directory, a slow device is acceptable tzt_options.localTempDir = "/tmp"; tzt_options.indexNestLevel = 3; tzt_options.sampleRatio = 0.01; tzt_options.terarkZipMinLevel = 2; // Start using TerarkZipTable from level 2

table_factory.reset( rocksdb::NewTerarkZipTableFactory(tzt_options, table_factory));

options.table_factory = table_factory;

// Open DB auto status = rocksdb::DB::Open(options, "/tmp/testdb2", &db);

// Operations std::string value; auto s = db->Put(rocksdb::WriteOptions(), "key1", "value1"); s = db->Get(rocksdb::ReadOptions(), "key1", &value); assert(s.ok()); assert("value1" == value);

s = db->Delete(rocksdb::WriteOptions(), "key1"); assert(s.ok());

is this your need?

vipsland commented 2 years ago

no, i mean i'm ediitng this file directly https://github.com/bytedance/terarkdb/blob/dev.1.4/options/options.cc

how to force it to use terarkzip as default? (doing so to make my language binding easier. no need to write binding)

the code i shown is not working