Valgrind: still reachable after system shutdown.

[konghaiya]

Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://groups.google.com/forum/#!forum/rocksdb or https://www.facebook.com/groups/rocksdb.dev

Expected behavior

There is accessible memory when the system is shut down

Actual behavior

problem 1

==00:00:02:03.368 32868== 216 bytes in 1 blocks are still reachable in loss record 179 of 270
==00:00:02:03.368 32868==    at 0x4C2B7DB: operator new(unsigned long) (vg_replace_malloc.c:422)
==00:00:02:03.368 32868==    by 0x2569FFC: rocksdb::(anonymous namespace)::CreateThreadStatusUpdater() (in /stonedb56/install/bin/mysqld)
==00:00:02:03.368 32868==    by 0x256E486: rocksdb::(anonymous namespace)::PosixEnv::PosixEnv() (in /stonedb56/install/bin/mysqld)
==00:00:02:03.368 32868==    by 0x256EA3C: rocksdb::Env::Default() (in /stonedb56/install/bin/mysqld)
==00:00:02:03.368 32868==    by 0x2451FEA: rocksdb::DBOptions::DBOptions() (in /stonedb56/install/bin/mysqld)
==00:00:02:03.368 32868==    by 0x1EFA7F6: rocksdb::Options::Options() (options.h:1104)
...

problem 2

==00:00:02:03.368 32868== 240 bytes in 1 blocks are still reachable in loss record 181 of 270
==00:00:02:03.368 32868==    at 0x4C2B7DB: operator new(unsigned long) (vg_replace_malloc.c:422)
==00:00:02:03.368 32868==    by 0x25F6F7D: rocksdb::ThreadLocalPtr::Instance() (in /stonedb56/install/bin/mysqld)
==00:00:02:03.368 32868==    by 0x25F6F3A: rocksdb::ThreadLocalPtr::InitSingletons() (in /stonedb56/install/bin/mysqld)
==00:00:02:03.368 32868==    by 0x256E9FF: rocksdb::Env::Default() (in /stonedb56/install/bin/mysqld)
==00:00:02:03.368 32868==    by 0x2451FEA: rocksdb::DBOptions::DBOptions() (in /stonedb56/install/bin/mysqld)
==00:00:02:03.368 32868==    by 0x1EFA7F6: rocksdb::Options::Options() (options.h:1104)
...

Steps to reproduce the behavior

We are using version 6.4.6

[akankshamahajan15]

Can you share a unit test of what you are trying to do causing this error?

[konghaiya]

#include <mutex>
#include <string>
#include <thread>

#include "rocksdb/compaction_filter.h"
#include "rocksdb/convenience.h"
#include "rocksdb/db.h"
#include "rocksdb/utilities/transaction_db.h"
#include "rocksdb/utilities/write_batch_with_index.h"

class KVStore final {
 public:
  void Init(); //Called at system startup
  void UnInit(); //Called when the system is shut down

 private:

  rocksdb::BlockBasedTableOptions bbto_;

  rocksdb::TransactionDB *rdb;

};

void KVStore::Init() {
  rocksdb::TransactionDBOptions txn_db_options;
  std::vector<std::string> cf_names;
  std::vector<rocksdb::ColumnFamilyDescriptor> cf_descr;
  std::vector<rocksdb::ColumnFamilyHandle *> cf_handles;
  rocksdb::Options options;
  options.create_if_missing = true;

  bbto_.no_block_cache = false;
  bbto_.cache_index_and_filter_blocks = true;
  bbto_.block_cache = rocksdb::NewLRUCache(stonedb_sysvar_index_cache_size << 20);
  options.table_factory.reset(rocksdb::NewBlockBasedTableFactory(bbto_));

  rocksdb::DBOptions db_option(options);
  auto rocksdb_datadir = kv_data_dir / ".index";
  int max_compact_threads = std::thread::hardware_concurrency() / 4;
  max_compact_threads = (max_compact_threads > 1) ? max_compact_threads : 1;
  db_option.max_background_compactions = max_compact_threads;
  db_option.max_subcompactions = max_compact_threads;
  db_option.env->SetBackgroundThreads(max_compact_threads, rocksdb::Env::Priority::LOW);
  db_option.statistics = rocksdb::CreateDBStatistics();
  // get column family names from manifest file
  rocksdb::Status status = rocksdb::DB::ListColumnFamilies(db_option, rocksdb_datadir, &cf_names);
  if (!status.ok() &&
      ((status.subcode() == rocksdb::Status::kNone) || (status.subcode() == rocksdb::Status::kPathNotFound))) {
    STONEDB_LOG(LogCtl_Level::INFO, "First init rocksdb, create default column family");
    cf_names.push_back(DEFAULT_CF_NAME);
  }

  rocksdb::ColumnFamilyOptions rs_cf_option(options);
  rocksdb::ColumnFamilyOptions index_cf_option(options);
  // Disable compactions to prevent compaction start before compaction filter is
  // ready.
  index_cf_option.disable_auto_compactions = true;
  index_cf_option.compaction_filter_factory.reset(new IndexCompactFilterFactory);
  for (auto &cfn : cf_names) {
    if (IsRowStoreCF(cfn))
      cf_descr.emplace_back(cfn, rs_cf_option);
    else
      cf_descr.emplace_back(cfn, index_cf_option);
  }
  // open db, get column family handles
  status = rocksdb::TransactionDB::Open(options, txn_db_options, rocksdb_datadir, cf_descr, &cf_handles, &rdb);
  if (!status.ok()) {
    throw common::Exception("Error opening rocks instance. status msg: " + status.ToString());
  }
   cf_manager.init(cf_handles);
   status = rdb->EnableAutoCompaction(cf_handles);
  // Enable compaction, things needed for compaction filter are finished
  // initializing
  status = rdb->EnableAutoCompaction(cf_handles);
  if (!status.ok()) {
    throw common::Exception("RocksDB: Failed to enable compaction.");
  }
}

void KVStore::UnInit() {
  // Stop all rocksdb background work
  rocksdb::CancelAllBackgroundWork(rdb->GetBaseDB(), true);

  if (bbto_.block_cache) bbto_.block_cache->EraseUnRefEntries();

  delete rdb;
  rdb = nullptr;
}

Our actual calling process is like the above code, which calls "init()" when the system is started and "uninit()" when the system is shut down

[riversand963]

Is this by any chance similar to #7183?

[konghaiya]

It's a similar situation. Do you conclude that this situation can be ignored?

[riversand963]

Right, that's my understanding. You can also build the test with asan and see what the result is.

[konghaiya]

Well, thank you very much.