rockset / rocksdb-cloud

A library that provides an embeddable, persistent key-value store for fast storage optimized for AWS
http://rocksdb.org
GNU General Public License v2.0
734 stars 117 forks source link

db_bench asserts in RemapFilename #38

Open dhruba opened 6 years ago

dhruba commented 6 years ago

I am testing db_bench on aws, and when I run this:

./cloud/benchmarks/bulkload_seq.sh

I get this stack trace:

Load 500M keys of size 1 KB each sequentially into rocksdb-cloud.....
      AwsEnv.src_bucket_prefix: dbbench.dhruba
      AwsEnv.src_object_prefix: 
      AwsEnv.src_bucket_region: us-west-2
     AwsEnv.dest_bucket_prefix: dbbench.dhruba
     AwsEnv.dest_object_prefix: 
     AwsEnv.dest_bucket_region: us-west-2
            AwsEnv.credentials: [given]
AwsEnv connection to endpoint in region: us-west-2
Initializing RocksDB Options from the specified file
Initializing RocksDB Options from command-line flags
db_bench: cloud/cloud_env.cc:84: std::__cxx11::string rocksdb::CloudEnvImpl::RemapFilename(const string&) const: Assertion `cloud_manifest_' failed.
Received signal 6 (Aborted)
#0   /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38) [0x7ff3191ee428] ??  ??:0    
#1   /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a) [0x7ff3191f002a] ??   ??:0    
#2   /lib/x86_64-linux-gnu/libc.so.6(+0x2dbd7) [0x7ff3191e6bd7] ??  ??:0    
#3   /lib/x86_64-linux-gnu/libc.so.6(+0x2dc82) [0x7ff3191e6c82] ??  ??:0    
#4   ./db_bench() [0x868baa] rocksdb::CloudEnvImpl::RemapFilename(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const /data/home/dhruba/rocksdb-cloud/cloud/cloud_env.cc:84 (discriminator 1) 
#5   ./db_bench() [0x840c5b] rocksdb::AwsEnv::NewWritableFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<rocksdb::WritableFile, std::default_delete<rocksdb::WritableFile> >*, rocksdb::EnvOptions const&) /data/home/dhruba/rocksdb-cloud/cloud/aws/aws_env.cc:611    
#6   ./db_bench() [0x7550d5] rocksdb::NewWritableFile(rocksdb::Env*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<rocksdb::WritableFile, std::default_delete<rocksdb::WritableFile> >*, rocksdb::EnvOptions const&)  /data/home/dhruba/rocksdb-cloud/util/file_reader_writer.cc:653  
#7   ./db_bench() [0x5d3917] rocksdb::DBImpl::NewDB()   /data/home/dhruba/rocksdb-cloud/db/db_impl_open.cc:232 (discriminator 1)    
#8   ./db_bench() [0x5d917c] rocksdb::Status::operator=(rocksdb::Status&&)  /data/home/dhruba/rocksdb-cloud/./include/rocksdb/status.h:295  
#9   ./db_bench() [0x5d9ece] rocksdb::DBImpl::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<ro ksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**, bool)    /data/home/dhruba/rocksdb-cloud/db/db_impl_open.cc:1025 (discriminator 1)   
#10  ./db_bench() [0x5db7d3] rocksdb::DB::Open(rocksdb::Options const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rocksdb::DB**)  /data/home/dhruba/rocksdb-cloud/db/db_impl_open.cc:961  
#11  ./db_bench() [0x5293de] rocksdb::Status::operator=(rocksdb::Status&&)  /data/home/dhruba/rocksdb-cloud/./include/rocksdb/status.h:295  
#12  ./db_bench() [0x535dde] rocksdb::Options::~Options()   /data/home/dhruba/rocksdb-cloud/./include/rocksdb/options.h:918 
#13  ./db_bench() [0x537190] rocksdb::Benchmark::Run()  /data/home/dhruba/rocksdb-cloud/tools/db_bench_tool.cc:2406 
#14  ./db_bench() [0x5120d4] rocksdb::db_bench_tool(int, char**)    /data/home/dhruba/rocksdb-cloud/tools/db_bench_tool.cc:5487

@igorcanadi : is this related to CloudManifest cf5e241a3ca38ff4feabae335bae04309f716537

igorcanadi commented 6 years ago

@dhruba This happens because we open the database with DB::Open() instead of DBCloud::Open() in db_bench, while feeding it the CloudEnv. We should add the better error in case that happens, though.

dhruba commented 6 years ago

I enabled db_basic_test on travis. This test actually uploads data to S3 bucket. I will verify it and check why db_bench fails.