search

dain / leveldb

Port of LevelDB to Java
Apache License 2.0
1.54k stars 429 forks source link

Is openDB slow? #56

Closed chakrit closed 9 years ago

chakrit commented 9 years ago

From some initial startup time profiling of the Android app I'm working on, this line seems to be the current biggest offender:

Options opts = new Options()
        .createIfMissing(true)
        .errorIfExists(false)
        .paranoidChecks(false)
        .compressionType(CompressionType.NONE)
        .cacheSize(cacheSize);

return Iq80DBFactory.factory.open(new File(path), opts);

It takes about a whole second or two on the open call.

Is this normal? What are some ways to optimize this?

rnarubin commented 9 years ago

If you're opening an existing database, you'll have to read from several files; a few of them are relatively small, e.g. the manifest, however the biggest culprit will be the log file, which contains the latest entries you'd written to the database that didn't get flushed to table files. This file can be as large as your opts.writeBufferSize(), and upon opening the database you have to read this file, parse the data, then write it into new tables.

So yes, a call to open can involve a lot of IO with both reads and writes to disk, which can be slow -- especially on Android if you're writing to an external SD card. As for optimizations, you could try lowering your writeBufferSize to try and lessen the IO on startup, but in general I wouldn't recommend this as your runtime performance would suffer, your writes would be less efficient.

chakrit commented 9 years ago

@rnarubin thanks for the explanation : ).

I guess there is no working around this besides opening in a background thread and adding code to route all db calls through it.