Opening existing database in read-only mode fails

[ghost]

I have to compatibility with c code.

(headerFile's type is uint64_t, val is 0)

  MDB_txn *otxn;
  mdb_txn_begin(_env, NULL, 0, &otxn);
  char dbName[21];
  sprintf(dbName, "%" PRIu64, headFile);
  mdb_dbi_open(otxn, dbName, MDB_CREATE, &_db);
  mdb_txn_commit(otxn);

go code:

topic.currentPartitionID's val is 0

err = env.View(func(txn *lmdb.Txn) error {
        topic.currentPartitionDB, err = txn.CreateDBI(uInt64ToString(topic.currentPartitionID))
        return err
    })
    if err != nil {
        log.Println("Call env.View failed: ", err)
        return err
    }
}

when run, I got error:

Call env.View failed: mdb_dbi_open: permission denied

How to debug this?

[bmatsuo]

When debugging this kind of stuff its useful to go back to the C documentation. The error says that mdb_dbi_open failed with the error "permission denied" (i.e. the errno value EPERM). So it is best to see if the docs for that function make any mention regarding the meaning behind that error code.

http://lmdb.tech/doc/group__mdb.html#gac08cad5b096925642ca359a6d6f0562a

In this case, EPERM is not mentioned. But it does make mention of read-only transactions with regards to the MDB_CREATE flag, which is implied by calling Go method txn.CreateDBI().

So, first I would try calling txn.OpenDBI() instead of of txn.CreateDBI(). I suspect if you call txn.OpenDBI() with no flags you will see a different error.

You may want to make sure that the C code is checking for errors. Also, printing out the name of the database in both programs will help ensure the database name strings are, infact the same in both programs.

If you are ever unsure about which databases actually exist in an environment, it is also possible to iterate over the root database to find all of the existing named databases. Code like the following will print the names of all databases (along with any other keys that happen to be stored in the root database).

err = env.View(func(txn *lmdb.Txn) (err error) {
    dbi, err := txn.OpenRoot(0)
    if err != nil {
        return err
    }
    scanner := lmdbscan.NewScanner(txn, dbi)
    for scanner.Scan() {
        log.Printf("DATABASE %q", scanner.Key())
    }
    return scanner.Err()
})

Note: I am using the exp/lmdbscan package instead of creating a Cursor because it is simpler, but a simple Cursor iteration is possible too. Also, I didn't actually compile the above code so there may be minor errors.

Let me know if you have questions about this.

[ghost]

Thank you @bmatsuo , In C code, I used to store data in root database, and no named database used. After changed to use named db to compatibility with CreateDBI, I forgot to set maxdbs before mdb_dbi_open, which cause db doesn't exist, and got this error. Now fixed

[ghost]

@bmatsuo Is there a way to use unamed database? I want to impl a queue like kafka, which an env means a partition. So there only one db in an env. It's too strange to set Maxdbs to 1. Is there any danger to store data into unamed database?

[bmatsuo]

@zwb-ict If you do not intend to use more than one database per environment, you may use OpenRoot alone. And you do not need call SetMaxDBs if you only call OpenRoot. You can even set flags on the root database when you first open it.

The only problem that can arise is if you decide to use more than one database in a future version of your application. You cannot have a database named "FOO" and a data item stored in a key called "FOO" in the root database. Furthermore, iterating over key-value pairs in the root database will become tricky because the cursor will iterate over items representing each named database (as I demonstrated in my code example from a previous comment) in addition to any data items your application has stored in the root database.

Other than the problems above I am not aware of other issues that present themselves when storing data in the root database. Indeed, feel free to use the root database to store data if your application architecture allows it.

I personally recommend always using a named database because it let's your application stay flexible. I don't think the additional call to SetMaxDBs is a significant issue. But it is a personal preference.

[ghost]

Thank you @bmatsuo, I think you are right. Keep the application stay flexible is more important. I'll keep using named database.