with lmdb.Append get error: mdb_put: MDB_KEYEXIST: Key/data pair already exists

[ghost]

I call

 txn.Put(qp.partition, []byte("12345"), v.mem, lmdb.Append)

Got error:

mdb_put: MDB_KEYEXIST: Key/data pair already exists

Below is my code, the qp.partition type is lmdb.DBI, inited before I call push

qp.partition, err = txn.CreateDBI(uInt64ToString(qp.curPartitionID))

func (qp *QProducer) push(batch []QPItemType) error {
    isFull := false
    err := func() error {
        err := qp.qt.qe.env.Update(func(txn *lmdb.Txn) error {
            offset, err := qp.qt.getProducerMeta(txn)
            if err != nil {
                return errors.Wrap(err, "Call getProducerMeta failed")
            }
            for _, v := range batch {
                offset++
                err = txn.Put(qp.partition, []byte("12345"), v.mem, lmdb.Append)
                if err != nil {
                    if err == lmdb.MapFull {
                        isFull = true
                        break
                    }
                    return errors.Wrap(err, "Call txn.Put failed")
                }
            }
            if !isFull {
                qp.qt.updateProducerMeta(txn, offset)
            }
            return nil
        })
        if err != nil && err == lmdb.MapFull {
            isFull = true
            return nil
        }
        return errors.Wrap(err, "Call env.Update failed")
    }()

The loop is only once, and I delete db everytime before I run this test, also set the key to a sepcial key,

err = txn.Put(qp.partition, []byte("12345"), v.mem, lmdb.Append)

So It cann't EXIST in db.

I don't know how to debug this.

[ghost]

#MDB_APPEND - append the given key/data pair to the end of the database. No key comparisons are performed. This option allows fast bulk loading when keys are already known to be in the correct order. Loading unsorted keys with this flag will cause a #MDB_KEYEXIST error. I thought this may be the reason, after create a DB, how to APPEND without set compare functions? my data looks like: | Key | Value | | :-- | --: | | offset | data(bin) | | offset + 1 | data(bin) | | offset + 2 | data(bin) | | offset + 3 | data(bin) | | offset + 4 | data(bin) | | ... ... | ... ... | | offset + n | data(bin) |

[bmatsuo]

You say that the database is empty and you only call txn.Put() once? If that's the case then it doesn't seem like you should get that error (were that to happen, it would be a bug in the C library). So I wonder if something is wrong with your test, and if you are putting more items in than you realize.

You can check how many entries are in a db using txn.Stat(). Then you could, in fact, verify that the database is empty before your call to txn.Put() is made (by ensuring the stat's Entries field is 0).

As far as the key encoding goes -- it looks like they are supposed to be unsigned integers. Right? In that case you don't need a custom comparison function as long as you use binary.BigEndian to encode your keys. A big-endian encoding for uint64 will result in keys being sorted in increasing order, which appears to be what you want. If you use a little-endian encoding then the resulting keys would not sort in a desirable order.

[ghost]

Thank you @bmatsuo, you are right, I checked, and found I passed a wrong env.