Data read, but end of buffer not reached

[anywhichway]

I am occasionally getting Data read, but end of buffer not reached errors after a single putSync into a root database:

const id = db.putSync("Person@1",{ "name": "joe", "age": 21, "address": { "city": "New York", "state": "NY" }, "created": "@Date(1684608677657)", "aRegExp": "@RegExp(/abc/)", "unindexed": "unindexed", "#": "Person@1" })

 Data read, but end of buffer not reached 0

      425 |     if(keys.length===0 && scan) {
      426 |         const start = cname+"@";
    > 427 |         for(const entry of this.getRange({start})) {

However, db.get(id) works as expected.

I do not experience this with child databases.

I have been unable to create a simple example independent of the rest of my code. If I change the encoding type of json, then I get a parse error for an unexpected ,. Seems odd I should be able to read the object but not do a getRange.

[anywhichway]

The error gets thrown on the last iteration across getRange. Some terminating character must not be getting written during a transaction. I have wrapped my code in a try/catch block for now and it works. However, it does make me a little nervous.

[kriszyp]

Are you doing this in the root (null name) database and have child databases? Child databases are entries in the root database, and have a special LMDB encoding that throws this error when attempting to decode, so generally you can't have entries in the root database and child databases (if you are going to have child/named databases, best to keep all data in child/named databases).

[kriszyp]

I realize this is a hazard and poorly documented, I will see if I can put some guardrails in place for this.

[anywhichway]

Aha!

No big deal, will just create another child for the core data, the others are indexes.

On Sun, May 21, 2023 at 8:09 AM Kris Zyp @.***> wrote:

Are you doing this in the root (null name) database and have child databases? Child databases are entries in the root database, and have a special LMDB encoding that throws this error when attempting to decode, so generally you can't have entries in the root database and child databases (if you are going to have child/named databases, best to keep all data in child/named databases).

— Reply to this email directly, view it on GitHub https://github.com/kriszyp/lmdb-js/issues/234#issuecomment-1556203137, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABF2US2IMZYDS5GSC6SC7FLXHIVZPANCNFSM6AAAAAAYI4T424 . You are receiving this because you authored the thread.Message ID: @.***>

[anywhichway]

I have confirmed using another child and not the root fixes this. You can close if you wish, or leave open to drive the documentation/code updates you desire.

Let me know if you need a hand. I can fork and request a pull if necessary on the doc side of things.

[burrana-joshuaattlee]

Hey, I just came up against this exact issue. Would be good if the openDb documentation was updated to note this, and perhaps a Common Error FAQ added - also happy to fork because I think the 'encoder' option is missing from the typing?