josephg / node-foundationdb

Modern Node.js FoundationDB bindings
Other
115 stars 17 forks source link

Error when retrieving value after atomic add operation #57

Closed rizqme closed 3 years ago

rizqme commented 3 years ago

Hi, when I try to use add for counting I run to this issue that can be replicated as following:

    const dbRoot = open('', {
        transaction_retry_limit: 5,
        transaction_timeout: 5000,
    });

    let dir = await directory.createOrOpen(dbRoot, 'fdbtest');

    const db = dbRoot
        .at(dir)
        .withKeyEncoding(encoders.tuple)
        .withValueEncoding(encoders.json);

    await db.doTransaction(async tx => {
        tx.clearRangeStartsWith([]);
        tx.set(["counter"], 0);
        tx.add(["counter"], 1);
    })

    await db.doTransaction(async tx => {
        console.log(await tx.get(["counter"]));
    })
})()

It throws following error:

 undefined:1
a
^

SyntaxError: Unexpected token a in JSON at position 0
    at JSON.parse (<anonymous>)
    at Object.unpack (/home/rizqme/workspace/fdb-test/node_modules/foundationdb/dist/lib/index.js:79:35)
    at /home/rizqme/workspace/fdb-test/node_modules/foundationdb/dist/lib/transaction.js:181:76
rizqme commented 3 years ago

It looks like an issue with JSON encoder

rizqme commented 3 years ago

Ok, it seem to work with int32BE encoder. perhaps I will PR some documentation regarding atomic operation

josephg commented 3 years ago

Yeah - the atomic add / sub / etc operations require data to be encoded using raw bytes, not JSON. Eg from the python documentation:

The integers to be added must be stored in a little-endian representation. They can be signed in two’s complement representation or unsigned. You can add to an integer at a known offset in the value by prepending the appropriate number of zero bytes to param and padding with zero bytes to match the length of the value. However, this offset technique requires that you know the addition will not cause the integer field within the value to overflow.

Its a good idea to make this clearer in the documentation.