kriszyp / lmdb-js

Simple, efficient, ultra-fast, scalable data store wrapper for LMDB
Other
484 stars 39 forks source link

The reading of 255 chars length string value is different from writing one #91

Closed cftang0827 closed 2 years ago

cftang0827 commented 2 years ago

Hi, I really appreciate such awesome library, it's really handy and powerful. :) Thank you so much!

However, I got an issue about the difference between reading & writing of value string when the length of the string is 255 . There are no issues for 254 & 256, but only for 255. (string length)

Here is the sample code for testing,

const { open } = require('lmdb-store');

// 255 chars long string
const value = 'RRZG9A6I7xupPeOZhxcOcioFsuhszGOdyDUcbRf4Zef2kdPIfC9RaLO4jTM5JhuZvTsF09fbRHMGtqk7YAgu3vespeTe9l61ziZ6VrMnYu2CamK96wCkmz0VUXyqaiUoTPgzk414LS9yYrd5uh7w18ksJF5SlC2e91rukWvNqAZJjYN3jpkqHNOFchCwFrhbxq2Lrv1kSJPYCx9blRg2hGmYqTbElLTZHv20iNqwZeQbRMgSBPT6vnbCBPnOh1W'

console.log("length of long value string: ", value.length)

let main = async () => {
    // init lmdbStore
    let lmdbStore = open({
        path: "./data",
        compression: true,
        commitDelay: 10,
        dupSort: false,
        syncBatchThreshold: 100000000
    });
    console.log("Init lmdb success")
    await lmdbStore.put("key", value)
    let get = lmdbStore.get("key")
    console.log("get value: ", get, get.length)
    console.log("original value: ", value, value.length)

    if (get == value) {
        console.log("two string are the same")
    } else {
        console.log("two string are different")
    }
}
main()

And the result is

(base) ➜  lmdb-sdk-loading-test node testLong.js
length of long value string:  255
syncBatchThreshold is no longer supported
Init lmdb success
get value:  �RRZG9A6I7xupPeOZhxcOcioFsuhszGOdyDUcbRf4Zef2kdPIfC9RaLO4jTM5JhuZvTsF09fbRHMGtqk7YAgu3vespeTe9l61ziZ6VrMnYu2CamK96wCkmz0VUXyqaiUoTPgzk414LS9yYrd5uh7w18ksJF5SlC2e91rukWvNqAZJjYN3jpkqHNOFchCwFrhbxq2Lrv1kSJPYCx9blRg2hGmYqTbElLTZHv20iNqwZeQbRMgSBPT6vnbCBPnOh1 255
original value:  RRZG9A6I7xupPeOZhxcOcioFsuhszGOdyDUcbRf4Zef2kdPIfC9RaLO4jTM5JhuZvTsF09fbRHMGtqk7YAgu3vespeTe9l61ziZ6VrMnYu2CamK96wCkmz0VUXyqaiUoTPgzk414LS9yYrd5uh7w18ksJF5SlC2e91rukWvNqAZJjYN3jpkqHNOFchCwFrhbxq2Lrv1kSJPYCx9blRg2hGmYqTbElLTZHv20iNqwZeQbRMgSBPT6vnbCBPnOh1W 255
two string are different

I guess there could be some parsing issues or type casting issues in node js wrapper for C lib. Or there could be some wrong setting of mine, so I cannot get the correct result?

May you have a look if you have time? Thank you so much, and I really appreciate for your hard work :)

kriszyp commented 2 years ago

Thank you for the detailed report, and the encouragement! It looks like this was actually an issue msgpackr. I fixed the issue and published it in msgpackr v1.4.7. I will get an lmdb-store version published with the update dep too, soon.

kriszyp commented 2 years ago

Ok, v1.6.9 should point to the upgraded msgpackr. Let me know if you see any remaining issue.

cftang0827 commented 2 years ago

Hi @kriszyp, I really appreciate the update, according to my test, it seems like the bug had been fixed. Thank you! :)

截圖 2021-10-18 下午11 42 11