search

vedang / clj_fdb

A thin Clojure wrapper for the Java API for FoundationDB.
https://vedang.github.io/clj_fdb/
Eclipse Public License 1.0
25 stars 9 forks source link

String mismatch -- what we store is not what we get #25

Closed manish-2014 closed 3 years ago

manish-2014 commented 3 years ago

Describe the bug I used me.vedang.clj-fdb.core/set method to store a string in the foundationdb with a key. When I use the key to retrieve stored value with the same key, I do get a string back whic look similar to the saved string but the string comparison fails.

To Reproduce Please see my test code below

Please see test source code gist for full test code

Anyways this is the test that is failing: ` (deftest a-test (testing "string set get test." (let [ value-to-store "really important stuff" key-tpl (ftup/from "level1" "level2" "level3") val-tpl (ftup/from value-to-store) _ (let [fdb (cfdb/select-api-version cfdb/clj-fdb-api-version)] (with-open [db (cfdb/open fdb)] (fc/set db key-tpl val-tpl)))

      val-from-fdb (let [fdb (cfdb/select-api-version cfdb/clj-fdb-api-version)]
        (with-open [db (cfdb/open fdb)] (fc/get db key-tpl :valfn #(bs/convert % String))))
      ]
  (is (= val-from-fdb value-to-store))     
  )

))

`

Expected behavior The string"val-from-fdb" should be equal to string " value-to-store" . Whne you print the string val-from-fdb , you will see that it looks same as the value-to-store. However, the string comparison fails constitently for any value of "value-to-store" string.

It seems to me that "byte-streams" is either adding 2 extra bytes .

Additional context jvm : openjdk 11

manish-2014 commented 3 years ago

I just realized that I could pass :valfn #(-> % ftup/from-bytes ftup/get-items first str) as the value function and get the "correct" string back. I am concerned that this package relies upon byte-streams for serailizing the keys and values.

vedang commented 3 years ago

I agree with you @manish-2014 , byte-streams is unnecessarily heavy for the requirements of this library. I've opened #17 sometime ago to address this problem, I'll make the change as soon as I can!

In the meanwhile, PRs for this as welcome!

vedang commented 3 years ago

Closing this ticket. While I have removed byte-streams from the library (yet to push the change), there was no problem with byte-streams in the above example. If you look carefully at the code, you will see that you are encoding the value into a tuple using (ftup/from value-to-store) but decoding it into string directly using #(bs/convert % String) . This is why it is not working. If you decode with #(bs/convert % Tuple) it will work correctly.

In the upcoming release, I have removed byte-streams and simplified the API of the library a bit.