Handling strigns bigger than 64K

[ghost]

Hi,

readUTF and writeUTF cannot handle strings more than 64K in length.

I wrote these to specify the length as an int:

(defn ^{:private true} write-long-string [^DataOutputStream stream ^String string](let [ba %28.getBytes string) size (alength ba)] (.writeInt stream size) (.write stream ba 0 size)))

(defn ^{:private true} read-long-string ^String [^DataOutputStream stream](let [size %28.readInt stream%29 ba %28byte-array size%29 _ %28.read stream ba 0 size%29] %28String. ba)))

and used them for these entries:

(freezer String id-string (write-long-string s x)) (freezer Object id-reader (write-long-string s (pr-str x)))

and in thaw-from-stream!*

 id-reader  (read-long-string s)

.. id-string (read-long-string s)

I tested these with several hundred kilobyte strings and it works flawlessly. I chose to use an integer length, a long might be an overkill. Up to you to change it to long if you see the need. I did not want to optimize this further, a two bytes "penalty" looks reasonable to me to avoid the whole issue.

Thank you,

Luc P.

[ptaoussanis]

Hi Luc,

Thanks for getting in touch about this- nice catch! I didn't realise the .writeUTF/.readUTF stuff was limited like that.

I've pushed v0.9.2 to GitHub and Clojars with a fix (it's backwards compatible with 0.9.1 for thawing).

Please let me know if you have any further problems!

Cheers,

Peter Taoussanis ptaoussanis@gmail.com

• 66 (0)82 597 2595

[ghost]

Welcomed,

We use nippy as part of an optimization effort.

Thank you,

Luc

Hi Luc,

Thanks for getting in touch about this- nice catch! I didn't realise the .writeUTF/.readUTF stuff was limited like that.

I've pushed v0.9.2 to GitHub and Clojars with a fix (it's backwards compatible with 0.9.1 for thawing).

Please let me know if you have any further problems!

Cheers,

Peter Taoussanis ptaoussanis@gmail.com

• 66 (0)82 597 2595

---

Reply to this email directly or view it on GitHub: https://github.com/ptaoussanis/nippy/issues/1#issuecomment-7139091

Softaddictslprefontaine@softaddicts.ca sent by ibisMail from my ipad!