Suggestion: builtin support for byte buffers

[lvh]

Similar to byte arrays but ... different!

[lvh]

FWIW I started writing this and got here:

(nippy/extend-freeze
 ByteBuffer ::byte-buffer
 [^ByteBuffer buf data-output]
 (let [arr (byte-array (.capacity buf))]
   (-> buf .duplicate .clear (.get arr))
   (#'nippy/write-bytes data-output arr)))

(nippy/extend-thaw
 ::byte-buffer
 [data-input]
 (ByteBuffer/wrap (#'nippy/read-bytes data-input len???)))

I figured it'd make sense to use nippy's already extant support for byte arrays, but I couldn't figure out how the length prefixing would work.

[ptaoussanis]

Hi @lvh, getting this implemented should be straight forward (I can assist) - but it's not obvious to me this'd be a good idea. Can you please motivate the use-case a little if you're still interested in this?

Some concerns off the top of my head:

• What if it's a non-direct byte buffer?
• Should we also de/serialize the buffer's position?
• What if another thread is mutating the buffer while serializing?

Thanks!

[lvh]

Great questions! I don't know that my preferences are universal but I can tell you what I did:

• All byte buffers are treated equally.
• The entire byte buffer is serialized.
• You get incorrect behavior.

I appreciate that this might not be behavior you want to happen silently though.

[lvh]

FWIW: I feel like the answers should be pretty similar than for byte-arrays, right? You'd also get into trouble if a different thread mutates it while you're trying to serialize.

[ptaoussanis]

The entire byte buffer is serialized.

So just to clarify: you're talking about also capturing state like capacity, limit, position, limit? Mark as well?

[lvh]

Sorry, yeah, that was ambiguous. I would expect capacity is captured, but limit and position are not; the deserialized buffer should have a pos of 0 and a limit of cap, but otherwise have the same information as the underlying array (assuming there is one and it is contiguous). In my experience limit and position are "transient" qualities about a process you're almost certainly not going to capture anyway, but capacity (of course) is not.

[ptaoussanis]

Think it may be fair to close this.

I'm hesitant to implement given the stated concerns, and the apparent lack of demand over the years.

And to add a thought: Nippy intentionally doesn't support de/serialization of atoms. On the surface it seems like it'd be a reasonable+convenient to support them - but my concern is that it would be easy to get surprising behaviour.

I'd prefer that folks explicitly deref to get a point-in-time value before freezing. Likewise, I think similar semantics should maybe apply here.

I hope that seems reasonable?