GoogleCodeExporter
commented
8 years ago While they have the option to perform random access, most (but not all)
serializers
access the buffer sequentially and could indeed be written using
InputStream/OutputStream. However, the first time random access is needed, a
stream-based approach degrades to the current situation where a (possibly large)
buffer is required.
For example, the Compressor class wraps a Serializer to perform compression and
decompression. It must write the length of the compressed data, then the
compressed
data. The length is needed first because the compressed object may not be at
the root
of the object graph, and upon decompression we need to know how many bytes to
process. For this to be stream-based, the Compressor class would have to do its
own
buffering of the output data so it could write the length to the OutputStream
before
the data.
As you mentioned, an interface could be passed around instead of ByteBuffer to
hide
whether ByteBuffers or streams are used. The question is, would this API change
be
worth it? The current approach is ideal if you need a byte array or ByteBuffer
in the
end anyway, or if you need to write the length first (eg, when writing to a
stream-based protocol such as TCP). The current approach is less ideal when you
have
an arbitrarily large object graph, because you may not know the serialized size
beforehand.
You may find the ObjectBuffer class useful. It provides methods to serialize and
deserialize using byte arrays and streams. It handles the necessary buffering.
It can
be given an initial and maximum size. If an operation fails because the buffer
is too
small, its size will be doubled (up to the maximum) and the operation retried.
In a multithreaded environment, if an ObjectBuffer per thread is too much memory
(thread count * max size), it may be acceptable to use a thread safe pool of
ObjectBuffers. The pool size would be less than the number of threads. When
necessary, threads would block until an ObjectBuffer becomes available.
To serialize an object graph, the graph is normally going to fit in memory. The
buffer size needed to serialize the object graph is normally going to be less
than
the memory used by the Java object representation. These days memory is
abundant. I'm
not sure how much trouble it is worth to try to avoid buffering the serialized
bytes.
Is it possible that this bug is more about the unfriendliness of the API when
the
buffer is too small? Maybe ObjectBuffer should make its ByteBuffer available,
so the
same buffer growing functionality would be available to users who need a
ByteBuffer
rather than a byte array or stream.Original comment by nathan.s...@gmail.com on 21 Mar 2010 at 12:43
Original issue reported on code.google.com by
kezzlera...@gmail.comon 20 Mar 2010 at 1:29