Strech / avrora

A convenient Elixir library to work with Avro messages, schemas and Confluent® Schema Registry
https://hexdocs.pm/avrora
MIT License
98 stars 33 forks source link

"Messages encoded with OCF are wrapped in a List" what for? #79

Closed romul closed 3 years ago

romul commented 3 years ago

Is there any reason why messages encoded with Codec.ObjectContainerFile are wrapped in a list?

I propose to rewrite ObjectContainerFile.decode/1 as:

  def decode(payload) when is_binary(payload) do
    with {:ok, {_, _, [decoded]}} <- do_decode(payload), do: {:ok, decoded}
  end
Strech commented 3 years ago

Hi @romul, thanks for you proposal! Unfortunately, I can't accept it because it is done to comply with the Avro specification.

The concept of the OCF is well described by this quote from the official docs

Avro includes a simple object container file format. A file has a schema, and all objects stored in the file must be written according to that schema, using binary encoding. Objects are stored in blocks that may be compressed.

So in other words you have a container (file, or bucket so to say) and you stuff it with objects (multiple records or a special case - 1 record). And because of that we should keep a list, even tho for the case with 1 record it seems redundant.