fxamacker / cbor

CBOR codec (RFC 8949) with CBOR tags, Go struct tags (toarray, keyasint, omitempty), float64/32/16, big.Int, and fuzz tested billions of execs.
MIT License
748 stars 61 forks source link

feature: stream infinite array #330

Open ghost opened 2 years ago

ghost commented 2 years ago

Is your feature request related to a problem? Please describe.

Currently, there is no API call to unmarshal an infinite array into a stream (e.g. a channel) of messages.

Describe the solution you'd like

I would like to be able to unmarshal an infinite array one item at a time.

Describe alternatives you've considered

Don't use infinite arrays and simply write the items back to back into the file.

Additional context

Use case: audit logging for ContainerSSH

fxamacker commented 2 years ago

@janosdebugs Thanks for suggesting this. This feature would involve either A. updating the API of decoder or B. creating new decoder type to handle indefinite length array

To keep the API and implementation simple, I prefer "B" and maybe limit stream decoding so we don't have to use the same decoder to stream decode indefinite arrays along with other data types at the same level.

Do you need to decode regular CBOR data and stream decode indefinite array using the same decoder? Also, do you need to stream decode nested infinite length array?

ghost commented 2 years ago

At this time we have a simple message format that involves creating a single infinite array and message structures within. It would be nice if we could decode them as they go through the pipe. We have no need for nested infinite arrays, and we are happy to use a separate decoder for this purpose.