Now that @arhamchopra has taken care of fast json conversion, we should move on to fast Apache Arrow conversion. We should have 2 mechanisms for conversion:
converting an individual struct, necessary but probably less efficient overall
collecting lists of structs and converting to record batches on either a timer or after a certain window size
Native arrow conversion will make certain use cases substantially faster, including in our csp-based webserver framework, but should also have benefits for other adapters like Kafka and websockets.
Now that @arhamchopra has taken care of fast json conversion, we should move on to fast Apache Arrow conversion. We should have 2 mechanisms for conversion:
Native arrow conversion will make certain use cases substantially faster, including in our csp-based webserver framework, but should also have benefits for other adapters like Kafka and websockets.