Open slominskir opened 3 years ago
mhowlett
commented
3 years ago avro-gen was not on our radar (seems to be new). It may be a good approach, we haven't looked at it.
We need to re-visit all of this and get Python serdes in a better state. I plan on doing this, but have competing priorities. Feel free to leave any opinions you may have, we'll take note.
slominskir
commented
3 years ago I haven't formed an opinion yet, but digging a little I found some additional information to consider:
@scottbelden is probably more knowledgeable about this.
The advantages of a class over a dict include:
The disadvantages may include:
scottbelden
commented
3 years ago If I were to re-write fastavro from scratch I would probably have it use classes (or dataclasses) instead of regular dictionaries. It would solve some of the ambiguity problems that currently exist in fastavro and could theoretically allow for faster serialization/deserialization since the classes could be generated with custom, schema specific encoders/decoders.
However, that's all a large enough change that it would almost certainly have to be in a completely new version two library and so I don't imagine fastavro ever moving away from using dictionaries for things like records.
slominskir
commented
3 years ago So when does fasteravro (fastavro2) come out? 👍
scottbelden
commented
3 years ago Whenever someone decides to write it 😛
Description
Is there any discussion on how to handle serde transformation from a Python Dictionary to a Class? The Consumer API for example currently accepts a callable function from_dict and is demonstrated in the examples.
This appears to preclude using a generator such as avro-gen, since it apparently simply wraps the Dictionary instead of transform it and toss it out as is done with _fromdict. Maybe avro_gen wrapping is a bad approach? Maybe classes aren't "Pythonic" anyways? Just trying to figure out how best to use the confluent_kafka_python API in my own code and figure someone has already thought about this.
How to reproduce
This is an API / Documentation question.
Checklist
None