Closed gedw99 closed 11 months ago
Hello @gedw99, thanks for your interest in this project ! :)
To be honest, I don't have plan for this (part of it being because I'm not that familiar with event broker yet and haven't been confronted to it yet), but I'm familiar with the concept of migration for SQL databases, so not completely lost. I'll take a look to existing solutions to see what's could be the best.
However, I would be glad to have some advice on how I could handle it if you're familiar with it.
For now, only NATS and JSON are implemented, but we could see other brokers (Kafka, RabbitMQ, etc) and other protocols (such as protobuf) as well (even if a part of me feels that even with that, we should have a solution for classic JSON transmission).
What do you think ?
I added a possible solution using protobufs while you were writing your answer :)
And regarding other messsge brokers … I prefer nats due to operational simplicity.
that’s just my own bias though.
Well, I think I'll take a look how to add protobuf to this project soon ! I'll have to check also how NATS handles binary messages (I'm more familiar with text in NATS).
I'll keep you updated, but feel free to keep in touch with me if you have any idea or advice :)
Nats has built in proto support and json support.
https://github.com/nats-io/natscli#translating-message-data-using-a-converter-command Was added to allow devs to convert to other types
Thanks ! I was about to say 'nevermind, I forgot that I already send text as []byte{}
in the NATS implementation', it's already in binary :smile:
It should be too hard then. I'll keep you updated !
I added the versioning support in the last version of this repository: https://github.com/lerenn/asyncapi-codegen#versioning
So I won't be implementing the protobuf support for now, but let me know if that's a problem !
Do you have a game plan of how your going to support Schema Evolution ?
We have things in 2 places .. There is a schema description in the yaml and the data living inside nats that conforms to a version of the types described by the yaml schema.
this is not at all unique to your project but in fact an issue with nats itself in my opinion.
The problem is reduced if protobufs are used instead of json . It async-api wants to use json . So then a work around is if we can store the data inside nats as protobuf and convert it to json on the way in and out … So then you won’t need to go back and adjust the pending data inside nats.
just an idea..
anyways it’s a cool project @lerenn