Open yanivbh1 opened 1 year ago
Just an idea maybe its possible to implement a plug-able protocol adapter and let the community build any kind of adapter they would like, Pulsar did this pretty well and already have support for kafka, amqp(rabbit, mqtt and more
https://github.com/apache/pulsar/wiki/PIP-41%3A-Pluggable-Protocol-Handler
Hey @danielsinai, Adding @idanasulinmemphis to this thread as well. Definitely. Most of the pieces in Memphis would be modular and extensible by the community as much as possible, perhaps that's the place to start. We'll create a discussion on that, would love to learn more from your perspective.
GitHub discussion #857
Let's continue with the discussion over here https://github.com/orgs/memphisdev/discussions/857 please
Hi @yanivbh1 I'd like to participate Please assign it to me
Some notes regarding modular adapter:
Personally I think that using dedicated process per adapter it's preferred mode at least for protocols required web server.
[Performance] Possible changes in the future on the server side - support unix domain sockets for negotiation with modular adapter
Dev environment:
If will be decided that one process should be host for different adapters - also repository
@g41797 https://github.com/memphisdev/memphis-protocol-adapter I completely agree with your approach, and regarding the name itself, worst-case scenario, it will be changed to your suggested name.
Adding my 2 cents here, I think that Kafka API should be the highest priority... currently tons of companies use Kafka, or better to say misuse Kafka and they could replace it with Memphis seamlessly if the Kafka protocol would be supported
@g41797, the next step of the syslog adapter would be to combine it with the REST gateway, meaning that both will be served by one component to avoid deploying yet another container just for a protocol. It means taking the work that has already done with the REST GW and combining it based on the pattern you implemented in the syslog adapter or vice-versa.
memphis-protocol-adapter repo is in the same state as memphis cli i don't feel than anyone needs this project also i asked to provide me permissions to merge forked repo with syslog and prepare memphis-protocol-adapter to production but sill didn't get any response i don't see technical restriction to combine rest and syslog, but at least syslog should be tested alone in production mode any requirement from customers side - how this information should be consumed, protocols, number of simm. clients etc i did not see unit tests within rest-gateway. I hope there are external tests. How to combine and/or create simm. tests for two adapters within one process How to validate results Let's talk
Opened #1402
Information
Currently, The main protocols supported are NATS using Memphis client SDKs and REST through the REST GW. NATS will continue to be the main protocol, but the REST Gw component should be refactored into a modular adapter where other developers and contributors would be able to add more protocols to that adapter/component.
High-level plan
Protocols roadmap (1-highest)
GitHub discussion #857