myzhan / avroipc

Avroipc is a pure-go-implemented client for flume's avro source
MIT License
4 stars 6 forks source link

Split the client to the Flume and Avro IPC related parts #35

Closed vykulakov closed 4 years ago

vykulakov commented 4 years ago

Apache Avro Specification defines many aspects of communication between a client and a server using Avro encoding/decoding mechanisms and message transports. But it does not describe particular RPC interface of the server and types of requests, responses and errors - it is a responsibility of the server to provide a particular message protocol.

So, in fact, we have two logical clients in our single current client - the Flume client which implements the Flume RPC client interface and describes the format of messages and the Avro client which implements the Apache Avro Specification and describes all other things. These two clients are coupled highly and it is hard to understand where one client ends and another client starts.

As a result, having two separate clients gives us some advantages:

  1. It will be possible to implement other message protocols to support more servers.
  2. The code of the two clients will be much simpler and more understandable.
  3. It will be simpler to test the clients separately.