direct-transport missing features

orbs-network / orbs-network-go

Orbs node virtual chain core reference implementation in Go

MIT License

48 stars 12 forks source link

direct-transport missing features #182

Open talkol opened 6 years ago

talkol commented 6 years ago

services/gossip/adapter/direct_transport (this is the production e2e gossip for v1)

auth on connect and keepalive (TLS style challenge response)
exponential backoff for retries

talkol commented 6 years ago

Potential performance improvements (requires benchmarks):

Adding buffered IO may improve performance (bufio reader/writer) https://stackoverflow.com/questions/49580394/sending-millions-of-short-messages-over-tcp-socket-in-golang
Adding buffer pools may reduce allocations

talkol commented 6 years ago

Measure ping in keepalives (logging and reporting metrics of the turnaround time it takes to communicate with every node)

talkol commented 6 years ago

Improving the way we cleanup connections and goroutines when context is ended. Here is a proposal (unmerged branch) to do it with a watchdog: https://github.com/orbs-network/orbs-network-go/tree/feature/experimental-gossip-watchdog

talkol commented 6 years ago

Handle context intents

talkol commented 5 years ago

QoS for send queues

quoting from closed issue #273 (that provided initial spec):

Actually we will have more than one queue. The number of queues is configurable. Per queue you will need to provide config of: (a) max queue size in bytes (b) weight of the queue in terms of qos.
When the socket is free to send another message, it will pick one of the queues randomly according to the weights (only among the queues that actually have data waiting in them), and send a single message from this queue (dequeue it and send it over the socket in a blocking way).
The number of queues will be configurable. For every topic, we will configure which queue index it's part of. Theoretically we can have a separate queue per topic - or put all topics in the same queue.

electricmonk commented 5 years ago

Summary of tasks left to do, excluding performance optimizations; we need to decide which, if any, we deal with; some of these are product features and some are tech debt. @OdedWx - thoughts?

[ ] QoS for Send Queues
[ ] Handle context intents
[ ] Improve the way we cleanup connections and goroutines when context is ended
[ ] Measure ping in keepalives (logging and reporting metrics of the turnaround time it takes to communicate with every node)
[ ] auth on connect and keepalive (TLS style challenge response)
[ ] exponential backoff for retries

electricmonk commented 5 years ago

@OdedWx please review