search

VolantMQ / volantmq

High-Performance MQTT Server
Apache License 2.0
982 stars 169 forks source link

Optimize goroutine and memory usage #47

Open troian opened 7 years ago

troian commented 7 years ago

Currently active connection requires 4 running goroutines with 4K of stack per each plus incoming ring buffer of 8K. If scale to 1M active connections we get into 12G of memory. If client does send nothing and no publishes for subscribed topics it just waste of memory. Reasonable to optimize by packing into less goroutines and spin them only when events available.

  1. Receiving packets. If there is no data on socket it blocked on epoll until there is some data. On github.com/golang/go, there is the issue. Wait until it's completed

  2. Sending goroutine spin only when there is messages to transmit otherwise it should be down

ddatsh commented 7 years ago

greate idea

funny-falcon commented 6 years ago

There are existed experience with using epoll explicitely for websocket server: https://medium.freecodecamp.org/million-websockets-and-go-cc58418460bb

troian commented 6 years ago

Was using it for a bit in the project. Still does not comply with all the requirements for example DeadLines does not work properly. Btw, did you get them working?