search

mng12689 / stack-viz

0 stars 0 forks source link

performance redesign #15

Open mng12689 opened 6 years ago

mng12689 commented 6 years ago

at 100 req/s, the UI is slow even if React doesn't have to reconcile the virtual DOM with the real DOM since the js event loop is essentially hogged by calculation. could we redesign the system such that the proxy server is the one doing all the calculation?

options:

  1. Browser client sends a websocket message with the renderMode its in, proxy sends messages relevant to UI changes based on mode

    • proxy receives incoming messages and performs calculations. it only sends messages to the client when information relevant to UI updates occur. for example, if renderMode == threshold, send metric updates that exceed new thresholds. if renderMode == value, change ??

  2. Proxy acts as buffer

    • proxy performs calculations and stores in memory.
    • proxy pushes a batch of calculations on some "near real time" scheduled interval. if calculations spawn threads, threads can update the buffer on the main thread opportunistically (worst case, the calc gets there next interval)
    • maybe browser client can apply backpressure if its overwhelmed?

  3. use in memory cache as buffer and allow proxy to fetch from cache

mng12689 commented 6 years ago

work breakdown:

  1. proxy sends batch statistics/app receives and processes batch statistics
  2. proxy implements multiple source modes a. push to port over network protocol - store buffer in memory (this wont scale past a single box) b. pull from remote buffer cache or queue (redis, kafka)