Suggestion: GUI performance

[ekienmeier]

I started fiddling around with tribeca a few weeks ago and it works quite well so far, I'm very happy with how it works and the results I get. Big thanks for the efforts you put into this project!

However, I noticed that in certain scenarios the web UI can't keep up with the constant stream of multiple orders being opened and cancelled in a very short amount of time. This is especially true in combination with AK-47 and a bullet setting > 1. I constantly see the browser (Chrome, Firefox, Edge) going completely out of sync with the server, and I see up-to-date data only when the browser refreshes. Sometimes upon logging in the browser goes into a refresh loop and doesn't show anything useful at all.

Suggestion: Instead of sending all data immediately through the web socket to the client, perform an update in a configurable interval, say 200ms or 500ms. This would greatly reduce the performance impact on the browser. I am no node or angular developer, so I have no idea how complicated it would be to implement.

[ctubio]

Hello ekienmeier, (yUp im analpaper xD) i have planned to make the GUI async soon, not sure about the delay or the grouping but we will see :P

I didnt experienced the gui out of sync, but is true that the websocket is very heavy currently (in the last commits the bandwidth has been reduced by +50%; but is not async yet).

Thanks for your suggestion!

[ctubio]

since 9182857 the gui is async, please let me know in case you still have issues to consider to add a limit/delay/grouping (for me now works niiiice so i didnt limited it in any way yet).

[ekienmeier]

The changes you commited in the last few days dramatically improved the performance of the server and reduced the CPU usage of the node and mongodb processes (I'm using Docker installation). Right now, with my settings (Boomerang on Coinbase) it works quite well, stable, and fast.

However, the UI remains the big bottleneck. I let tribeca run for the last 2 days, and now I'm unable to load the main website. The "Positions", "Market Data and Quotes", and "Quoting Parameters" are loaded, but the site hangs completely when attempting to load the "Order List" and "Market Trades". And that's where after a while Chrome reloads the website, and Edge stops completely.

Chrome: CPU: 1 core running at 100% RAM: between 200MB and 800MB

Edge: CPU: 2 cores running at 80% RAM: between 150MB and 500MB

It seems the UI problem arises as soon as there are multiple tables to refresh with data.

If tribeca is running but disabled using an empty MongoDB, I see the "Market Data and Quotes" tables being refreshed very frequently with up-to-date market data.

Browsers (Chrome and Edge, doesn't matter) go wild as soon as I enable tribeca and it starts sending its own orders. If orders are canceled and reopened very fast in succession and market data changes quickly, CPU and memory usage goes up.

It's quite hard from a user perspective to pinpoint the cause of the problems, so if any more input, data, or settings are required, please let me know.

[ctubio]

Hello'¡ i didnt experienced this problem yet of extremely high frequency of data in market quotes and open orders grid, but will migrate this next weekend to a hopefully very low latency environment where i will be able to experience this.

Many many thanks for all the information provided in advance for the debug'¡!

[ctubio]

eeep! in branch gui_deferred now we have a gui data delay of 1 second (also grouping between intervals to save bandwidth/cpu),

the only problem is that this "1" is hardcoded, will add some option to configure it soon (i will keep in another branch until finnish the ui config of it [dont know if with a slider or a numeric input xD]).

this makes every display (the open orders, market data, positions, safeties, quote values.. etc) less than history of trades and history of market trades delayed and grouped by 1 second, both history grids are still realtime, and also the buttons of cancel/submit/save are realtime too.

no worry to much about the new branch, i will merge with master soon.

[ekienmeier]

The delayUI setting works perfectly. Very happy with this improvement!

[ctubio]

just saying, still pending 2 small updates (in orders grid when delayUI>0 [data must be a snapshot of only the curent open orders instead of a incremental group of open and closed orders], and when delayUI=0 [RAM usage and daynight theme remains unchanged forever]).

[ekienmeier]

Just as a small feedback - I had it running yesterday in the background for several hours (checked out minutes after your commit to master) and the webpage never ran in any issues at all. CPU usage is minimal, so is RAM consumption. Thank you very much! 🥇