arby is leaking memory

[kilrau]

These are two arby instances (simnet+testnet, latest commit), consuming 5/6GB RAM each, constantly growing over time.

Nothing special in the logs, arby just issuing orders:

2020-07-26 04:15:05.055 [OpenDEX] trace: Starting to update OpenDEX orders
2020-07-26 04:15:05.055 [OpenDEX] trace: Removed all open orders for ETH/BTC
2020-07-26 04:15:05.055 [OpenDEX] trace: Creating ETH/BTC buy order with id ec69a0b1-32e7-48a1-abde-aa1b8fe1c26a, quantity 3.09738824 and price 0.03045994
2020-07-26 04:15:05.055 [OpenDEX] trace: Creating ETH/BTC sell order with id 300e7fa8-f48d-42bb-bd42-f67b6eda8f0e, quantity 0.1129079 and price 0.03234406
2020-07-26 04:15:05.055 [OpenDEX] trace: Order 300e7fa8-f48d-42bb-bd42-f67b6eda8f0e created
2020-07-26 04:15:05.055 [OpenDEX] trace: Order ec69a0b1-32e7-48a1-abde-aa1b8fe1c26a created
2020-07-26 04:15:05.055 [OpenDEX] trace: OpenDEX orders updated

[erkarl]

Lowering the priority of this because as a workaround we're restarting arby every 6 hours: https://github.com/ExchangeUnion/xud-docker/pull/637 https://github.com/ExchangeUnion/market-maker-tools/pull/62

Also setting the NODE_ENV=production should reduce the memory usage.

[kilrau]

2GB Ram on my Pi. Guess we should bump prio again?

[kilrau]

Bumping this back to P1 since this randomly pushes things to the slow swap file and today I had e.g. the status command timing out and I think it's because it was forced onto the slow swap file.

Todo:

• find memory leak and fix it, let run for 24+ to confirm it's fixed
• remove regular arby restarts from xud-docker

[erkarl]

I've reduced the restart interval to 1 hour which should reduce the memory usage.

Putting a bounty of 0.015 BTC on this issue to find a more long term solution.

[kilrau]

Any updates here?

[erkarl]

Any updates here?

Haven't dedicated time to this as of now.