We are currently tracking sending metadata per ws connection: if we have not sent metadata for a context-path combination we will send metadata and add a marker that is formed from concatenating context & path so that for next value delta we don't send the metadata.
In a busy AIS environment this will cause a lot of string values to be retained for each WebSocket connection: 1000 ais targets x the number of paths per AIS target + self context & paths. The string values are nontrivial, consisting of context + path. The marker keys are formed on the fly, so even if the values are the same for each ws connection they are not shared.
Another problem here is that there is no mechanism for purging this data: the marker values for an AIS target are never cleared, even if the target goes away.
Fixes I can think of:
Share the marker keys: keep a separate record of canonical keys that are shared across ws connections. This would eliminate the times ws connections part of the consumed memory
Purge the markers every once in a while, removing all markers or add last sent value delta tracking and remove inactive ones (or both)
Use hashes instead of full values. This may cause some metas to not be sent.
Rethink meta delta mechanism. Do we really need them per context, since they are the same for each path?
We are currently tracking sending metadata per ws connection: if we have not sent metadata for a context-path combination we will send metadata and add a marker that is formed from concatenating context & path so that for next value delta we don't send the metadata.
In a busy AIS environment this will cause a lot of string values to be retained for each WebSocket connection: 1000 ais targets x the number of paths per AIS target + self context & paths. The string values are nontrivial, consisting of context + path. The marker keys are formed on the fly, so even if the values are the same for each ws connection they are not shared.
Another problem here is that there is no mechanism for purging this data: the marker values for an AIS target are never cleared, even if the target goes away.
Fixes I can think of: