search

autonomys / substation

Polkadot Telemetry service
GNU General Public License v3.0
3 stars 2 forks source link

Add second local shard #40

Closed i1i1 closed 1 year ago

i1i1 commented 1 year ago

I discovered weird behavior with shard. Memory consumption goes up at some number of nodes, before that it is very low. I believe at some point a single shard can't handle some amount of data (as some components are not parallelized) and we just start to see increase of memory usage (maybe due to increase of some queues).

Let's try having 2 shards and see if it helps

i1i1 commented 1 year ago

Sorry, didn't submit description somehow. Updated now