How does latency vary with increasing number of members and messages/minute?

[holmesworcester]

For the HOPE talk, we want to answer the question "does it scale?"

One part of this is knowing how well messaging works as groups get bigger and as groups get busier.

The best way to answer this is to show graphs of how message latency varies with the number of members, and with the rate of messages.

So let's hold messages/minute steady (e.g. some group of users in steady conversation) and increase the number of silent-but-online members in the community.

Then let's hold the number of users steady (at 10, 30, 50, 100) and increase the number of messages/minute to see how latency changes. Here I expect it will be great until it hits some critical point of CPU or bandwidth limit.

If we think CPU is the limit, it might be best to run this test on multiple separate machines comparable to a typical laptop.

Then let's try a test that combines the two stressors, and hold messages per member steady, and increase the number of members.

I expect we'll see a dropoff once we pass the point where peers cannot connect directly. (And we might need to modify our backend so that it stops trying to connect to all peers after some threshold e.g. 10 or 20.)

Once we get these graphs we might want to try some experiments to bring latency down to some acceptable threshold for common scenarios (3-5 seconds?) if we think there are things we can do that wouldn't be too much work.

We also might want to create issues for IPFS or OrbitDB based on this data.

[siepra]

Splitting up:

1. Make spambot control the intensity (message/minute) and number of users in time
2. Research graph tools (prepare json template for presented data)
3. (blocked) Prepare utils for collecting analytics data

[siepra]

What we measure: latency between sending message (client A) and receiving message (client B)

[siepra]

Test scenario 1: Hold messages/minute steady and increase the number of silent-but-online members in the community.

[siepra]

Test scenario 2: Hold the number of users steady (at 10, 30, 50, 100) and increase the number of messages/minute.

[siepra]

Test scenario 3: Hold messages per member steady and increase the number of members.

[holmesworcester]

@siepra this makes sense to me, but all we need to do is export a table and we can make the graph in google sheets or similar. We don't need to do that part with software.

[holmesworcester]

Also I see an issue created for 1 in this list but not for 2 or 3. We should create them, right?

[siepra]

all we need to do is export a table and we can make the graph in google sheets or similar. We don't need to do that part with software.

Yeah, that's how we see it also

[siepra]

Also I see an issue created for 1 in this list but not for 2 or 3. We should create them, right?

I created it because I just started working on it. Feel free to create the rest of them if you want

[holmesworcester]

• Make spambot control the intensity (message/minute) and number of users in time
• Research graph tools (prepare json template for presented data)
• (blocked) Prepare utils for collecting analytics data

Created issues for 2 and 3: #639 and #640.

siepra created #635