Receive seems to be able to handle around 9,000 small web socket messages a second. Increasing that number slightly slows down the producer. Increasing that number by a large amount causes Receive to crash / roll / disconnect.
AC
[ ] Receive should not disconnect / roll when given more messages than it can process. (Either slow down or drop messages, need to discuss)
[ ] Receive should handle ?x? messages a second when using websockets
[ ] Slowing down Receive's clients / producers may not be an option. In that case we may want to drop messages instead of forcing the client to slow down. (Discuss and convert into AC statement)
Tech Notes
These tests were done with very small messages. Larger messages significantly decrease performance / exacerbate these issues. Given a dataset that had around 50 fields slowed receive down to about 400 messages a second. We expect that nested messages would see the same results as we believe performance is directly tied to number of bytes.
Data
Messages/second
Messages Sent
Messages Received
Messages Dropped
Percent Dropped
Drift of last message in seconds
Time Diff between first and last message on kafka topic (seconds) [expected without lag is 1000]
Messages / Second
Notes
1,000
1,000,000
1,000,000
0
0%
0.6
1018
982
10,000
10,000,000
10,000,000
0
0%
1.3
1104
9,058
Run with TCP no delay
10,000
10,000,000
10,000,000
0
0%
0.1
1183
8,453
Run without TCP no delay
20,000
20,000,000
NA
#VALUE!
#VALUE!
Disconnected before finishing payload
15,000
15,000,000
NA
#VALUE!
#VALUE!
Disconnected before finishing payload
12,500
12,500,000
10,008,243
2,491,757
20%
1105
9,057
Significant messages dropped. Run with TCP delay on
Receive seems to be able to handle around 9,000 small web socket messages a second. Increasing that number slightly slows down the producer. Increasing that number by a large amount causes Receive to crash / roll / disconnect.
AC
Tech Notes
Data
More Notes here