aloknnikhil
commented
4 years ago Even though the intention was to set a rate that could publish messages as fast as possible, the implementation did not do so. We ran repeated experiments and the produce rates were ranging from 2 to 50,000 messages/s (1KB messages). The algorithm, used to calculate this rate, relied on the consumer backlog to determine the rate at which there was no consumer backlog. However, the algorithm did not factor in any warmup times for the 3 systems and based its rate on only 1 minute of the actual running time during which these decisions were made. We did not want to modify the algorithm any further and instead relied on the system metrics (disk/CPU) to manually determine the maximum possible rate for each system.
Please feel free to contribute any changes you think might help fix the algorithm.
The rate at which producers produce messages (per second). Note: a value of 0 means that messages are produced as quickly as possible, with no rate limiting. Is there any problem as described in the official openmessaging documentation? Why was situation with no rate limiting cancelled? If you are free, I hope you can help answer my question, thank you very much.