arriven / db1000n

MIT License
1.18k stars 208 forks source link

Things to optimize - performance #539

Closed deputinizer closed 2 years ago

deputinizer commented 2 years ago
[Spoiler] pprof CPU ![image](https://user-images.githubusercontent.com/100740281/167301028-f716191c-ba1b-4dbe-8498-2b133b79e9b1.png) ![image](https://user-images.githubusercontent.com/100740281/167301430-cf73f8ca-56b8-4a10-bb30-aaddbbd2fc45.png) ![image](https://user-images.githubusercontent.com/100740281/167301489-83bd7141-e065-49c1-b84b-5b730408e7bf.png) ![image](https://user-images.githubusercontent.com/100740281/167301568-aef33cb1-bc70-4d69-aeb6-443a1358ee73.png)
arriven commented 2 years ago

for getNextPacket: have you tried setting StaticPacket to true in the config? should work on tcp/udp jobs as well as they just forward that argument to packetgen

deputinizer commented 2 years ago

Well I'm using the default config. https://github.com/db1000n-coordinators/LoadTestConfig/blob/e3ed4702c7d19347668ac5842d058172b596c329/config.v0.7.json#L435

There's no StaticPacket there

arriven commented 2 years ago

I'm not sure you can optimize other two without defeating their purpose completely. The only other thing that could be done is porting that StaticPacket thing to http flood to enable sending more traffic when no variablility is needed

deputinizer commented 2 years ago

Well there should be something faster, maybe:

deputinizer commented 2 years ago

Because compiling config is a no-go? :yum: https://github.com/valyala/quicktemplate

arriven commented 2 years ago

pure function and arg could work, but it would make the config even more bloated and I'm not completely sure it would be much faster, especially compared to static version which parses templates only once when new config is received. Also current implementation is already not that bad - in go you have separate Parse and Execute functions for templates where Parse takes significantly more resources than Execute. I don't think it's the most effective implementation but it does its job quite well

arriven commented 2 years ago

There's no StaticPacket there

Yeah, updating all the admin processes takes some time

deputinizer commented 2 years ago

Nice, you added static :)

[Spoiler] Here's dynamic ![image](https://user-images.githubusercontent.com/100740281/167306967-29e88664-4584-4b51-94ae-da01db709980.png)
arriven commented 2 years ago

hmm, having thought about it more, with current usage it might be easier to switch the code to use static by default and dynamic would have to be specified explicitly

arriven commented 2 years ago

it won't affect memory usage but the CPU profile should get a lot better