search

swift-server / guides

Guides for building, debugging and deploying Swift Server applications
257 stars 21 forks source link

Linux perf & allocation flame graph guides #35

Closed weissi closed 3 years ago

weissi commented 3 years ago

Very good idea 👍

Nice intro, just about enough to get folks started I think

Thanks! I just discovered that "Flame Charts" (ordered by time) are actually much better than "Flame Graphs" (stacks sorted) for events that we don't need to sample.

For CPU, I think Flame Graphs are normally better because the events trigger at random points so we need the aggregation for them to make sense. For allocations however, we don't need to sample. We can get all events in the correct order.

Slight problem is that stackcollapse-perf.pl from the flame graphs repo has a small bug and always pre-aggregates the stacks so flame charts (flamegraph.pl --flamechart) don't actually work. But I've got a patch ready for it.

That way you can get cool graphs like these:

IMG_0349 IMG_0348

See how you can spot every single allocation (time is not on the X axis) of when AHC takes a connection out of the pool and puts it back? We can see that it wastes lots of allocations for (dubious reasons) modifying the ChannelPipeline each time it takes a connection out of the pool or puts it back.

tomerd commented 3 years ago

this is great. should we replace some of the content in https://github.com/swift-server/guides/blob/main/docs/performance.md with links to this?

weissi commented 3 years ago

thanks a lot @kmahar & @glbrntt for your suggestions. Applied them all, will work through the other suggestions which are also great!

weissi commented 3 years ago

this is great. should we replace some of the content in https://github.com/swift-server/guides/blob/main/docs/performance.md with links to this?

I did check and there's pretty much no duplication. I did however link the "getting perf to work" guide from performance.md.