Add PE support for x86_64

[afranchuk]

It also expands UnwindRegs to allow all gp registers and adds caching for common PE unwind cases.

Related: #3.

[afranchuk]

@ishitatsuyuki do you mind making a new bug to cover the improvements you mention?

[mstange]

@ishitatsuyuki Thank you so much for chiming in! I was going to do the samply changes and ask you to test them, but if you're already working on them, that's even better! Actually, is there an easy way for me to test the PE-on-Linux scenario? E.g. are there any cheap Proton-ified Steam games that ship with enough symbols?

[ishitatsuyuki]

Most Unity game is OK as Unity provides a symbol server. Those with an enterprise contract that allows custom build from source will not work, though. You will also need https://github.com/mstange/samply/commit/3f01aafabca619a568eb047f098b64e97d59096b as both PDB debug name resolution and local PDB lookup when absolute path is coded into PDBFilename is broken right now.

Aimlab seems to be both free and use a public Unity build.

For profiling you can just build the forks and attach to the game process. Following https://gist.github.com/ishitatsuyuki/09eef31e16e5247718063cb2390cdc37#use-nsenter-to-change-samplys-mount-namespace will additionally make Unix libraries unwind correctly.

The old branches are at https://github.com/ishitatsuyuki/framehop/tree/seh-poc and https://github.com/ishitatsuyuki/samply/tree/poc-old. The new branches are https://github.com/ishitatsuyuki/framehop/tree/poc-new and https://github.com/ishitatsuyuki/samply/tree/poc-new. Compared to the old branch, the new branch still seems to fail to unwind around 1% of the samples fully, even after the chained unwind info fix. I'm still working on identifying why.

[ishitatsuyuki]

I open #21, #22, #23 to continue the discussions.

[ishitatsuyuki]

The remaining unwinding instability seems to come from epilogue analysis. For example, both of the following in-function jumps are identified as end of function when they are not:

I thought I implemented epilog analysis in my fork at some point too, but looks like the branch I currently use have no epilog analysis and always use SEH unwind rules. The stacks are reasonably correct there, as expected.

There are a few thing I plan to work on to improve the accuracy:

• Limit instruction analysis to top frame only (#22).
• Stricter parsing rules based on .NET's unwinder (I think this is probably reference quality).
• The same .NET unwinder contains logic to identify inter-function branches from intra-function branches, although it's fairly complicated as it tracks across chained frame infos to identify canonical entries.
• Allow falling back to plain SEH unwind rule when .text section is not available.
• Ability to specify multiple executable sections for some edge cases (#24).