proposal: debug/gosym: expose function start line and inline tree

[brancz]

Expose function start line through an API in the debug/gosym package.

The .gopclntab section includes data for the starting line number of a function, however, the debug/gosym package does not export an API that allows reading this data. I propose adding it to the API surface.

Use case: I happen to work on the Parca open-source continuous-profiling project and we quite frequently run into Go binaries, that don't include DWARF debuginfos (which we discourage but it's out of our control) but of course keep the .gopclntab, so we would like to be able to support those binaries as best as possible. The function starting line number in particular is important for the new PGO features, as noted in the documentation appendix.

[seankhliao]

What's the exact api that would be added for this?

[brancz]

Adding a StartLine field to the Func struct, and load it around here in the go12Funcs() function by adding a new function to funcData struct, called startLine, which like cuOffset would load the startLine (just field 9 instead of field 8). Doing all of this would make it be loaded when instantiating the table in NewTable.

This would only work for binaries that are Go >=1.3 as far as I understand, which would be fine for my case, since I'm targeting >=1.20 binaries in my use case.

[ianlancetaylor]

CC @golang/runtime

[prattmic]

For PGO, profiles need to contain inlined frames as well. Thus, I imagine that you want to add access to the inline tree to debug/gosym as well?

(This came up recently in another context that wants inlining information and we were considering a proposal to add it to debug/gosym, or perhaps an x/debug package).

[brancz]

I was going to open a separate proposal for that, but yes!

[cherrymui]

This would only work for binaries that are Go >=1.3 as far as I understand, which would be fine for my case, since I'm targeting >=1.20 binaries in my use case.

The start line in the pclntab is added last cycle (Go 1.20, specifically for PGO). So this would only work with Go 1.20+ binaries. For older binaries maybe fall back to 0?

[brancz]

The start line in the pclntab is added last cycle (Go 1.20, specifically for PGO). So this would only work with Go 1.20+ binaries. For older binaries maybe fall back to 0?

I think that would make sense.

[gopherbot]

Change https://go.dev/cl/473455 mentions this issue: debug/gosym: read start line of a function from gopclntab

[prattmic]

@aclements, @mknyszek, and I are thinking of tying this in with https://github.com/golang/go/issues/57447#issuecomment-1439394218. i.e., instead of continuing to extend debug/gosym's strained API (particularly difficult w.r.t. inlined functions), new packages initially in x/exp/debug, and eventually in x/debug will provide access to a more complete set of data.

[brancz]

I’m ok with forking debug/gosym for the moment (that’s what I’m doing right now anyway), while the new package API is being worked, out if that’s the stance of the project. I just thought I’d attempt to upstream this since already had it.

[gopherbot]

Change https://go.dev/cl/474541 mentions this issue: debug: add module

[gopherbot]

Change https://go.dev/cl/474543 mentions this issue: debug/gobinary: add initial package sketch

[gopherbot]

Change https://go.dev/cl/474542 mentions this issue: debug/gobinary: add package

[prattmic]

Just as a point of reference, Go 1.21 adds start line to DWARF (#57308), so this data is now available, just not from debug/gosym.

(We should still expose the gosym version. I'm just leaving this breadcrumb in case anyone is blocked on this.)

[prattmic]

x/vuln has a forked version of debug/gosym, which it uses to extract data from the inline tree: https://go.googlesource.com/vuln/+/refs/heads/master/internal/vulncheck/internal/buildinfo/additions_scan.go#49

One pain point that came up there is that the references to funcdata (where the inline tree lines) in the pclntab section are stored as offsets from the go:func.* symbol. If the binary is stripped, we can't find the symbol and thus can't get the inline tree.

One possible resolution here is to change go:func.* from just a symbol to a full section (like .gopclntab), as section headers are generally not removed by stripping.

(cc @zpavlinovic)