cmd/cgo: arrange to pass unmodified Go source files to compiler, go/types

[mdempsky]

Currently cgo support is implemented with cmd/cgo as a separate tool that analyzes cgo-using Go code and transforms it into standard Go code. This proposal is to extract the analysis logic into a separate internal package that can be reused by cmd/compile and go/types directly without needing source rewrites. I.e., turn them into native cgo compilers, rather than needing a separate "cgofront" preprocessor step.

The expected benefits are:

1. Better error messages. E.g., currently cmd/compile emits error messages about calls to _Cfunc_foo instead of C.foo.
2. Better position information. Currently when using golang.org/x/tools/go/loader to analyze cgo-using source, the column location and byte offset information is wrong because it applies go/types to the post-transformation Go source files. Thanks to //line directives, we can at least identify the original line number, but it's troublesome/imprecise to write type-sensitive refactoring tools that rely on column information and want to support cgo-using packages.
3. Better desugaring of cgo calls. Currently cmd/cgo needs to rewrite expressions into valid Go code to be accepted by cmd/compile, which is sometimes subtle (e.g., #7757, #9557, #13635, #13930, #16591). By desugaring within the compiler itself, we have more flexibility about the sorts of fundamental operations we can use.

Potential downsides and counter-arguments:

1. cmd/cgo still needs to exist for gccgo, but cmd/cgo already has substantially different behavior for supporting cmd/compile vs gccgo.
2. Might complicate #15681 (scheduling cmd/cgo earlier), but I expect an alternative strategy is equally viable: schedule the C compilations later. Currently the only reason we need to schedule the C compilations before the Go compiler is to produce _cgo_imports.go. But this file only contains //go:cgo_import_dynamic and //go:cgo_dynamic_linker directives, which don't actually affect Go compilation; cmd/compile just writes them back out again so cmd/link can ultimately handle them.
3. Increases coupling between cmd/compile and cgo, but cmd/compile already is / needs to be cgo-aware, and cmd/go already has extensive special case code for cmd/cgo (unlike any other tool that generates Go source).

Alternative ideas that might achieve similar benefits while generalizing to tools other than cmd/cgo:

1. Add a //go:errorname directive so cmd/cgo can express that _Cfunc_foo should be referred to as "C.foo" in error messages. It's not obvious this would generalize fully though; e.g., cmd/yacc rewrites $1 to yyDollar[1], which would need something more powerful to express.
2. Generalize //line directives to something that records column/offset information too (similar to JavaScript Source Maps). Again, I'm not sure how useful in practice this would actually be for tools other than cmd/cgo.

/cc @ianlancetaylor @alandonovan @griesemer

[wildoranges]

Sorry for any disruption caused by revisiting this proposal.

The proposal in #70274 raises concerns related to cgoCheckPointer. The desired solution involves integrating cgo into the compiler, which aligns with the general idea of this proposal. However, a notable difference is that this proposal does not seem to address cgoCheckPointer or provide a plan for resolving the associated issues. For reference, #70274 suggests leveraging compiler pass like type inference and pointer analysis to address these concerns.

I’m leaving this comment as a marker to ensure that, as this proposal progresses, the concerns raised in #70274, particularly those regarding cgoCheckPointer, are taken into consideration and resolved.