Go compiler and runtime meeting notes

[jeremyfaller]

Google's Go compiler and runtime team meets periodically (roughly weekly) to discuss ongoing development of the compiler and runtime. While not open to the public, there's been desire by the community to learn what the compiler and runtime team is working on. While we learn what is a good format, and what works and doesn't, we will start publishing meeting minutes here.

These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

This meta-issue records minutes of Google's Go compiler and runtime meetings as issue comments, so that they can be cross-linked easily with the relevant issues. This meta-issue is for minutes only; comments that are not meeting minutes will be deleted.

[jeremyfaller]

2021-01-19

These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

• We continued a discussion about increasing the bootstrap version of Go from 1.4 to 1.16 after the 1.17 release. We wanted to make sure it wasn't hard for people to build from scratch, but we would like to use features that have been added over time. Consensus was that this would be okay to submit as a formal proposal.
• We discussed Austin's benchmark unit proposal. Consensus was hopeful that it would be accepted.
• We discussed the weak maps proposal. A number of corner cases, gotchas, and subtleties were discussed. Consensus was that the discussion on the issue was up to date.
• After falling off the prior week's agenda, we began a discussion about releasing the compiler and runtime meeting notes. We all felt communication with the community was important, and we would start with an internal publication of meeting notes. No dates were set for a final decision.

[jeremyfaller]

2021-01-26

These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

• We continued discussing whether or not there were any concerns with publishing compiler and runtime meeting notes. No issues were raised, so it's likely I will go ahead and start publishing the notes.
• We discussed the redirecting crash output proposal. No major concerns were raised, but consensus was that a call would be best approach (against environment variables, etc).
• We discussed the pain merging the development branches (master -> dev.regabi -> dev.typeparams). The consensus was that without acceptance of the generics proposal, we need to keep living with the pain.
• We briefly discussed using the type checker in dev.typeparams as the default type checker. Using that type checker is blocked on either acceptance of generics proposal or a separate proposal to move to a new type checker.

[jeremyfaller]

2021-02-02 These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

• We continued our discussion of moving to the new default typechecker on dev.typeparams. Engineering effort continues at a break-neck pace, and we're hopeful we'll be able to have this early.
• We discussed the issue with spinning M's, and have seen large improvements for unique workloads within Google. Consensus was that this was an excellent find.
• We had a brief discussion of integer constant resolution.
• We briefly discussed merging the dev.regabi branch back to master when the tree opens. Consensus was that it would be safe to merge when the tree opened.
• We discussed some recent additions to dev.typeparams, that added support for generic functions and types. It's possible the backend work will happen on the branch's compiler soon.

[jeremyfaller]

2021-02-08

These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

• We briefly discussed the status of 1.16.
• We discussed the tree reopening, and landing of the dev.regabi branch, and the dev.typeparms branches on master. Current plans are to merge dev.regabi during the first week. If both proposals (1, 2) are accepted, dev.typeparams will also be merged during that week.
• We discussed the new pacer proposal, and its benefits.
• We also discussed the status of the register abi. While functionality is gated behind a flag, we don't want to go into code-freeze, enabling the ABI that late. We discussed the amount of soak time we think we will need.

[jeremyfaller]

2021-02-16

These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

• We had a small celebration about the acceptance of the Generics Proposal.
• We discussed merging the dev.typeparams branch to master. There are public API typechecking changes on that branch that aren't stable. As a result, we will likely hide the APIs using buildtags.
• We continued a long-running discussion on making types2 the default typechecker. We still don't need to make a decision here, so back into the oven it goes.
• We discussed revising the syntax for typelists in generics. An internally floated proposal is similar to public proposals. More to come.
• We discussed adding attendance to entries in this document. As no one objected to having their name added, I will start doing that. I won't start by CC-ing people, just linking to their github profile for now.

Attendees: Carlos Amedee David Chase Austin Clements Matthew Dempsky Jeremy Faller Robert Griesemer Than McIntosh Martin Möhrmann Patrik Nyblom Michael Pratt Keith Randall Dan Scales Ian Lance Taylor Cherry Zhang

[jeremyfaller]

2021-02-23

These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

• We celebrated the reopening of tree for 1.17 development.
• We discussed some of the features we want to land for 1.17, including the register ABI, foundational work for generics, the new gc pacer, improvements to the scheduler, escape analysis improvements, and possibly some phase changes to the compiler. We also had a more in-depth discussion of subtle points in types2.
• We discussed the work on register ABI. A hello world is working (in review, not fully committed). More work for return values has also started, and we're having some success there.
• A new ABI with G in R14 and a zeroed X15 has shown reasonable promise to offer good runtime performance improvements. Work is ongoing to confirm results, and clean up some poorly understood overhead in wrappers. Also, not all of the places where we use G in assembly have been converted over. More data to come.
• We discussed an issue that turning on stack poisoning seems to bork the openbsd-amd64-68 trybot.
• We discussed a proposal to add zeroing for cryptographic functions. Consensus was that this is non-trivial.

Attendees: Carlos Amedee David Chase Austin Clements Matthew Dempsky Jeremy Faller Robert Griesemer Than McIntosh Martin Möhrmann Patrik Nyblom Michael Pratt Keith Randall Dan Scales Ian Lance Taylor Cherry Zhang

[jeremyfaller]

2021-03-02

These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

• We discussed the current status of our 1.17 and 1.18 goals, including the register ABI, and generics.
• We discussed a small issue on sum types. Likely more external communication to come.
• We discussed how best to handle the meeting notes (this issue), and questions and feedback. We decided it would be best to move queries to the mailing list(s) for now.
• We discussed moving GOEXPERIMENT, and how important it would be to help the registerABI work to land.
• We discussed a subtle issue in the register ABI to simplify the internal representation and requirements for go & defer. Specifically, having the compiler rewrite defer f(x) ⇒ x' := x; defer func() { f(x') }().

Attendees: David Chase Austin Clements Matthew Dempsky Jeremy Faller Robert Griesemer Than McIntosh Patrik Nyblom Michael Pratt Keith Randall Dan Scales Cherry Zhang

[jeremyfaller]

2021-03-09 These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

• Again, we discussed the status of our 1.17 and 1.18 goals.
• We hope that move GOEXPERIMENT knob is accepted, as the current plan with the register ABI is to use it for turnup.
• We discussed the progress on type lists. An internal proposal likely addresses all concerns, and an external document will be available.

Attendees: David Chase Austin Clements Matthew Dempsky Jeremy Faller Robert Griesemer Than McIntosh Martin Möhrmann Patrik Nyblom Michael Pratt Keith Randall Dan Scales Ian Lance Taylor Cherry Zhang

[jeremyfaller]

2021-03-16

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• We discussed typelists again.
• We discussed possibly updating MIPS to MIPS32r2. The thinking is that R2's been around for a long time (10years?), and there is significant performance to be gained in the crypto libraries if we adopt it. Previous attempts have failed because of Loongson users. Discussion will likely continue.
• We discussed how to address amd64 per-architecture optimizations. Haswell specific optimizations have shown large improvements in certain workloads, and it might be worth providing for these wins. Discussion will likely continue.

Attendees: The notetaker would like to apologize for leaving off Michael Knyszek from previous attendance. I don't have a good system yet, and it's quite manual. David Chase Austin Clements Matthew Dempsky Jeremy Faller Robert Griesemer Michael Knyszek Than McIntosh Martin Möhrmann Patrik Nyblom Michael Pratt Keith Randall Dan Scales Ian Lance Taylor Cherry Zhang

[jeremyfaller]

2021-03-23

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• Most of the meeting was a discussion about 1.18.
• We discussed changes to the standard library for generics in 1.18
• We discussed followon work for the register ABI (beyond amd64).
• We discussed measuring PGO and other GC (Immix) ideas.

Attendees: I was absent from the meeting, and unable to grab the screenshot.

[jeremyfaller]

2021-03-30

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• We discussed the remaining outstanding work, now that we're 1 month from the code freeze.
• Windows/ARM: completed, status of cgo is unknown
• RegisterABI: most of the rest of the changes are in review, will be able to start debugging in earnest.
• GC Pacer: Not complicated, but currently unscheduled. Register ABI is taking precedence, and if we get a chance, it'll get in.
• Scheduler Improvements: Most CLs are ready for review, need to do more testing for metrics.
• Improving Escape Analysis: Most CLs are ready for review
• Haswell Improvements: Needs a proposal first.
• Compiler phase changes: (interleaving front-end and SSA) mostly delayed.
• We discussed using GOEXPERIMENT as a one true feature gating system. Register ABI work has found this to be helpful

Attendees: Lost to the sands of time. I forgot to take a screenshot.

[jeremyfaller]

2021-04-06

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• We discussed turning up the register abi. Our current plan is to turn the five flags that control it one-by-one, and watching the builders/trybots. The date for this turn up is TBD.
• We discussed branching again (during the freeze) for register abi and generics. Our current plan is to use a single branch for both.
• We discussed how debug information for generics was going to work. We talked about how other compilers (gcc, swift) do it. Some issues might be brought up in a future meeting with the delve folks.

Attendees: David Chase Austin Clements Matthew Dempsky Jeremy Faller Robert Griesemer Michael Knyszek Patrik Nyblom Michael Pratt Keith Randall Dmitri Shuralyov Ian Lance Taylor Cherry Zhang

[jeremyfaller]

2021-04-20

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• We discussed regabi which has been completely enabled in master. Please let us know if there are any instabilities. Performance (while still changing) is showing ≥ 5% wins, and smaller binaries by 2%, with 10% smaller .TEXT sections.
• We discussed the status of types2. It's not in yet, but we intend to get it in before release.

Attendees: David Chase Matthew Dempsky Jeremy Faller Robert Griesemer Michael Knyszek Than McIntosh Martin Möhrmann Patrik Nyblom Michael Pratt Keith Randall Dan Scales Ian Lance Taylor Cherry Zhang

[jeremyfaller]

2021-05-11 These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

I'm a little behind updating these notes. I'll consolidate the last couple of meetings into this one.

• The freeze is on. Generics and the register ABI work will continue on a the dev.typeparams branch.
• The register ABI is holding true on the ~5% performance improvement, and smaller binaries. There are some regressions in some benchmarks, likely related to live variables over loops with register pressure, but we might not feel comfortable fixing these for 1.17.
• On May 11, we discussed the Uber blog post. We think it's being adequately discussed on the proposal.
• On May 4, we discussed Ian's slices proposal.
• We continue to track the 1.17 release milestone. So far, it looks like we're on track for the 1.17 beta.

[changkun]

Ping. Is the meeting stopped since May? :)

[thanm]

2021-11-30 These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

Resuming notes after a fairly long hiatus.

• 1.18 release freeze continues; C&R team has been working mainly on bug fixing and preparing for a beta "1" release predicted to come out some time next week.
• Generics status:
  • for the front end, Robert reports that existing bugs break down mainly into two categories: problems with type inference (including some that cause infinite recursion), and problems with error reporting (error message confusing or not quite as clear as it could be). None of these should block the first beta.
  • back end is in good shape, delve / debugging works well, no known issues.
  • documentation: a generics tutorial is in the works and being reviewed
• Notable bugs: the team has been focusing on a collection of memory corruption bugs (#49209, #49259, #49453, #49689, #49686 ). Key recent findings:
  • these bugs are 100% correlated with NetBSD/OpenBSD on AMD processors (Intel processors don't seem to be impacted, ditto linux)
  • the bugs happen both on VMs (GCP, AWS) and on bare metal
  • the bugs are reproducible going back several Go releases (we've seen crashes on Go 1.10 and Go 1.12 as part of the bootstrap)
  • we are leaning towards not keeping these as release blockers given the above
• Notable bugs part 2: there is a new issue (#49138) with the race detector issue on macOS 12.
  • Cherry reports: the race detector SIGABRTs on MacOS 12. MacOS has a new malloc (“nano malloc”); setting env var MallocNanoZone=0 disables it. We are hearing from apple that the new malloc maps at a specific address, and TSAN also maps something fixed there. Apple probably won’t change their malloc. Cherry working on a fix.

Attendees: Jeremy Faller Austin Clements Robert Griesemer Michael Knyszek Than McIntosh Martin Möhrmann Patrik Nyblom Michael Pratt Keith Randall Dan Scales Cherry Mui

[thanm]

2021-12-07

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

[Resuming after yet another hiatus, hope to put these out on a more regular timetable at some point].

• 1.18 release freeze continues
• C&R team has been working mainly on bug fixing
• Discussion of bugs as blocking beta release of 1.18
  • runtime/pprof: new test TestLabelSystemstack is flaky Fix in review
  • all: rewrite interface{} to any Waiting on one x/tools vet CL (and vendoring)
  • doc: write Go 1.18 release notes misc discussion.
• discussion of the 1.18 GC pacer changes and their impact on builder instability
  • structured builder output would help
  • should we consider changing things so that we never stop on the first error when doing builder all.bash runs?

[Attendees list missing, apologies from note-taker].

[thanm]

2022-01-05

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• 1.18 release freeze continues
• C&R team has been working mainly on bug fixing
• 🎉 Go 1.18 beta 1 released (2021-12-14)
• FreeBSD memory corruption problems
  • folks from the release team are now updating the builders to incorporate the kernel fix
  • bugs expected to be fixed by this include #46272, #49967, #49752
• Notable bugs and bugs discussed in detail

  • 50427

  • 50321

  • 50276

  • 50177

  • 49912

  • 46794

  • 48656

  • 48619

  • 43056

  • 50097

  • 50084

  • 49959

  • 50391

  • 50321

  • 50190

  • 50417

[Attendees list missing, apologies from note-taker].

[thanm]

2022-01-18

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• 1.18 release freeze continues
• C&R team has been working mainly on bug fixing
• 🎉 FreeBSD corruption issues are now behind us (kernel bug resolved). Much rejoicing, this was a very painful problem to track down.
• Discussion of notable bugs
• Discussion of areas where we are looking ahead for post 1.18 work
  • Go vision, runtime vision
  • Core health
  • PGO
  • cmd/cover
  • Generics polishing
  • SetMemoryLimit
  • Performance monitoring

[Attendees list missing, apologies from note-taker].

[thanm]

2022-01-18

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• 1.18 release freeze continues
• C&R team has been working mainly on bug fixing
• Austin has circulated a spreadsheet with ideas on things we can do to improve "core project health", e.g. improve developer velocity, catch bugs earlier, reduce toil on the part of the release team. Team is reviewing and adding to it.
• Generics
  • Several bugs fixed in generics since the first beta
  • we recently had a user-visible change to export data
  • google internal testing doesn't specifically stress generics to the degree that it does with non-generic code
  • conclusion: it would help to have a second beta release
• Notable bugs

  • 49912 Robert is going to be looking into this. Marked as release blocker.

  • 50417 Dan has committed fix.

  • 50619 Just found; this may trigger a second beta

  • 50642 fix is in progress for this.

[Attendees list missing, apologies from note-taker].

[thanm]

2022-01-25

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• 1.18 release freeze continue
• C&R team has been working mainly on bug fixing
• Notable bugs
  • team members are working on triaging a couple of potential 1.18 bugs identfied as part of Google internal testing. More work needed to identify root cause.
  • release blocker #50685 fix is being tested.
• Generics status:
  • Several fixes have been checked in recently, another fix anticipated shortly for #50619.
  • expectation is that we'll start preparing a 2nd beta release shortly after that goes in

Attendees: Jeremy Faller Ian Lance Taylor Austin Clements Robert Griesemer Michael Knyszek Than McIntosh Martin Möhrmann Michael Pratt Keith Randall Dan Scales Cherry Mui

[thanm]

2022-02-01

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• 1.18 release freeze continues
• second 1.18 beta release went out earlier this week (yay)
• C&R team has been working mainly on bug fixing
• Notable bugs:
  • discussion of #35006 (updating C compiler on windows builders)
  • issue #50936 (SIGSEGV in gentraceback during SIGPROF handling); this issue (now fixed) came up during Google-internal testing
  • issue #50113 (deadlocks during syscall.AllThreadsSyscall); fix being tested CL 381534
  • we have four remaining release blockers, #45867, #49912, #50427, and #50887
• team members are brainstorming to identify + prioritize ideas to improve "core team health" (developer productivity)

Attendees: Jeremy Faller Ian Lance Taylor Austin Clements Robert Griesemer Michael Knyszek Than McIntosh Martin Möhrmann Michael Pratt Keith Randall Dan Scales Cherry Mui Alex Rakoczy Matthew Dempsky

[thanm]

2022-02-15

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• 1.18 release freeze continues
• C&R team has been working mainly on bug fixing
• Bug highlights:
  • no C&R release-blocking bugs at the moment, which is good

  • 50113 has now been fixed (thank you Michael Pratt); this was a tricky bug

  • discussion of #49679 (currently no way to reproduce)
  • discussion of #45867 (may want to change the test here)
  • discussion of #48505 Consensus seems to be towards adding GOAMD64=V1 and GOAMD64=V3 builders at some point.
• discussion of out-of-tree ports (arch and os)

Attendees: Austin Clements Cherry Mui Dan Scales David Chase Dmitri Shuraylov Ian Lance Taylor Jeremy Faller Keith Randall Matthew Dempsky Michael Knyszek Michael Pratt Than McIntosh

[thanm]

No meeting this week (Feb 22, 2022).

[thanm]

2022-03-01

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• 1.18 release branch cut, 1.19 tree open for development
• Discussion of issues relating to enabling of unified IR
  • How many changes do we anticipate would have to be made on both the non-unified-IR and unified-IR paths? How long do we keep both working after enabling unified IR?
  • consensus seems to be that we have to maintain both code paths until we commit to unified IR.
  • we’ve basically been doing this for months now. mdempsky@ isn’t too concerned.
  • discussion of testing strategy (we need both yes- and no-unified builders)
  • austin@: can we break unified IR into multiple sub-experiments like we did regabi?
  • considering making extra dictionary stuff a separate flag/experiment.
    • mdempsky@: Working on dictionaries right now (currently all stenciled), but want to get that working before flipping the switch.
  • How long do we think we need to soak?
    • mpdempsky@: most likely enable next week
• Discussion of tentative major Go 1.19 C&R items/deliverables
  • Switch to unified IR
  • Generics bugs
  • Generics tech debt
  • Compile speed (#49569)
  • Type checker error messages
  • khr@: Jump tables
  • SetMemoryLimit (#48409)
  • drchase@: Revisit initial heap size?
    • mknyszek@: enabling currently not in scope, but can be turned on via a GOEXEPRIMENT nob
  • cmd/cover overhaul
    • thanm@: about to put the proposal into the proposal process. Working on implementation. Getting good results.
  • Proposals: arenas (CL), runtime.Pinner (CL), key erasure, extra runtime/metrics, turn off idle mark workers for periodic GC, external time
  • khr@: Larger starting stack size
• Discussion of (#51410)
  • recently caused a performance problem for a google-internal application

Attendees: Alex Rakoczy Austin Clements Cherry Mui Dan Scales David Chase Dmitri Shuraylov Eli Bendersky Ian Lance Taylor Jeremy Faller Keith Randall Matthew Dempsky Michael Knyszek Michael Pratt Robert Griesemer Than McIntosh

[thanm]

2022-03-01

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• Discussion about triaging of builder problems
  • what sort of automation can we develop to make overall process easier?
  • often when a bug is filed for a specific builder test failure, someone will check a change to the test to skip the specific scenario (e.g. GOOS/GOARCH) until the bug is fixed.
  • what are we doing to ensure that these skips aren't forgotten about and left in forever?
  • ... ideally we could encode these skips in a way that we can see if they stop failing.
• impromptu GitHub project demo
• more discussion of #51410
  • not the only known performance cliff
  • austin points out that our map is pretty old; at what point do we rewrite the map using the latest and greatest techniques?
  • gri@: now that we have generics, it may make sense to do this in userland.
  • iant@: different people have different requirements for maps. It may make sense to have a range of map implementations that aren’t in the runtime at all.
• cmd/cover rewrite proposal has been released to public process
  • robpike@ still argues for a source transform
  • mdempsky@: The original userland fuzzing was a source transform and the compiler-native support is way less intrusive and much easier to maintain.
• Status
  • unified IR
    • mdempsky@: still working on dictionary stuff. Importer was blocked, now unblocked. Finding some bugs and working on them. Still on track to land in next week or two.

Attendees: Alex Rakoczy Austin Clements Cherry Mui David Chase Dmitri Shuraylov Eli Bendersky Ian Lance Taylor Matthew Dempsky Michael Knyszek Michael Pratt Robert Griesemer Than McIntosh

[thanm]

2022-03-15

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• Go 1.18 just went out!🎉🎉🎉 (much cheering)
• The compiler now requires the -p flag (CL 391014)
  • The code paths to support a missing -p flag still need to be cleaned up.
  • plan is to eventually eliminate the "". convention from the linker if possible
• Status
  • Generics
    • [Robert]: given that we have only 1.5 months for development in this cycle, not enough time for huge projects (parameterized aliases, big compile time improvements)
    • focus will most likely be mainly on bugs
    • [Keith] working on some generics performance CLs
  • Unified IR
    • [Matthew]: planning on landing soon (now that 1.18 release is out)
    • held up by annoying dead code issue
    • continuing to work on shape types in unified IR
  • SetMemoryLimit
    • [MichaelK]: big stack of CLs in progress.
    • hope to land by the end of March
  • cmd/cover
    • [ThanM]: Based on proposal feedback, now looking at a hybrid design that combines source-to-source rewriting with compiler support. Solves tricky problems with source position information in the compiler and some tricky problems with source rewriting.
    • rollout: still optimistic it can be landed in the next month and a half. austin@: Worth identifying descope options.

Attendees: Alex Rakoczy Austin Clements Cherry Mui David Chase Eli Bendersky Jeremy Faller Keith Randall Martin Möhrmann Matthew Dempsky Michael Knyszek Michael Pratt Robert Griesemer Than McIntosh

[thanm]

2022-03-22

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• ABI spec now has a shortlink: go.dev/s/regabi
• meta-remark: planning on moving the "center of gravity" for front-end discussions to this meeting
• status
  • Type checker
  • Doing cleanups. Looking into performance. Minor fixes. Thinking about generics support that we postponed (does that need proposals?)
  • Clearing out a lot of unnecessary code from non-types2-based checking. Taking advantage of required -p flag.
  • CL is out to not require -p, but make unlinkable objects. Idea: if it’s package main, treat as -p main.
  • Generics
  • Slow trickle of user issues. Nothing major. (🎉)
  • Unified IR
  • Burning down tech debt to make this switch easier. Making dictionary code easier to reuse.
  • SetMemoryLimit
  • Heap goal is responding to the limit. Next step is the scavenger. Then testing everything. Some CLs are out for review.
  • cmd/cover
    • Focusing on cmd/go. Many script tests verify odd aspects of coverage. More or less convinced that the hybrid approach is the right way to go.
• Supporting out-of-tree OS ports
  • Examples: Ron Minnich’s repository, Fuchsia
  • What is the incentive for us to actually make this easier?
  • What support would we provide for out-of-tree ports?
  • [Robert]: Do we have builders for out-of-tree ports?
  • [Austin]: I think that’s one thing we most want to get out of.
  • [Keith]: We should kick out one or more of our least-maintained ports (we all know what they are) and see what it actually takes.
  • [Austin]: we could also do our own out-of-tree port and own it ourselves for a little while.
  • [MichaelP]: cleanups in the syscalls in the runtime package make make it easier (as a side effect) to support out-of-tree ports.
  • [Than]: If we do this, it would be helpful to have a way for an out-of-tree maintainer to have a way to get our blessing at some hash that passes all.bash. The blessing would be in the eye of the general public.
  • [Keith]: We could have a wiki page that says “if you’re interested in running on Dragonfly, go over here, we think this ports works.”

Attendees: Alex Rakoczy Austin Clements Cherry Mui Dmitri Shuraylov Eli Bendersky Jeremy Faller Keith Randall Martin Möhrmann Matthew Dempsky Michael Knyszek Michael Pratt Robert Griesemer Than McIntosh

[thanm]

2022-03-29

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• discussed plans for the typechecker in Go 1.19
• discussion of Russ's sync/atomic types CL.
  • this is an interesting CL since it represents the first real use of generic types in std
  • some potential/possible fallout from static analyzers and other tools that don't yet grok generics
• discussion of freeze timeline and team meeting dates
• Updating Windows builder C toolchain
  • [Than] plan is to wait until all the CLs are in, then sync with the release team to organize the actual builder update. CLs have to wait for proposal approval due to API change made in debug.pe
• discussion of timeline for updating bootstrap version of Go on builders
  • core team will own/prioritize this; may slip to next release
  • general feeling at this point is that it’s a lot more work to update the builders than to live with the status quo for another ~6 months.
• discussion of recent dvyukov@ CL to update linux_amd64 race syso (removes goroutine limit)
  • would be great to get this rolled out to all applicable builders for 1.19
  • [Than]: I want to make sure it gets built with the new Windows C compiler.
  • [Cherry]: we have a tool that automates the update via gomotes, once new compiler is in place we can just use that
• status
  • Front end
    • Just fixing bugs in type checker and parser, plus "-p" cleanup.
  • Unified IR
    • Added a go/types importer.
    • Other generics issues
    • Bugs have been easy to diagnose and solve.
  • SetMemoryLimit
    • Updating CLs from comments.
    • Stuck on not being able to ready a goroutine from inside the allocator. Working on it.
    • Targeting end of week for having all the CLs ready.
  • cmd/cover
    • Working on cmd/go. Unit tests are passing with both new and old coverage. A lot more unit tests to write.
    • After that, going to focus on reporting changes.
    • Committed to the hybrid approach at this point.

Attendees: Austin Clements Cherry Mui David Chase Dmitri Shuraylov Eli Bendersky Ian Lance Taylor Matthew Dempsky Michael Knyszek Michael Pratt Robert Griesemer Than McIntosh

[thanm]

2022-04-12

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• one month to go (roughly) before release freeze
• Updates
  • Front-end/generics
    • [Matt] compiler is now switched over to unified IR export format, which caused a bunch of problems with x/tools tests. For now, disabled trybot. Need a better solution for major export format changes.
    • generics issues: bugs are coming in, but the rate overall seems to be manageable at the moment. Several failures are in consistency changes with new code going through the old importer for things where those checks no longer make sense.
  • SetMemoryLimit
    • moving closer to land this feature
    • plan is to document limitations on small heaps (with plan to improve in the future), feature-flag big change but don’t worry about small corner-case differences
    • Michael P and Austin to review CLs
  • cmd/cover
    • Plan: drop old compiler-only support from CL stack, put hybrid CLs out for review. Prioritize matching current functionality, with new functionality for later.
    • What should “go build -cover” cover? “go test -cover pkgs…” covers just pkgs, but that doesn’t seem like the right default for go build. Cover the main module by default?
    • [Than] The argument for instrumenting everything is that you can filter it down at reporting time.
    • [Michael K]: if I were to use “go build -cover”, instrumenting everything is what I would expect.
    • [Ian] I suspect most people aren’t going to care about coverage of std. And probably “low level libraries” though we don’t know what that is.
    • [Than] defaulting to packages in the main module may be best.
    • [Kieth] one factor is run-time cost of coverage instrumentation.
    • [Than] benchmarking needed here. Definitely faster than -race, but not sure exactly how much.
  • discussion of https://github.com/golang/go/issues/52209 [cmd/compile, cmd/link: generate dwarf type info in compiler] and related stack of CLs
    • Than to review and keep an eye on progress.
    • Noted that this is a fairly large project, will probably take a while to get through all the details.

Attendees: Austin Clements Carlos Amedee Cherry Mui David Chase Dmitri Shuraylov Eli Bendersky Heschi Krienek Ian Lance Taylor Keith Randall Martin Möhrmann Matthew Dempsky Michael Knyszek Michael Pratt Than McIntosh

[thanm]

2022-05-10

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

[no meetings previous 2 weeks due to quiet week, internal team summit]

• Go 1.19 release freeze has started
• Call for discussion of outstanding CLs that need final reviews this week
  • [Michael K] Sent out a series. Fix for SetMemoryLimit, several CLs to add metrics, resolve an external proposal for more metrics. (Sent to austin@ and drchase@, but anyone can look.)
  • [Robert] rfindley@ has a bunch of type checker CLs to follow-up on a bug fix. (gri@ to review or adonovan@)
  • [Keith] Larger starting stack CL sequence (mknyszek@ still reviewing)
  • [Ian] Good time to check issues with 1.19 milestone and fix them or push them to 1.20 or backlog.
• please update the release notes!
• Generics plan for H2’2022? (1.20 and early planning for 1.21)
  • [Eli] we want to get a jump on the planning process for this work
  • candidates
    • performance work
    • possibly/potential interaction with PGO
    • unified IR
    • type inference improvements
    • features like internal types (may be low demand)
    • figure out what to do about x/exp packages that use generics (graduating things to std and additional proposals that haven’t even made it to exp)
    • [Robert] There are ~5 proposals for how the built-in type comparable should work. We have to resolve this sooner than later. (Problem is that interfaces are comparable but may panic and “comparable” only includes types that are comparable without panics.)
    • elimination of implementation restrictions we currently have. (And should we?)
    • [Robert] error message tuning
    • [Keith] compiler speed
  • [Matthew] are we going to create a 1.20 dev branch? Would like to land unified IR fixes so its ready.
  • [Austin] I’d be fine with a dev.unified.
• do we want to have an external monthly version of this meeting?
  • discussion about logistics (possibly something like existing tools meeting)
  • Lighterweight option: office hours with C&R team?

Attendees: Austin Clements Carlos Amedee Cherry Mui David Chase Dmitri Shuraylov Eli Bendersky Ian Lance Taylor Keith Randall Matthew Dempsky Michael Knyszek Robert Griesemer Than McIntosh

[randall77]

2022-06-21

(Sorry we've been slack about this in the past few weeks, it fell through the vacation cracks.)

• New performance dashbard https://perf.golang.org/dashboard/
  • Diff from most recent release
• Discussions about better release automation and a submit queue
  • [Robert] Would this affect how we run try bots?
  • [Michael K] We could be much more aggressive about culling the try bot set so it’s really fast. That would encourage running try bots more, and the SQ could catch everything else.
  • [Matthew] Can we mail reviewers only once trybots have passed?
• Generics updates
  • [Matthew] Big stack of CLs for adding rtype fields to IRs for runtime type information. For dictionaries in the unified IR world. Adding regress tests on master, marked as broken in non-unified mode.
  • [Robert] Mostly fixing bugs, clearing up spec. Biggest outstanding issue is comparable.
• Discussion of release blockers

  • 53378. Rhys has a proposed fix.

  • 53374

    • [Cherry] Range of possible solutions. 1.18 changed ARM64 prologue slightly: previously, small frames didn’t have a signal-unsafe window. We could go back to that (though this probably affects other LR arches).
    • [Iant] This implies we’re doing our prologues incorrectly.
    • [Cherry] We do it that way so we can always unwind. C avoids this problem because it has richer unwinder information that can say “we’ve decremented SP, but haven’t saved LR yet.” We could enrich our unwinder information with that extra bit. I’m hesitant to do that at this point for 1.19.
    • [Cherry] Another angle is from the runtime: immediately at thread entry from assembly, set up a signal stack. Involves tons of assembly code across several architectures.
    • [Austin] I’m inclined to revert prologue for small frames for 1.19 and do the better metadata for 1.20. That’s fragile, but it’s just a stop-gap.
    • [Cherry] That doesn’t help other LR arches, but maybe those issues are old anyway. We would have to manually inspect the frames up to setting the signal stack to make sure they’re small.
• External runtime meeting notes
  • [Keith] We haven’t released external notes for about a month.
  • [Austin] Than’s been out and I guess we don’t have any bench depth on this.

Attendees: Alex Rakoczy Austin Clements Cherry Mui David Chase Dmitri Shuraylov Eli Bendersky Ian Lance Taylor Keith Randall Matthew Dempsky Martin Möhrmann Michael Knyszek Michael Pratt Robert Griesemer Than McIntosh

[thanm]

2022-06-28

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• 1.19 release freeze continues
• we reviewed the list of release blocking bugs
• Michael K held the first C&R issue triage meeting last week
  • Michael P reports: We discussed process. Plan is to go through new issues (and maybe performance) to determine general priority and who should look at it next using the assignee field. MK is planning to get gopherbot to at least auto-add label and maybe add to the project and track which issues are not getting updated.
  • Generics updates/Unified IR
  • Matthew: Landed all CLs last week. A few more on implicit conversions and tricky cases around for-range. When that’s done, I’ll wire up stuff for creating dictionaries.
  • Robert: Minor updates on error messages.
  • Ian: I’ve updated the iterator proposals again. Added an optional Done method that will be called by range loops. Simplified MakePipe to runtime.Iterate(func(yield func)) that creates a goroutine to run the generator function and makes it very similar to Russ’ and Jonathan’s proposals (plus clever runtime scheduling). We can apply a compiler optimization so instead of a separate goroutine, we can do everything in-line.
    • Keith: My worry is there will be a big performance cliff depending on whether the compiler can figure that out.
    • Ian: I think we can make the goroutine thing pretty efficient, too, but that is a concern.
• note from hana: expect more improvement work in compiler/runtime-delve interaction than in the editor integration domain near future, wonders if someone from the C&R team might be willing to attend the dlv meetings. thanm@ will reach out.
• https://perf.golang.org/dashboard/ now has change detection
• no meeting next week

Attendees (partial): Austin Clements Cherry Mui David Chase Dmitri Shuraylov Ian Lance Taylor Keith Randall Matthew Dempsky Michael Pratt Robert Griesemer Than McIntosh

[thanm]

2022-07-12

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• Go 1.19 rc1 is out! 🎉
• Go 1.20 planned items
  • Core project health
    • Austin and Heschi to prioritize again and start assigning
  • PGO preview
    • MVP is compiler support for reading profiles with one or two simple optimizations. Considering what cmd/go support we have beyond that.
  • Unified IR
    • Coming along. David’s benchmarking shows 8% build time slow-down, but there’s been minimal optimization effort. Unified IR export format has debug markers for checking sync (moving that from static to run-time flag). Dictionary support is close.
  • Go generics planning for H2’2022 - still need to narrow down scope
  • cmd/cover
    • Slipped from landing at tree-open; bottle neck at the moment is getting the cmd/* CLs reviewed.
    • More work needed on reporting changes.
  • Arenas?
• Discussion of Go 1.19 release blocking issues
• Discussion of Go 1.19 non-release blocking issues
  • runtime: upgrade to v3 version of race detector #49761
    • [Than] still working on updating windows syso to V3
    • problem area seems to relate to differences in how TSAN shadow memory is handled between v2 and v3
    • also work needed on Go linker to support import symbol thunks
    • thunk support needed when using updated C toolchain, and V3 tsan requires updated toolchain, see #53540
    • Q: What happens if we don’t fix this for 1.19? A: we’ll be stuck with the v2 race detector on Windows and will continue to run into problems using Go with more up-to-date system toolchains.
  • There are a couple unified IR bugs. Should those just move to 1.20? sounds like answer is yes
  • runtime: modified timer results in extreme cpu load #51654
    • backport likely
    • [Michael P]: Probably just figured out what’s wrong.
  • global variable init bug https://github.com/golang/go/issues/51913
    • [Matthew]: May cause programs to be slightly larger or take slightly longer to init. Confident in fix correctness.
    • Q: Would we do something different in 1.20?
    • [Matthew]: Would probably land the same fix. We might be able to improve the optimization.
    • [Cherry]: Do people depend on this buggy behavior?
• New GC guide has rolled out: https://go.dev/doc/gc-guide

[thanm]

2022-07-19

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• Go team is participating in a "friction fixit week"
• Friction fixit week
• Go 1.19
  • No C&R release blocking bugs at the moment
  • Non-release blockers
    • Matthew: We should stop backporting generics fixes to 1.18. Maybe keep doing backports to 1.19.
    • We think this is okay per policy because these things never worked.
    • runtime: upgrade to v3 version of race detector #49761
      • Dmitri sent comments on latest version of LLVM CL. On hold at the moment.
      • Than: We’re going to need Windows builders with both the old and new toolchain for a while. Old versions of Go won’t understand host objects from new toolchains, so if we just rolled out the new C toolchain, we couldn’t test old supported versions of Go. So we’ll need to test old Go versions on the old C toolchain and the new Go versions with the new C toolchain. Newer versions of Go still support the older toolchain (unless it’s a TSAN build once we upgrade to TSAN v3).
      • tracking issues for this work: update Windows toolchain (#35006), syso update (#49761, #53539), problems with new compilers and new builders (#53540)
• Go 1.20 early in cycle status
  • Unified IR
    • On a branch during the freeze. We’ll merge that ASAP but keep it behind an experiment.
    • Matthew: I've got CLs out for explicit RTTI everywhere. Cuong has reviewed them. Waiting for another reviewer on them. Then working on populating RTTI from dictionaries. David is working on internal google testing.
  • cmd/cover
    • Review progress:
      • cmd/cover CL reviewed by Bryan, comments incorporated
      • Bryan also looked over the cmd/go CLs; working on his comments now.
      • one or two C&R CLs that need reviewing.
      • Hadn’t anticipated how things work on Android, so there may be extra work there. Go test -cpuprofile=x doesn’t work on Android because cmd/go creates a directory on the host but then the guest tries to write to that path and it doesn’t exist. Currently, -cover works but -coverprofile doesn’t, but with new cmd/cover not even -cover works.
  • Michael K: might want to get in my ("remove cache misses from findObject") and Keith’s (one bit bitmaps) GC changes just to soak.
  • Keith: Mine shouldn’t cause conflicts, but it would be good to let it soak.
• Other nasty bugs
  • runtime: random crashes on macOS 13 Ventura Public Beta (#53800)
  • Cherry: This is a very weird bug. It looks like some old value suddenly overwrites a new value on a stack. Now trying to clobber with timestamp bits and after a stack gets copied a bunch of times it suddenly shows old timestamps.
  • Keith: Are we sure it’s just stacks?
  • Cherry: No.
  • Michael P: Sounds like maybe a kernel CoW or page moving race condition.
  • Matthew: Can we send a repro to Apple to bisect on their end what might have changed?
  • Cherry: been exchanging emails with Apple. They said it started between a couple of different kernel intermediate versions ("seeds")
  • Than: I wonder if running under a debugger with a hardware watchpoint might help? Have to know what’s going to get clobbered.
• runtime: running cmd.exec within goroutine sometimes leaves process with 100% CPU #53863
  • Darwin-specific. Has a C repro, but we could work around the kernel bug.
  • Cherry: I can send Apple an email.

Attendees: Austin Clements Cherry Mui David Chase Keith Randall Matthew Dempsky Michael Knyszek Michael Pratt Than McIntosh

[randall77]

2022-07-26

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• Discussion about WebAssembly stack switching
• Go 1.19
  • Release scheduled next Tuesday.
• Go 1.20
  • CLs that need to land before the tree opens?
  • Clean-up p and g (mpratt, high priority, lots of conflict potential)
  • Lock ranking redo (austin, medium priority, would conflict hard with changes to the lock rank graph, but I’m not sure we’re expecting any of those)
  • When is the tree reopening?
  • August 8th
  • Unified IR will be merged, but may not quite be ready right at opening.
  • May set to enabled on the branch for more coverage before merging.
  • One bit heap bit CLs from Keith.
  • MK’s pacer changes.
  • Others?
• “Quarterly” ARM meeting
  • New ARM assembler stuck in review. Goal is to support easy addition of SVE instructions and general maintainability, but there are design disagreements, mostly around the handling of generated code versus hand-written code.
  • PMU-based pprof also stuck in review. Some questions around approach and API design. mpratt is on it.
  • SIMD API. Saw community proposal go by. Russ is active on it. We’re all agreed that we want something platform-independent and vector-length agnostic. ARM is especially interested in it being vector-length agnostic because of new SVE instructions. austin told them not to expect this any time soon. They’re okay with that and just want to be involved in any design work that does happen.
  • ARM is looking at performance gaps where emulated x86 Go binaries on M1 outperform native ARM Go binaries. They’ll share their analysis.
  • ARM is working on Memory Tagging Extension support for Go for interoperating with C code compiled with MTE (sort of a hardware-accelerated ASAN).

Attendees: David Chase Keith Randall Matthew Dempsky Michael Pratt Than McIntosh

[thanm]

2022-08-09

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• Go 1.20 status review
  • All early-in-cycle changes are in and the tree is generally open 🎉
  • Overall generics status
  • Go generics planning for H2’2022
  • [Keith]: Some small optimizations. Iterators???
  • User survey responses on generics adoption challenges are looking good, a lot of people are happy
  • Unified IR [Matthew]
  • Supports local type definitions within generic functions.
  • Merged to master, but is currently off. CL stack is almost ready for switching to shape-based stenciling and turning on unified IR.
  • There are some google-internal packages that fail to compile with OOMs or time-outs.
    • [David]: for the google-internal issues, we are down to two problems with unknown cause. One obvious time-out in progress. A few other failures.
  • Core project health [Austin]
    • AI for everyone: what do you want to work on?
  • PGO preview [Cherry]
    • Planning doc talks about our targets for 1.20 and later.
    • For 1.20, we’ll focus on the compiler and maybe do something simple on the cmd/go side.
    • At most we’ll automatically pick up profiles from the main package.
    • A longer version of the design doc is almost ready for team review.
    • We’re planning to use pprof format for this version (we may revisit that).
    • We’ll have to add a little more to the pprof profiles from the runtime.
    • Orthogonal to proposals to change runtime/pprof API.
  • cmd/cover [Than]
    • Almost done with Bryan Mills’s comments.
    • Biggest change is better support for Android/wasm (where you can’t fork/exec from a running test process).
  • updated Windows TSAN support mostly done now
    • New version of TSAN runtime requires updated builder compiler
    • will need to keep older windows builder configs alive as well since older Go releases need the older compilers.
  • One bit heap bitmap [Keith]
    • Checked in and then reverted.
    • Issue with accessing uninitialized bitmaps of noscan spans.
    • Will fix that and try again.
  • Pacer changes [Michael K]
    • Revisiting a design decision from a while ago to pick a better smoothing function.
  • Arenas [Michael K]
    • Planning to land GOEXPERIMENT=arenas. Already working on this.
    • Now it recycles address space and sets memory to fault (rather than unmapping it). Need to rewrite bitmap code to work with the 1-bit bitmap.
• Bootstrap is now Go 1.17 (#44505)
  • How often should we update the bootstrap? (#54265)
  • [Austin]: If we update every year, does that mean building from scratch scratch takes an extra step every year?
  • [Michael K]: What’s the value of building it from scratch scratch? Even ports don’t start there.
  • [Matthew]: “Trusting trust”. Bootstrapping to a new platform, though 1.4 already doesn’t work on all platforms. We could bootstrap off gccgo.

Attendees: Austin Clements Cherry Mui David Chase Eli Bendersky Jenny Rakoczy Keith Randall Matthew Dempsky Michael Knyszek Than McIntosh

[thanm]

2022-08-16

• Discussion of externally contributed 1.20 C&R work (from 1.20 planning thread)
  • Linux/riscv64 plugin support [Meng Zhuo mengzhuo1203@gmail.com] ⇒ Cherry to take a look?
  • "And riscv codegen from upstream" (unsure what that means)
    • [Michael P]: ARM folks mentioned generated codegen from an ISA description. Maybe it’s the same thing for RISC-V.
  • runtime.Pinner (CL, issue [Sven Anderson sven@redhat.com] ⇒ Austin to take a look
  • Ppc64le POWER10 assembler/compiler support [Lynn laboger@linux.vnet.ibm.com]
    • [Eli]: The new porting policy (see accepted proposal) is in place now. Will this require a significant investment of time from the C&R team to review? Or is this just a cursory review?
    • [Keith]: There’s a distinction between ports that have more than one maintainer who can review each other’s changes and ports that only have one maintainer.
    • [Eli]: Ports need to have two maintainers now.
    • [Keith]: I don’t think we’re there yet for all ports, so we have to keep helping people along.
    • [Austin]: We should be helping the port get out of the tree if it can’t get another maintainer.
    • [Eli]: If we keep doing what we’ve been doing, this policy will fall by the wayside. If there are important ports that are having trouble finding another maintainer, we can have a grace period, but we need to be explicit.
    • ⇒ David to take a look
  • Move DWARF type gen into compiler (CL stack, issue) [guangyuan zhou zhouguangyuan.xian@gmail.com]
    • [Matthew]: I’ve spot checked compiler stuff. Some of the CLs need work, communication is proceeding.
    • [Matthew]: Another concern: this requires descriptions of a lot of more runtime types in the compiler. The linker used descriptions from runtime source; the new scheme adds a dozen more types into builtins that have to be kept in sync with the runtime. We’re going to have to be much more careful about keeping those in sync, or we’re going to have to generate the builtins file from the runtime source. Are we willing to endure that pain?
    • [Austin]: I’m okay with pushing back on this. It’s definitely "nice to have", but not critical for anything.
    • [Than]: Linker parts look okay. If I had done this myself, I would generate DWARF in the compiler differently. Author just moved the current (weird) way we generate DWARF in the linker and moved that to the compiler. We can improve that later. It does cause a 2–3% compile time slow-down.
    • [Matthew]: How much do the object files get bigger?
    • A: unknown
    • [David]: Does the link time get better?
    • [Than]: There’s only a small link time win from the numbers I saw (surprising); could simply be due to larger objects. Might be worth digging into this a bit more.
    • [Austin]: What’s the motivation for this?
    • Link time reduction and complexity in the linker. Is it worth doing this?
    • [Than]: The author was motivated to improve debug info in shared libraries. There’s no way to connect the DWARF in a shared library with the DWARF in, say, the runtime. This CL shows how the new scheme for linkshared will work.
    • [Austin]: I think we need to step back and look at the reasons for doing this and whether this change is actually satisfying at least some of them.
  • //go:notinheap cleanup (https://go-review.googlesource.com/c/go/+/421878/7) [Cuong Manh Le]
    • [Matthew]: The first couple CLs seem good to land. I would rather runtime people approve the runtime side. It turns some arrays/non-structs into structs to do the field embedding.
    • [Keith]: I have this on my to-do list.

[thanm]

2022-08-23

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• Public iterators discussion
  • This is mostly agreeing on a standard interface, but has a few impacts on compiler (see "Range loops") and compiler/runtime (see "Appendix: Efficient implementation of iter.NewGen")
• Status
  • Unified IR (mdempsky@)
    • Turned on by default on tip, but not yet on for google internal use. Some useful bug reports, including issue https://github.com/golang/go/issues/54625 related to k8s
    • One last google-internal failure, hope to have it enabled for google internal use this week or next. If it takes longer, that’s concerning and we should revisit enabling it externally.
    • Gating criteria for build speed and RAM impact?
      • +10% build time.
      • Really focused on correctness and feature-completeness. A lot of string building and other things that we could reduce.
    • Plan: fix k8s, enable in google3, then build speed
  • Core project health (austin@)
    • We’re starting to assign the next round of tasks. Partly this has been blocked by wanting to team up people from releases and people from C&R, but not wanting to distract release folks from relui.
    • Release automation work continues, but Heschi and Jenny have freed up to work on other things.
    • Austin, Heschi, and Eli will touch base once more and really kick some things off.
  • PGO preview (cherryyz@)
    • Design doc shared with team. Good feedback. No major problems. Okay to proceed. Planning to make a simplified version of the tool doc and making that public to get feedback. For the implementation side, we’re going to sync with Uber.
    • [Austin]: Write a short proposal to send through the process? Very little of this actually has to go through proposal review.
    • [Cherry]: Just the cmd/go and workflow integration. Maybe link to detailed docs.
    • [MichaelP]: Might include runtime/pprof API.
  • cmd/cover (thanm@)
    • Finished all of Bryan’s comments, waiting for more feedback from Bryan.
    • IDE integration?
      • Haven’t figured out how to enable regular coverage display; hoping to meet with Hana when she gets back.
      • Pretty sure this is a VSCode Go thing, not gopls.
      • You can already point it at a previously collected coverage profile, it just doesn’t seem to show anything.
    • Start thinking about how to communicate this change to the community. “Integration test coverage” is a good, succinct phrasing that captures most of the point. Blog post?
  • One bit heap bitmap (khr@)
    • Landed! Had to fix arena bug. Another bug with noscan spans; need to chose between khr@ and mknsyzek@ fix.
    • Interaction with arenas?
      • [MichaelK]: It does interact with arenas, but the bitmap changes have already landed and I copied khr's change to the google3 patch into my version. Seems to work.
  • Pacer changes (mknyszek@)
    • [MichaelK]: Can land any time, but I want to experiment a bit more first. * Small CL.
  • Arenas (mknyszek@)
    • [MichaelK]: Ready for review; modulo a few questions over how to roll out.

Attendees: Austin Clements Cherry Mui David Chase Eli Bendersky Keith Randall Matthew Dempsky Michael Knyszek Michael Pratt Robert Griesemer Than McIntosh

[thanm]

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• discussion of demo on new functionality in cmd/cover coming in 1.20
  • [Robert]: does the metadata have a signature in case the binary changes?
  • [Than]: yes. filename incorporates hash.
  • [Austin]: differential coverage: would be interesting to have tooling that would report on a given commit what lines are no longer covered in addition to things that are newly covered.
  • [Hana]: go tool covdata subtract use case seems similar to go tool pprof -base, which is clear what is base.
  • [MichaelP]: In addition to the covdata tool, do we provide a package that people can use to build their own tools?
  • [Than] We’d like to, but haven’t yet. There are packages for this, but they’re all internal right now. It makes sense to eventually export those to an x/ repo.
  • [Keith]: Why not default to covering all packages? Or “all but std”? Since you can always filter later, you might as well collect all the data.
  • [Than]: I think it’s easier for the user to only think about packages in the main module. There also is some overhead here. For example, building the runtime with coverage slows things down.
  • [Keith]: I could see having it on for everything but the runtime (and maybe a few other std things) and having the reporting tools filter it by default.
  • [Alan]: Seems like it might be worth measuring at least
  • [Austin]: What do operations mean on coverage profiles with different metadata?
  • [Than]: Similar to today’s results when the source skews from the profile. For merging, it hashes the metadata of each function and if they’re different it treats them as different functions. Source file, lines and columns, and statement structure. If there are insertions above the function (but not in the function itself), that will be considered a change.
  • [Austin]: We’re using function-relative line offsets in PGO to address that.
  • [MichaelP]: That metadata will be in the binary at some point for PGO, and coverage could use it.
  • [MichaelK]: This sounds like what Linux perf does when you compare profiles from “different” binaries: lots of -s and lots of +s. Pprof just hashes on the symbol name, which works a lot better, at least at a function level.
  • [Than]:: For profiling, I see how you definitely want “best effort” matching.
  • [Austin]: better matching seems as though it would be important for differential coverage
  • [Robert]: We have “go fmt”, “go test”, etc. Now we have “covdata”, which seems inconsistent. Maybe "go tool cov" would be better?
• [Than]: Delve team folks are asking about how much work it would be to implement issue 31132 “runtime: stop single goroutine”.
  • [MichaelK]: What’s the use case?
  • [Austin]: Resuming a single goroutine seems clear. I’m not sure for stopping a single goroutine.
  • [Than] upstream delve issue is https://github.com/go-delve/delve/issues/1529
  • [MichaelP]: seems doable, but there are weird edge cases.
  • [Austin] Some of this is already there for GODEBUG=gcstoptheworld
  • [MichaelP]: We need to preempt all goroutines. After a breakpoint stops the process, you have to let them resume a little bit and then stop.
• Status
  • Go generics planning for H2’2022
    • Comparable, Iterators
    • [Robert] Ian and I are discussing a new comparable approach. Ian did a write-up. Plan is to share it at the summit next week. It will simplify what we have now and be backward-compatible and solve the comparable problem.
  • Unified IR
    • [Matthew]: k8s bug is fixed. internal google testing pending. Build speed and RAM are the next priorities.
  • PGO preview
    • [Cherry] new code drop from Uber
    • [Cherry] working on new external design doc
    • [MichaelP] started hacking on runtime support. Mostly about binary size. Already cut down 1% with pclntab change.

Attendees: Austin Clements Cherry Mui David Chase Eli Bendersky Jenny Rakoczy Keith Randall Matthew Dempsky Michael Knyszek Michael Pratt Robert Griesemer Than McIntosh

[thanm]

2022-10-04

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• FYI
  • Public “redefining for loop variable semantics” discussion. Almost entirely supportive!
  • CL 431919 looks like another example of where go:nosplitrec would have been useful.
• discussion around developing 3 year compiler/runtime strategy
  • [Matthew]: We’ve tossed around moving away from self-contained export data. With protobufs in particular, you get n^2 growth. There are trade-offs. Blaze would have to ship around a lot more .x files, though the total size would probably be a bit smaller.
    • [Austin]: How much of a problem?
    • [Matthew]: It’s definitely a nuisance. It was a problem with unified IR. People have had to modify their code to deal with this sometimes (though they were happy to do it because it sped up compiles).
  • [Ian] related: the “action chain” problem with Go limits parallelism.
  • [Matthew]: You don’t have to look at function bodies for type exports, at least. Java had something like this. We could at least type-check everything and then figure out inlining between packages.
  • [Ian]: We’ve had complaints about it in the past for very large builds. It may be somewhat dominated by link time.
• discussion of overhead, increases in "process" overhead relating to getting Go development work done
  • [Robert]: If we really want to increase our velocity, we have to reduce our processes. It starts with little things like the two Googler review rule. They add up. Can we remove red tape.
  • [David]: Important to automate new processes. You can’t necessarily automate all new process, but you could say any new process has to come pre-automated. E.g., for two-Googler rule, we now have a dashboard. That could have been required from the beginning.
  • [Austin]: Partly Go’s maturity has added a lot of internal process.
  • [Robert]: Last week there was a big uptick in contributors sending a huge number of tiny changes. A lot weren’t substantial improvements and could have introduced bugs. Should we say something to the outside? Or just ignore them?
  • [Ian]: These seem similar to an LSC. If anyone doesn’t want to look at them, don’t look at them. We can push back on churn. I think most recent ones have been improvements.
  • [Than]: I think we should push back more, at least to the extent of communicating to ensure people understand it’s a zero-sum game. The time we spend reviewing these takes away from time we could be improving go.
  • [Austin]: There’s an ingress vs egress problem.
  • [Ian]: Some folks who send these sorts of will turn into useful contributors over time, though most don’t. Many recent changes are coming from Chinese contributors, and if the large Chinese Go community is getting more involved, we want to encourage that.

[thanm]

2022-10-11

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• FYI
  • (Another situation showing the need for go:nosplitrec -- CL 441859)
  • Couple of interesting talks at GopherCon on using a WebAssembly VM written in Go to load and isolate plugins written in wasm into a Go program.
  • It’s not totally clear how a trend like this would affect us, except that it would be nice if those plugins could be written in Go and offer a good experience. (Probably requires WASI; if loading multiple plugins, maybe binary size matters?)
  • [MichaelP]: Do we know what language people were writing these plugins in?
  • [Ian]: Some were in Go. Another example was specific to writing the plugins in some particular language.
  • [Keith] The plugins also worked on all architectures.
  • [MichaelP]: Was this a security sandbox?
  • [Ian]: Partly. It was a good talk.
  • [Cherry]: Were they doing a single entry or lots of crossings?
  • [Ian]: They were running a subprocess and just passing a byte stream over stdin/stdout.
• Bitmap change
  • Post-mortem: the bitmap change caused a fair amount of roll-out and debugging pain. How can we improve the process in the future? Were there process steps we missed (e.g., GOEXPERIMENT-gating the change)?
  • [Keith]: We didn’t wrap it in any knobs. It’s difficult because we were changing the data structures, too, so it would be hard to keep around both the old and new code.
  • [Austin]: It could have been behind build tags to choose statically.
  • [Matthew]: Could we have put both paths behind build tags? What could have been different about how GOEXPERIMENT works today that would have made this easier?
  • [Keith]: #ifdef’ing individual functions (and struct fields) so you don’t have to move them all to a separate file first.
  • [MichaelK]: Separating it into files wouldn’t have been easy. You’d need to fork the heapArena type, or have an indirection. (Embedding helps with this.) I’m not sure how much build tagging this would actually have helped. There were some corner cases being caught only be cgo checks (e.g., those are one of the very few places we read bitmaps for noscan objects). It was pretty clear from the dashboard right away that this was the problem.
  • [Austin]: The breakage lasted a while on the dashboard, right?
  • [Keith]: We figured out that it was the bitmap change quickly, but it took a while to figure out the fix.
  • [Keith]: Doing it early in cycle was more valuable than gating it, since we could stabilize it long before the freeze.
• 3 year strategy
  • What are the structural barriers in the compiler? E.g., we've been saying for years that we'd like to move inlining to SSA, but that requires moving at least escape analysis to SSA, and maybe a high level SSA. Are there more things like that and are there common themes? Things we could address for the long term health of the compiler project?
  • [Matthew]: Get rid of the middle end. Do as much as possible in SSA and get to SSA as directly as possible. There’s a roadmap to that and I have some ideas. * [Robert]: I believe we still use the old type checker for parts. We should get rid of that.
  • [Matthew]: We still use it when we generate code internally in the backend. It’s used less and less. We don’t use it for checking user code. E.g., wrappers, equality and hashing functions, desugaring in some walk steps. Unified IR got rid of a lot of this in the frontend, but most of what’s left is in the backend. If we had a higher-level SSA with function calls and map lookups, etc, we could directly generate those, and desugaring those would happen as SSA passes.
  • [Austin]: Do you wind up with SSA level mismatches?
  • [Ian]: There is an analogous thing in gccgo where sometimes you’re at a lower level and you want to generate at a higher level. I just call the lowering steps by hand and that works.
  • [Eli]: What’s the cost of keeping the old type checker?
  • [Keith]: It’s just a convenience so you can generate IR without specifying all of the types. It can fill in the types for you. Getting rid of it would make existing code more verbose.
  • [Matthew]: As we’ve been adding IR nodes, we’ve been making their constructors fill in things like that. That makes the API simpler and leaves less room for mistakes.

Attendees: Austin Clements Cherry Mui David Chase Eli Bendersky Ian Lance Taylor Keith Randall Matthew Dempsky Michael Knyszek Michael Pratt Robert Griesemer Sameer Adjmani Than McIntosh

[thanm]

2022-10-18

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

2022-10-18

• approximately one month until release freeze (again)
• Unified IR export data size is apparently causing some unexpected load increase in one of the Google-internal build systems; discussion about what might be causing this and how to handle it.
  • [Matthew] notes:
    • Unified IR encodes the full package graph in the export data, which has more overhead than I expected, especially for small packages. Previously, we only wrote out the packages that were relevant. We could drop that precise graph as a way to reduce export data size.
    • In indexed export format, we used compressed positions, which isn’t implemented in unified IR yet. That would probably be an easy win.
    • Unified IR doesn’t do as aggressive unreachable method pruning, which could also be a factor.
    • note that any export data format change requires many steps, unfortunately
  • [Robert]: I think second item (compressed pos) is an easy win. It made quite a difference in the earlier format.
  • [Eli]: Would fixing this make build speed better, too?
  • [Matthew]: I think most of the issues are elsewhere. But might possibly help for the google-internal case, maybe.
  • [Matthew]: Prioritizing export format issues over build speed makes sense because of the length of the pipeline.
  • [Keith]: Q: Extra synchronization stuff in the format?
  • [Matthew]: That’s not on by default.
• Batching write barriers.
  • Keith has circulated an early stage doc laying out a design for reducing write barrier overhead, and an experimental CL
  • discussion about CL and of cgocheck=2 mode, which depends on the existing write barrier scheme
  • Keith to talk with Ian about how much cgocheck=2 is used
• Status
  • "Comparable" work
    • [Robert] We have a proposal. Sent to Axel Wagner and gotten some good feedback, but doesn’t see anything wrong with it. Waiting for Ian. Hopefully we can send it out this week. The actual change would be ~5 lines of code, so we could actually do it if we get it approved in time. That would fix the comparable issue and would be forward-compatible with the full change in the doc.
    • [Robert] implementation could be done behind a flag.
  • Build speed/RAM (unified IR)
    • David evaluating a GC hack targeting this
  • PGO preview (cherryyz@)
    • [Cherry]: on the compiler side, Uber’s CL seems in reasonable shape. We’re planning to bring their code in and iterate from there.
    • [Cherry]: inlining heuristics may need more work.
    • [Cherry]: Runtime start line numbers are in, but needs to be incorporated into the profile consumer. cmd/go flag needs work on building multiple targets.
    • [David]: Have we tried profiling the compiler?
    • [Cherry]: probably need the revised inlining heuristics for that work. Also, the compiler is hand-optimized.
  • Integration testing coverage
    • [Than]: all CLs in, turned on. Working through long tail of bugs.
  • Arenas
    • [MichaelK]: done, all systems go at this point.
  • Pacer changes
    • [MichaelK]: no updates
  • Core project health work
    • [Austin]: Proceeding on various fronts.

[thanm]

2022-10-25

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• ARM’s big asm change (go.dev/cl/424138) seems stalled. What is it blocked on?
  • [Cherry]: Some older CLs not the right approach. Need to take a look at newest iterations. Plan to take a look this week.
• C&R team met with ARM folks to sync up on Go related things
• Any objections to openbsd/ppc64 port? (issue #56001)
  • no objections in meeting so far
• [discussions of google-internal compiler deployment issues]
• [discussions of google-internal testing issues]
• topic: making it easy to change the compiler’s export info
  • brainstorming has been happening on this in various meetings
  • number of stakeholders who would benefit, including gopls, the vulnerability checker, etc.
  • [Matthew] I’ve been experimenting with approaches to speed up export data compilation
• General discussion of how we should go about categorizing/measuring tasks that we think of as "maintenance"
  • [Keith]: Responding to external CLs. How much are we reviewing CLs? How much time does it take to review? Time spent chasing bugs in these CLs? Time saved by contributors doing work for us?

Attendees:

Cherry Mui David Chase Eli Bendersky Ian Lance Taylor Keith Randall Matthew Dempsky Michael Knyszek Michael Pratt Robert Griesemer Than McIntosh Jenny Rakoczy

[thanm]

2022-11-01

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• Page tracer demo at team meeting(presented by mknyszek)
  • Should we land this?
  • mpratt@: I like it and I’m in favor of merging it. Keep behind a GODEBUG so we don’t have a compatibility promise.
  • mknyszek@: Ideally this would eventually just be part of the execution tracer.
  • khr@: Can we steal back the P page cache?
  • mknyszek@: It mostly impacts smaller heaps. We could flush it at the same time as mcaches.
  • austin@: I think the bar for landing tooling that’s primarily for our own debugging should be pretty low.
  • Should we have an x/exp subdirectory for debugging tools?
• FYI
  • public discussion of rsc@’s “user-defined iteration using range over func values”
• Status
  • Unified IR export data size increase
    • mdempsky@: I have plans for position compression. I might do just the function bodies since those dominate space and that won’t need the x/tools step. In the short term, I don’t know that there’s other stuff to do. Longer-term, thin export data or some larger-scale build restructuring.
  • David’s GOGC boost CL
    • khr@: I’m happy with it. It’s ugly but safe. It depends on finalizers (being prompt).
    • mknyszek@: What directly solves this problem is a memory target, but that’s another GC knob.
    • mdempsky@: I agree with Keith. Some of the GOGC computations are kind of magic, so it would be nice if Michael or Austin could look it over. Alt: SetMemoryLimit until we reach that, then restore GOGC?
    • mknyszek@: The calculations look correct. SetMemoryLimit: if the live heap grows fast and the finalizer doesn’t run promptly, you could get into thrashing and slow things down. The risk of the GOGC approach if the finalizer is delayed is that the GC uses more memory.
    • mdempsky@: I think the target heap size David has is low enough that even if we overshoot significantly, it’s okay.
  • PGO preview (cherryyz@)
    • Landed Uber’s compiler CL! Doing lots of improvements now and tuning heuristics. Doing benchmark measurements.
    • Considering switching from fixed percent threshold to a CDF threshold (this is what LLVM does).
    • Working on cmd/go changes. Not sure we’ll get to supporting multiple main packages. If we don’t get to that, default –pgo=off so you have to pass –pgo=auto explicitly (and only one main package).
    • Inlining testing?
    • mpratt@: With heavy inlining, something in the GC didn’t work and the compiler would hang. Maybe we should have a “heavyopt” builder just to stress the inliner (outside of PGO).
    • cherryyz@: We could introduce something like GOSSAHASH to do inline stressing. Useful for narrowing things down and debugging.
  • Integration testing coverage (thanm@)
    • Fixed more bugs, did a little more cleanup. Writing documentation.
  • Core project health (austin@)
    • Automated release process (relui)
    • Implementing new ports policy (iant@)
      • Just a couple of issues: Supporting building a cross-compile toolchain without using cgo (but that itself still defaults to using cgo). Add a list of broken ports to cmd/dist and refuse to build them by default.
      • cherryyz@: Timeline?
      • iant@: This release if possible, but not urgent.
    • Reduce builder maintenance toil (image management)
    • Test triage tooling (watchflakes)
    • Machine-readable all.bash output
    • Investigate alternate CI systems
    • [Deprioritized] Test artifact collection, submit queue
  • Pacer changes (mknyszek@)

Attendees:

Cherry Mui David Chase Ian Lance Taylor Keith Randall Martin Möhrmann Matthew Dempsky Michael Knyszek Michael Pratt Robert Griesemer Than McIntosh

[thanm]

2022-11-08

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• David’s compiler GC hack landed
  • big speedup in compile time, visible on performance tracker
  • khr@: Would the same hack help the linker?
  • mknyszek@: from looking at page trace, I think it might help a little, but not as much
  • Do we have a geomean build time delta of master vs. 1.19 release across the benchmarks?
  • https://perf.golang.org/search?q=upload%3A20221108.15+pkg%3Agolang.org%2Fx%2Fbenchmarks%2Fsweet%2Fbenchmarks%2Fgo-build+%7C+toolchain%3Abaseline+vs+toolchain%3Aexperiment
• Status
  • Core project health (austin@)
    • Reduce builder maintenance toil (image management)
      • thanm@: Initial meeting with Jenny. Next step is to write a brainstorming/design doc.
    • Test triage tooling (watchflakes)
      • cherryyz@: Russ is running it somewhat continuously on his machine. CLs under review. Once those are in, Heschi wants to work with Russ on getting it running in the Cloud.
    • Machine-readable all.bash output
      • austin@: I’m pretty close to mailing out a big refactoring that should make this “easy”.
      • cherryyz@: watchflakes depends on the current output.
      • austin@: The existing dashboard endpoint will continue returning text output (translated from JSON stored in back-end).
    • Investigate alternate CI systems
      • mknyszek@: We’re moving forward with investigating LUCI more deeply. Gap analysis and prototype.
      • mpratt@: Working on moving Macs to AWS. We canceled MacStadium. We’re also talking to some google internal teams.
    • Deprioritized] Test artifact collection, submit queue
  • Pacer changes (mknyszek@)
    • Landed today. It fixes a bug and simplifies the pacer, and the problems with it fixed themselves.
  • Comparable: new proposal #56548
    • Already implemented behind a compiler flag
    • gri@: “Clear” proposal is likely to be accepted soon. The CL on the type-checker side is implemented. That could go in to 1.20 depending on how the proposal process goes.

Attendees:

Austin Clements Cherry Mui David Chase Eli Bendersky Ian Lance Taylor Keith Randall Matthew Dempsky Michael Knyszek Michael Pratt Robert Griesemer Than McIntosh

[thanm]

2023-01-03

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• Happy New Year
• 1.20 Release blockers
  • discussion of #54778 and #51550; still need more investigation here.
• Latest build time results (do we need to update release notes)?
  • MichaelK: still trying to reproduce what we see on the dashboard on other machines
  • problems with benchmarking on magna, but MichaelP was able to repro on workstation.
  • Separate machine shows smaller (½) improvement than shown on dashboard
  • David: new compiler may be using GOMAXPROCS, older compiler may be looking at ncpus(). This could account for diffs.
  • MichaelK and David to collaborate on revised release notes wording
• VSCode HaTS responses on debugging
  • many comments in open responses related to debugging
  • not clear whether how much there is that C&R can help with, depending
  • several mentions of calling functions from the debugger (assume this is about the limitations on calling functions getting in the way)
  • a few mentions of better support for printing expressions, which might be about DWARF or might not.
  • ThanM to discuss with delve team during checkin next week
  • Cherry: there may be issues calling functions where an argument is escaped (debugger doesn’t know escapiness). We could add tags in the DWARF for this. Example: fmt.Printf. Could delve add special casing?
  • David: can we come up with a helper function that would put something into the heap, so as to “heapify” things that are being passed to a func?
  • Keith: may not be possible? What happens if called function wants to modify object pointed to by param?
  • MichaelK: maybe allocate heap value, copy stack to heap, then copy heap back to stack on return? not perfect, but might be better than what we have now
  • Cherry: interesting idea, but many potential pitfalls (especially if params are copied into globals, etc). maybe with a new special debugger mode (“unsafe call”)
  • Keith: need to consider stack objects
  • Keith: idea: delve already runs with flags to turn off optimization – could we turn off stack allocation entirely?
• Discussion of enterprise research survey
• Unified IR caused a regression in GoAWK: https://groups.google.com/g/golang-dev/c/8blU7CnuhZ4/m/d21xuN8hAQAJ
  • Keith: threshold for “very large function” looks like the issue here; with unified the func in question is slightly bigger due to diffs in assignments for multi-return function calls. This results in lowering of the 'inlinable function' cost threshold, meaning that a number of key calls don't get inlined.
  • may not need conversions in this specific case (might be something we could change in the compiler)
  • proposal: bump up the magic number for large functions slightly to account for this case
  • specific large function size number is probably not important
  • Q: do we need an issue or bug here? A: Matthew will look into it, will file issue if needed
  • David: RC2 is already “leaving the station”, might need an RC3 if it is decided this change needs to be included.

Attendees:

Cherry Mui David Chase Keith Randall Matthew Dempsky Michael Knyszek Robert Griesemer Than McIntosh

[thanm]

2023-01-10

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• Next "Public Go C&R office hours" session scheduled for Thursday, Jan 19 at 3 ET/12 PT
• discussion of whether/when we might need to implement a large data model (e.g. equivalent of GCC's "-mcmodel=large") for Go
  • issue arises in some large Google-internal C/C++ programs that link in bits of Go code
  • [Keith] For code it might not be that bad because we have trampolines on other archs. Data is still a problem.
  • [Cherry] For C++ depending on Go, as long as Go (text) itself is < 2GB and we do access with GOT entries, we’re probably fine? Go code calling C is all function pointers, so maybe that’s also fine? There were a few issues with direct relocations, but we fixed those.
• 1.20 Release blockers
  • We still have quite a few 1.20 non-release blockers
• Tree opening early for 1.21.
  • Is there anything that needs to be submitted early?
  • [Matthew]: Big stack for removing non-unified. Probably won’t conflict, so probably not urgent. Should we wait until we see the 1.20 release go out before removing that or just do it as soon as the tree opens?
  • [Robert]: I have a stack I’d like to land
• discussion of shortened freeze (general outlook on it seems positive)
  • [Matthew]: It seemed like we did a good job of getting the risky stuff in early and stabilizing the tree before the freeze rather than leaving stability work for the freeze.

Attendees:

Austin Clements Cherry Mui David Chase Ian Lance Taylor Keith Randall Matthew Dempsky Michael Knyszek Michael Pratt Robert Griesemer Than McIntosh

[thanm]

2023-01-17

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• Discussion of team plans for first half of 2023, including items such as: dropping non-unified IR and removing associated GOEXPERIMENT, next steps for PGO, revamping the inliner, core health CI/CD tasks, batched write barriers, init ordering fixes, coverage trybots, long-term RAM efficiency efforts
• things are still in flux here, more work needed to produce a concrete plan

Attendees:

Austin Clements Cherry Mui David Chase Eli Bendersky Ian Lance Taylor Keith Randall Matthew Dempsky Michael Knyszek Michael Pratt Robert Griesemer Than McIntosh

[thanm]

2023-01-24

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

• Discussion about recent Google layoffs
• Discussion about whether to have the C&R team participate in the summer internship program
  • [Robert]: We have a bunch of proposals (e.g., language changes like lightweight function literals) we’d like to play with/try out. An intern could write prototypes. We haven’t made much progress. If someone would implement those, maybe even as code rewriters, it would give us more insight into what it would look like in real code. And it might be useful for the community to see and drive decisions. It doesn’t require a particular outcome. It’s fun for an intern. And we’re not dependent on it. Requires some affinity to language stuff, and often/frequent communication.
  • unsure whether this is going to happen however (schedule, etc)
• Discussion about dependencies in new Go features in 1.21-1.23
  • we have a number of features with long dependency changes, e.g. in order for us to roll out X we need Y, and Y needs Z, etc.
  • notable: issue #56986
  • Main lasting impact on us: “Commit to always adding a GODEBUG setting for changes allowed by the compatibility guidelines but that nonetheless are likely to break a significant number of real programs.” and “Guarantee that GODEBUG settings last for at least 2 years (4 releases).”
  • [Matthew] The current GODEBUGs are very specific things. Does this policy change that?
  • [Austin] This is codifying that practice.
  • [MichaelP] Do performance things count?
  • [Austin] I’m not sure.
  • [Cherry] regarding compatibility, how does this impact ares where we don't have the Go 1 compat guarantee (eg assembly code, odd things with "unsafe")?
  • [Austin] don't think so
  • [Cherry] What if old code doing weird things stops compiling? Is that okay? Maybe for assembly code?
  • [Matthew] I think assembly is outside Go 1 compatibility.
  • [Austin] “likely to break a significant number of real programs” is a judgment call
  • [Ian] I agree it’s a judgment call. It’s intended for run-time changes where we change a package that makes it behave differently from how it did before. I would expect very few cases where the runtime package itself needs this. It’s not intended for performance. We could add one for performance, but it gives us more knobs to support.
  • [Keith] If we’re doing this for k8s, we really need them to run the RCs and give us feedback. There are going to be things we change that we don’t realize they’re depending on.
  • [Austin] I think they do run the RCs.
  • [Matthew] They noticed issues with unified IR.
• proposal: extended forwards compatibility for Go #57001 is close to accepted
  • At minimum, this will require the compiler to understand per-file language versions, not just per-package. (AFAIK, the details of whether the compiler or cmd/go figure out the versions aren’t fleshed out yet.)
  • [Cherry] Do different versions of the language work together?
  • [Robert] Say the loop variable capture change is in one file, and it gets inlined into another file. We have to track that.
  • [Austin] We have to deal with that anyway for cross-package inlining.
  • [Keith] There are (potential) changes we couldn’t do this way. Like, making int 64-bits everywhere.
  • [Matthew] I have some similar concerns with per-file language, but I suspect it’ll work because we have to support per-package versions already. I’m not looking forward to the changes to go/types.
  • [Ian] I don’t think it’ll be a big deal. It’ll mostly be “in order to use this new feature you have to use a language tag.”

Attendees:

Austin Clements Cherry Mui David Chase Eli Bendersky Ian Lance Taylor Keith Randall Matthew Dempsky Michael Pratt Robert Griesemer Than McIntosh