Incremental compilation produces runtime crash

[mbrandonw]

Describe the bug

It is possible to make a program that when compiled and run works correctly, but then if a small change is made and run again it will consume a large amount of memory and then crash at runtime.

Further, if the target is cleaned and run again it works fine, which makes me think it is a bug with incremental compilation.

This happens in Swift 5.5-5.7 and perhaps earlier.

Steps To Reproduce Steps to reproduce the behavior:

1. Open the demo project: incremental-bug.zip
2. Navigate to Feature.swift
3. Run the application and observe that it completes successfully.
4. Uncomment the 3 lines in Feature.swift and run again.
5. Observe that the application consume \~1.5GB of memory and then crashes.
6. Clean the target (cmd+shift+K) and run again to observe that it completes successfully again.

Expected behavior The application should not crash and should print the following to the console:

Sending .action1("Hello")
Running  action1("Hello")
Sending .action2
Running  action2

Environment (please fill out the following information)

• OS: macOS 12.4 (21F79)
• Xcode 13.2.1 (13C100) / Swift 5.5

[mbrandonw]

It's worth noting that I have found a few ways of working around this runtime crash:

• Adding another case to the enum with an associated type, such as case action3(Int).
• Putting all of Feature.swift's contents in IncrementBug.swift
• Extracting the Action enum to be defined at the file scope rather than the Feature struct.

In all of those situations the incremental compilation of the application somehow does not cause the runtime crash.

[CodaFi]

This is a fantastic issue, thank you for reducing the problem so well. The package manager is at least reporting that it's rebuilding Feature.swift during all of this which is good. There is a named reference to the Feature struct, but no direct reference to Feature.Action in IncrementalBug.swift and if that file is touched and rebuilt as well then this bug doesn't reproduce.

At first blush the bug is that Store(Feature()) should imply a potential member dependency on the exact witness to P.A supplied by Feature - which is Feature.Action. This is corroborated by a diff of the .os in the baseline and busted cases. The busted case sees the old layout of Feature.Action - which has trivial layout since it's just the single action2 case, even though the addition of action1 selects a completely different layout and makes it a non-trivial type with a retainable pointer inside for the String.

Working:

_$s15incremental_bug14IncrementalBugV4mainyyFZAA7FeatureV6ActionOAHcfU_:
00000000000000fc    sub sp, sp, #0x40
0000000000000100    stp x29, x30, [sp, #0x30]
0000000000000104    add x29, sp, #0x30
0000000000000108    str x8, [sp, #0x10]
000000000000010c    stur    xzr, [x29, #-0x10]
0000000000000110    stur    xzr, [x29, #-0x8]
0000000000000114    ldr x8, [x0]
0000000000000118    str x8, [sp, #0x8]
000000000000011c    ldr x0, [x0, #0x8]
0000000000000120    str x0, [sp, #0x18]
0000000000000124    stur    x8, [x29, #-0x10]
0000000000000128    stur    x0, [x29, #-0x8]
000000000000012c    bl  _swift_bridgeObjectRetain
0000000000000130    ldr x10, [sp, #0x8]
0000000000000134    ldr x9, [sp, #0x10]
0000000000000138    ldr x8, [sp, #0x18]
000000000000013c    str x10, [x9]
0000000000000140    str x8, [x9, #0x8]
0000000000000144    ldp x29, x30, [sp, #0x30]
0000000000000148    add sp, sp, #0x40
000000000000014c    ret

Busted:

_$s15incremental_bug14IncrementalBugV4mainyyFZAA7FeatureV6ActionOAHcfU_:
00000000000000fc    ret

[CodaFi]

This reproduces all the way back in Xcode 13 albeit less spectacularly since it only eats a few megs before crashing trying to render the debug description of the zombie String enum payload.

[CodaFi]

I suspect this bug has been around for... basically since the incremental build was written. < Xcode 13 did not run its dependency tracking machinery past semantic analysis, and name lookups alone are not enough to actually net this dependency. We would (and still) need to record that the closure depends on the layout of Feature.Action, which is something only >= SILGen can do.

[mbrandonw]

FWIW we've had a few people run into this incremental compilation crash in our library: https://github.com/pointfreeco/swift-composable-architecture/discussions/1564.

[lukeredpath]

This issue is a constant source of frustration - we use the composable architecture and every time I change an associated value type (or add one) to one of our action enums, the app will crash and I will need to do a clean build to resolve it.