[SR-9865] swiftc segfault when inlining non-optional closure properties: MandatoryInlining

[swift-ci]

Previous ID	SR-9865
Radar	rdar://problem/47821242
Original Reporter	helge (JIRA User)
Type	Bug
Status	Closed
Resolution	Cannot Reproduce

Attachment: Download

Environment

2019-01-28(a) toolchain in Xcode 10.2b2 on 10.14.3 (18D42)

Additional Detail from JIRA

| | | |------------------|-----------------| |Votes | 0 | |Component/s | Compiler | |Labels | Bug, CompilerCrash, SILOptimizer | |Assignee | None | |Priority | Medium | md5: ad0879c98d697fb5111cc2d3f10ff4c6

Issue Description:

1.  While running pass #​6135 SILModuleTransform "MandatoryInlining".
0  swift                    0x000000010c7ab6e8 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
1  swift                    0x000000010c7aa965 llvm::sys::RunSignalHandlers() + 85
2  swift                    0x000000010c7abcf2 SignalHandler(int) + 258
3  libsystem_platform.dylib 0x00007fff72856b3d _sigtramp + 29
4  swift                    0x000000010dac2ae5 cmark_strbuf__initbuf + 131144
5  libsystem_c.dylib        0x00007fff727141c9 abort + 127
6  libsystem_c.dylib        0x00007fff726dc868 basename_r + 0
7  swift                    0x00000001096df44e runOnFunctionRecursively(swift::SILOptFunctionBuilder&, swift::SILFunction*, swift::FullApplySite, llvm::DenseSet<swift::SILFunction*, llvm::DenseMapInfo<swift::SILFunction*> >&, llvm::ImmutableSet<swift::SILFunction*, llvm::ImutContainerInfo<swift::SILFunction*> >::Factory&, llvm::ImmutableSet<swift::SILFunction*, llvm::ImutContainerInfo<swift::SILFunction*> >, swift::ClassHierarchyAnalysis*) + 6206
8  swift                    0x00000001096ddb8f (anonymous namespace)::MandatoryInlining::run() + 703
9  swift                    0x00000001096f29bb swift::SILPassManager::runModulePass(unsigned int) + 1019
10 swift                    0x00000001096f3424 swift::SILPassManager::execute() + 692
11 swift                    0x0000000108d1676b swift::SILPassManager::executePassPipelinePlan(swift::SILPassPipelinePlan const&) + 187
12 swift                    0x00000001096fbe54 swift::runSILDiagnosticPasses(swift::SILModule&) + 132
13 swift                    0x0000000108bc5ddc performCompile(swift::CompilerInstance&, swift::CompilerInvocation&, llvm::ArrayRef<char const*>, int&, swift::FrontendObserver*, swift::UnifiedStatsReporter*) + 11244
14 swift                    0x0000000108bc215d swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 3021
15 swift                    0x0000000108b740fe main + 686
16 libdyld.dylib            0x00007fff7266bed9 start + 1
17 libdyld.dylib            0x000000000000008a start + 2375631282
error: Abort trap: 6 

[swift-ci]

Comment by Helge Heß (JIRA)

Ahead there is this:

/Users/helge/.../WebSocket.swift:603:9: warning: 'Hashable.hashValue' is deprecated as a protocol requirement; conform type 'InnerWebSocket' to 'Hashable' by implementing 'hash(into:)' instead
    var hashValue: Int { return id }
        ^
Assertion failed: (isa<X>(Val) && "cast<Ty>() argument of incompatible type!"), function cast, file /Users/buildnode/jenkins/workspace/oss-swift-5.0-package-osx/llvm/include/llvm/Support/Casting.h, line 241.
Stack dump: 

and the mentioned class does this:

private class InnerWebSocket: Hashable {
...
    var hashValue: Int { return id }

Fixing this removes the warning, but leaves the assertion/dump.

[swift-ci]

Comment by Helge Heß (JIRA)

BTW: Xcode 10.2b1 builtin toolchain compiles everything, but fails at runtime w/ Radar #47598583.

[swift-ci]

Comment by Helge Heß (JIRA)

Same in 2019-02-04a

[belkadan]

@gottesmm, is there anything we can do without the project? :-/ (Helge, I assume you can't share it here, or you would have.)

[swift-ci]

Comment by Helge Heß (JIRA)

No, sorry, this isn't OpenSource. It is a medium sized Cocoa application. I'm also not sure how to isolate it.
Given the "MandatoryInlining", maybe it is related to `@inline(__always)`, I could try dropping such and see whether it helps.

[atrick]

I don't think `@inline(always)` will affect MandatoryInlining, and I assume you're not using `@_transparent`. Is this a Debug or Release build. Whole module optimization? It would be great if you reduced this just by stripping code away except for the part being inlined. Does the error output tell you which function was being inlined? If not, you can do this in asserts builds -Xllvm -debug-only=mandatory-inlining.

[swift-ci]

Comment by Helge Heß (JIRA)

I'm not using @_transparent. This is a Debug build, which I don't think does WMO. I don't think I'll have the time to "stripping code away except for the part being inlined" (i.e. isolate the issue), it's a big module in a somewhat sizeable application. No, the error output doesn't tell the function.

I can try your flags and see whether that produces extra info.

[swift-ci]

Comment by Helge Heß (JIRA)

OK, the debug switch yields (replaced some stuff w/ ...):

Inlining @$s12SlackService14InnerWebSocket33_4C5C1157562B8580B792527D07326A90LLC6ecloseyycvg into @$s12SlackService14InnerWebSocket33_4C5C1157562B8580B792527D07326A90LLC8copyOpen_12subProtocolsAD10Foundation10URLRequestV_SaySSGtF
Inlining @$s12SlackService14InnerWebSocket33_4C5C1157562B8580B792527D07326A90LLC6ecloseyycvs into @$s12SlackService14InnerWebSocket33_4C5C1157562B8580B792527D07326A90LLC8copyOpen_12subProtocolsAD10Foundation10URLRequestV_SaySSGtF
Assertion failed: (isa<X>(Val) && "cast<Ty>() argument of incompatible type!"), function cast, file /Users/buildnode/jenkins/workspace/oss-swift-5.0-package-osx/llvm/include/llvm/Support/Casting.h, line 241.
Stack dump:
0.  Program arguments: /Library/Developer/Toolchains/swift-5.0-DEVELOPMENT-SNAPSHOT-2019-02-04-a.xctoolchain/usr/bin/swift -frontend -c ... -I /tmp/DerivedData/.../Debug -F /tmp/DerivedData/.../Debug -enable-testing -g -module-cache-path /tmp/DerivedData/ModuleCache.noindex -swift-version 4.2 -enforce-exclusivity=checked -Onone -D DEBUG -D Xcode -D DEBUG -D Xcode -serialize-debugging-options -enable-anonymous-context-mangled-names -Xllvm -debug-only=mandatory-inlining -Xcc -I/tmp/DerivedData/.../swift-overrides.hmap -Xcc -iquote -Xcc /tmp/DerivedData/.../...-generated-files.hmap -Xcc -I/tmp/DerivedData/.../...-own-target-headers.hmap -Xcc -I/tmp/DerivedData/.../...-all-non-framework-target-headers.hmap -Xcc -ivfsoverlay -Xcc /tmp/DerivedData/.../all-product-headers.yaml -Xcc -iquote -Xcc /tmp/DerivedData/.../...-project-headers.hmap -Xcc -I/tmp/DerivedData/.../Debug/include -Xcc -I/tmp/DerivedData/.../DerivedSources-normal/x86_64 -Xcc -I/tmp/DerivedData/.../DerivedSources/x86_64 -Xcc -I/tmp/DerivedData/.../DerivedSources -Xcc -DDEBUG=1 -Xcc -working-directory/... -module-name ... -o /tmp/DerivedData/.../Objects-normal/x86_64/SeeSlackAccount.o -o /tmp/DerivedData/.../Objects-normal/x86_64/SlackTokenFieldEditor.o -o /tmp/DerivedData/.../Objects-normal/x86_64/WebSocket.o -index-store-path /tmp/DerivedData/Slick-cuurtfgacacaqudtltapmwordhsn/Index/DataStore -index-system-modules 
1.  While running pass #&#8203;6135 SILModuleTransform "MandatoryInlining".

Total inlines are 2210 up to that:

$ grep "^Inlining" /tmp/swift5-swiftc-crash-inlining-info.txt  | wc -l
    2210

The WebSocket library I'm currently using is actually OpenSource, this seems to be the last "Inline" which then fails: https://github.com/tidwall/SwiftWebSocket/blob/master/Source/WebSocket.swift#L591

Oh, it is actually a property being inlined into this function:

    var eclose : ()->() = {} 

Tried marking this `@inline(never)`, but that has no effect.

So looks like an issue when inlining properties containing closures?

[swift-ci]

Comment by Helge Heß (JIRA)

If I change that property to an Optional:

    var eclose : (()->())? 

it compiles, yay!

However, I still run into Radar #47598583 when attempting to run the program. But that is a separate problem.

[atrick]

Radar #47598583 was supposedly fixed in 10.2 beta 2. If it isn't fixed, let us know.

Now we're getting somewhere with the inlining crash! Can you tell me if you hit this just by building SwiftWebSocket, if so which scheme/target/config? I'm trying to reproduce it.

[swift-ci]

Comment by Helge Heß (JIRA)

As I mention in Radar 47598583, this is not fixed in 10.2 beta 2, and according to @jckarter there have been no Swift fixes in beta 2 anyways. It also is not fixed in the separate 5.0dev 2019-02-04(a) toolchain (which he suggested). I have a gut feeling it might be some threading thing.

I do not use the SwiftWebSocket as a project, I just included (copied) its single Swift file (WebSocket.swift) in my larger project (it is part of a static library which is being linked together with a macOS app). So it is kinda hard to isolate for me, sorry.

[atrick]

Well, clearly 47598583 it the bigger problem here. Regarding this inlining crash though, I haven't run into issues building SwiftWebSocket myself. I don't see what could go wrong with the simple assignment and don't understand why making it Optional helped.

Are there any other uses of the `eclose` property in your code? Do you have multiple swift modules and is the property being accessed across modules?

[swift-ci]

Comment by Helge Heß (JIRA)

I don't even touch that `eclose` myself, and no, the WebSocket is kinda immediately wrapped in a higher level event handler object and not exposed to anything at all. I just replaced all `public` from that to `internal`, and it still compiles.

[swift-ci]

Comment by Helge Heß (JIRA)

The WebSocket code has quite a few `@objc` for apparently no particular reason. Maybe I fix that later. This particular bug is fixed for me using the Optional, not sure how much time I can spend on isolating this 🙂

47598583 is what I need right now 😉

[atrick]

Well, I've tried variations on changing the visibility of `InnerWebSocket` and `eclose`. It looks like you're using -wmo because I see multiple .o files in your output from a single frontend invocation. I tried that, but still no crash.

If we can come back to this after 47598583, I'd like to get exactly the code for WebSocket.swift that crashes for you (just the diff from open source is fine).

If I still can't reproduce it, I could look at your output from:

-Xllvm -sil-print-on-error -Xllvm -sil-print-only-functions=<your mangled name of copyOpen> -Xllvm -sil-print-before=mandatory-inlining -Xllvm -debug-only=mandatory-inlining

[swift-ci]

Comment by Helge Heß (JIRA)

Attached the file I use. But sigh, I forgot to mention/notice that I'm running this in 4.2 compat mode (SWIFT_VERSION = 4.2). Maybe it is rooted in that.

I may be confused about the WMO, I thought it was an optimisation option, but now it seems to be a "compilation mode" , hm. Need to check/read-up that.

[swift-ci]

Comment by Helge Heß (JIRA)

The WMO thing might be an Xcode bug, I've checked all build settings and it sure is set to `Incremental` for Debug builds /shrug

[atrick]

Thanks for the additional info. The `-swift-version 4.2` is in your command line above. It's also possible that i'm confused about WMO. I'll look into that when I get a chance and update this SR when I know more.

[atrick]

You might try actually try setting the Debug Compilation mode to "Whole Module".

Alternatively, you can disable batch mode like this swiftc -disable-batch-mode. If both of those work, that tells us it's specific to batch mode.

[swift-ci]

Comment by Helge Heß (JIRA)

For now I went on, making the closure optional fixed the thing for me. Not sure when/if i find the time to investigate this in more detail. sorry!

[atrick]

Please reopen this bug if it can be reproduced with Xcode 11.