swiftlang / swift

The Swift Programming Language
https://swift.org
Apache License 2.0
67.66k stars 10.38k forks source link

[SR-15667] [AutoDiff] Providing exact type for generic parameter in @differentiable can lead to an assertion failure #57946

Open BradLarson opened 2 years ago

BradLarson commented 2 years ago
Previous ID SR-15667
Radar None
Original Reporter @BradLarson
Type Bug
Additional Detail from JIRA | | | |------------------|-----------------| |Votes | 0 | |Component/s | Compiler | |Labels | Bug, AutoDiff | |Assignee | None | |Priority | Medium | md5: 30987d0cfba9255fa519471be9e3d5bd

Issue Description:

When a differentiable function refines a generic parameter to a specific type, but a differentiable function called within that function isn't as refined, it can sometimes lead to an assertion failure of "(OrigType == signature.getCanonicalTypeInContext(origType)), function initSwiftType, file AbstractionPattern.h, line 533".

The following single-file reproducer triggers this:

import _Differentiation

@differentiable(reverse, wrt: (object) where Object: Differentiable, Member: Differentiable)
public func readSlowFrom<Object, Member>(_ object: Object, at member: KeyPath<Object, Member>) -> Member {
    return object[keyPath: member]
}
public struct TestKeyPaths<Root, Value>: Differentiable {
    @noDerivative
    public var keyPaths: Set<KeyPath<Root, Value>> = []

    @differentiable(reverse where Root: Differentiable, Value == Float)
    public func readAll(from root: Root) -> [Value] {
        return [readSlowFrom(root, at: self.keyPaths.first!)]
    }
}

To observe the assertion failure, place the above in a file and build via `swiftc file.swift`. The assertion failure goes away if the line

    @differentiable(reverse where Root: Differentiable, Value == Float)

is replaced with

    @differentiable(reverse where Root: Differentiable, Value: Differentiable)

This appears to not be a recent regression, and is present in current nightly toolchain snapshots. The full text of the assertion failure is as follows:

Assertion failed: (OrigType == signature.getCanonicalTypeInContext(origType)), function initSwiftType, file AbstractionPattern.h, line 533.
Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the project and the crash backtrace.
Stack dump:
0.  Program arguments: /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-12-23-a.xctoolchain/usr/bin/swift-frontend -frontend -c -primary-file /Users/bradlarson/Development/DifferentiableSwift/CompilerBugs/KeypathGenericSpecificationCrasher/KeypathGenericSpecificationCrasher/main.swift -emit-dependencies-path /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeypathGenericSpecificationCrasher-alxngiieimzuczddmdqiiywzdnak/Build/Intermediates.noindex/KeypathGenericSpecificationCrasher.build/Debug/KeypathGenericSpecificationCrasher.build/Objects-normal/x86_64/main.d -emit-reference-dependencies-path /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeypathGenericSpecificationCrasher-alxngiieimzuczddmdqiiywzdnak/Build/Intermediates.noindex/KeypathGenericSpecificationCrasher.build/Debug/KeypathGenericSpecificationCrasher.build/Objects-normal/x86_64/main.swiftdeps -serialize-diagnostics-path /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeypathGenericSpecificationCrasher-alxngiieimzuczddmdqiiywzdnak/Build/Intermediates.noindex/KeypathGenericSpecificationCrasher.build/Debug/KeypathGenericSpecificationCrasher.build/Objects-normal/x86_64/main.dia -target x86_64-apple-macos11.3 -enable-objc-interop -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -I /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeypathGenericSpecificationCrasher-alxngiieimzuczddmdqiiywzdnak/Build/Products/Debug -F /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeypathGenericSpecificationCrasher-alxngiieimzuczddmdqiiywzdnak/Build/Products/Debug -enable-testing -g -module-cache-path /Users/bradlarson/Library/Developer/Xcode/DerivedData/ModuleCache.noindex -swift-version 5 -enforce-exclusivity=checked -Onone -D DEBUG -new-driver-path /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-12-23-a.xctoolchain/usr/bin/swift-driver -serialize-debugging-options -resource-dir /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-12-23-a.xctoolchain/usr/lib/swift -enable-anonymous-context-mangled-names -Xcc -I/Users/bradlarson/Library/Developer/Xcode/DerivedData/KeypathGenericSpecificationCrasher-alxngiieimzuczddmdqiiywzdnak/Build/Intermediates.noindex/KeypathGenericSpecificationCrasher.build/Debug/KeypathGenericSpecificationCrasher.build/swift-overrides.hmap -Xcc -iquote -Xcc /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeypathGenericSpecificationCrasher-alxngiieimzuczddmdqiiywzdnak/Build/Intermediates.noindex/KeypathGenericSpecificationCrasher.build/Debug/KeypathGenericSpecificationCrasher.build/KeypathGenericSpecificationCrasher-generated-files.hmap -Xcc -I/Users/bradlarson/Library/Developer/Xcode/DerivedData/KeypathGenericSpecificationCrasher-alxngiieimzuczddmdqiiywzdnak/Build/Intermediates.noindex/KeypathGenericSpecificationCrasher.build/Debug/KeypathGenericSpecificationCrasher.build/KeypathGenericSpecificationCrasher-own-target-headers.hmap -Xcc -I/Users/bradlarson/Library/Developer/Xcode/DerivedData/KeypathGenericSpecificationCrasher-alxngiieimzuczddmdqiiywzdnak/Build/Intermediates.noindex/KeypathGenericSpecificationCrasher.build/Debug/KeypathGenericSpecificationCrasher.build/KeypathGenericSpecificationCrasher-all-target-headers.hmap -Xcc -iquote -Xcc /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeypathGenericSpecificationCrasher-alxngiieimzuczddmdqiiywzdnak/Build/Intermediates.noindex/KeypathGenericSpecificationCrasher.build/Debug/KeypathGenericSpecificationCrasher.build/KeypathGenericSpecificationCrasher-project-headers.hmap -Xcc -I/Users/bradlarson/Library/Developer/Xcode/DerivedData/KeypathGenericSpecificationCrasher-alxngiieimzuczddmdqiiywzdnak/Build/Products/Debug/include -Xcc -I/Users/bradlarson/Library/Developer/Xcode/DerivedData/KeypathGenericSpecificationCrasher-alxngiieimzuczddmdqiiywzdnak/Build/Intermediates.noindex/KeypathGenericSpecificationCrasher.build/Debug/KeypathGenericSpecificationCrasher.build/DerivedSources-normal/x86_64 -Xcc -I/Users/bradlarson/Library/Developer/Xcode/DerivedData/KeypathGenericSpecificationCrasher-alxngiieimzuczddmdqiiywzdnak/Build/Intermediates.noindex/KeypathGenericSpecificationCrasher.build/Debug/KeypathGenericSpecificationCrasher.build/DerivedSources/x86_64 -Xcc -I/Users/bradlarson/Library/Developer/Xcode/DerivedData/KeypathGenericSpecificationCrasher-alxngiieimzuczddmdqiiywzdnak/Build/Intermediates.noindex/KeypathGenericSpecificationCrasher.build/Debug/KeypathGenericSpecificationCrasher.build/DerivedSources -Xcc -DDEBUG=1 -Xcc -working-directory/Users/bradlarson/Development/DifferentiableSwift/CompilerBugs/KeypathGenericSpecificationCrasher -module-name KeypathGenericSpecificationCrasher -target-sdk-version 11.3 -o /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeypathGenericSpecificationCrasher-alxngiieimzuczddmdqiiywzdnak/Build/Intermediates.noindex/KeypathGenericSpecificationCrasher.build/Debug/KeypathGenericSpecificationCrasher.build/Objects-normal/x86_64/main.o -index-store-path /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeypathGenericSpecificationCrasher-alxngiieimzuczddmdqiiywzdnak/Index/DataStore -index-system-modules
1.  Apple Swift version 5.6-dev (LLVM c8e0f2fe28693ea, Swift 68eb340a2b7690f)
2.  Compiling with the current language version
3.  While evaluating request ExecuteSILPipelineRequest(Run pipelines { Mandatory Diagnostic Passes + Enabling Optimization Passes } on SIL for KeypathGenericSpecificationCrasher)
4.  While running pass #&#8203;148 SILModuleTransform "Differentiation".
5.  While processing // differentiability witness for TestKeyPaths.readAll(from:)
sil_differentiability_witness [serialized] [reverse] [parameters 0 1] [results 0] <Root, Value where Root : Differentiable, Value == Float> @$s34KeypathGenericSpecificationCrasher12TestKeyPathsV7readAll4fromSayq_Gx_tF : $@convention(method) <Root, Value> (@in_guaranteed Root, @guaranteed TestKeyPaths<Root, Value>) -> @owned Array<Value> {
}

 on SIL function "@$s34KeypathGenericSpecificationCrasher12TestKeyPathsV7readAll4fromSayq_Gx_tF".
 for 'readAll(from:)' (at /Users/bradlarson/Development/DifferentiableSwift/CompilerBugs/KeypathGenericSpecificationCrasher/KeypathGenericSpecificationCrasher/main.swift:12:12)
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  swift-frontend           0x00000001080ab127 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 39
1  swift-frontend           0x00000001080aa345 llvm::sys::RunSignalHandlers() + 85
2  swift-frontend           0x00000001080ab760 SignalHandler(int) + 288
3  libsystem_platform.dylib 0x00007fff20367d7d _sigtramp + 29
4  libsystem_platform.dylib 0x00007fb0eb0f3808 _sigtramp + 18446743737810336424
5  libsystem_c.dylib        0x00007fff20277406 abort + 125
6  libsystem_c.dylib        0x00007fff202767d8 err + 0
7  swift-frontend           0x000000010825ef83 swift::Lowering::AbstractionPattern::initSwiftType(swift::CanGenericSignature, swift::CanType, swift::Lowering::AbstractionPattern::Kind) (.cold.3) + 35
8  swift-frontend           0x000000010385ef2a swift::Lowering::AbstractionPattern::initSwiftType(swift::CanGenericSignature, swift::CanType, swift::Lowering::AbstractionPattern::Kind) + 106
9  swift-frontend           0x0000000103379a4f (anonymous namespace)::SILTypeSubstituter::visitType(swift::CanType) + 287
10 swift-frontend           0x0000000103370844 (anonymous namespace)::SILTypeSubstituter::substSILFunctionType(swift::CanTypeWrapper<swift::SILFunctionType>, bool) + 1092
11 swift-frontend           0x000000010337003b swift::SILType::subst(swift::Lowering::TypeConverter&, llvm::function_ref<swift::Type (swift::SubstitutableType*)>, llvm::function_ref<swift::ProtocolConformanceRef (swift::CanType, swift::Type, swift::ProtocolDecl*)>, swift::CanGenericSignature, bool) const + 283
12 swift-frontend           0x0000000103360303 swift::GenericEnvironment::mapTypeIntoContext(swift::SILModule&, swift::SILType) const + 131
13 swift-frontend           0x00000001039e53ad swift::autodiff::LinearMapInfo::remapTypeInDerivative(swift::SILType) + 93
14 swift-frontend           0x00000001039e6ed2 swift::autodiff::LinearMapInfo::addLinearMapToStruct(swift::autodiff::ADContext&, swift::ApplyInst*) + 722
15 swift-frontend           0x00000001039e4dbb swift::autodiff::LinearMapInfo::generateDifferentiationDataStructures(swift::autodiff::ADContext&, swift::SILFunction*) + 939
16 swift-frontend           0x0000000103a08c50 swift::autodiff::VJPCloner::Implementation::Implementation(swift::autodiff::VJPCloner&, swift::autodiff::ADContext&, swift::SILDifferentiabilityWitness*, swift::SILFunction*, swift::autodiff::DifferentiationInvoker) + 640
17 swift-frontend           0x0000000103a0a0bf swift::autodiff::VJPCloner::VJPCloner(swift::autodiff::ADContext&, swift::SILDifferentiabilityWitness*, swift::SILFunction*, swift::autodiff::DifferentiationInvoker) + 79
18 swift-frontend           0x0000000103b32c7a (anonymous namespace)::DifferentiationTransformer::canonicalizeDifferentiabilityWitness(swift::SILDifferentiabilityWitness*, swift::autodiff::DifferentiationInvoker, swift::IsSerialized_t) + 6922
19 swift-frontend           0x0000000103b30768 (anonymous namespace)::Differentiation::run() + 1432
20 swift-frontend           0x0000000103b95011 swift::SILPassManager::runModulePass(unsigned int) + 689
21 swift-frontend           0x0000000103b9a0ea swift::SILPassManager::execute() + 634
22 swift-frontend           0x0000000103b91bf8 swift::SILPassManager::executePassPipelinePlan(swift::SILPassPipelinePlan const&) + 72
23 swift-frontend           0x0000000103b91b81 swift::ExecuteSILPipelineRequest::evaluate(swift::Evaluator&, swift::SILPipelineExecutionDescriptor) const + 65
24 swift-frontend           0x0000000103bb2b7d swift::SimpleRequest<swift::ExecuteSILPipelineRequest, std::__1::tuple<> (swift::SILPipelineExecutionDescriptor), (swift::RequestFlags)1>::evaluateRequest(swift::ExecuteSILPipelineRequest const&, swift::Evaluator&) + 29
25 swift-frontend           0x0000000103b9cc91 llvm::Expected<swift::ExecuteSILPipelineRequest::OutputType> swift::Evaluator::getResultUncached<swift::ExecuteSILPipelineRequest>(swift::ExecuteSILPipelineRequest const&) + 241
26 swift-frontend           0x0000000103b91e32 swift::executePassPipelinePlan(swift::SILModule*, swift::SILPassPipelinePlan const&, bool, swift::irgen::IRGenModule*) + 82
27 swift-frontend           0x0000000103b9fd27 swift::runSILDiagnosticPasses(swift::SILModule&) + 87
28 swift-frontend           0x000000010330b1bc swift::CompilerInstance::performSILProcessing(swift::SILModule*) + 60
29 swift-frontend           0x00000001032a407d performCompileStepsPostSILGen(swift::CompilerInstance&, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule> >, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, swift::PrimarySpecificPaths const&, int&, swift::FrontendObserver*) + 637
30 swift-frontend           0x00000001032a3695 swift::performCompileStepsPostSema(swift::CompilerInstance&, int&, swift::FrontendObserver*) + 693
31 swift-frontend           0x00000001032a5a3b swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 3803
32 swift-frontend           0x0000000103234159 swift::mainEntry(int, char const**) + 553
33 libdyld.dylib            0x00007fff2033df3d start + 1
34 libdyld.dylib            0x0000000000000047 start + 18446603339975893259
philipturner commented 2 years ago

I have narrowed down the regression further. Perhaps the differentiable function being called here is Optional.!. It originated some time between the 2021-01-27 and 2021-07-07 toolchains.

import _Differentiation

// `Root` can be constrained to `Differentiable` here, rather than in the
// extension. The crash happens either way.
struct TestKeyPaths<Root, Value>{}

// Adding `where Value == Int` causes the crash. Either removing that or
// substituting with `where Value: Differentiable` eliminates the crash.
extension TestKeyPaths where Root: Differentiable, Value == Int {
  @differentiable(reverse)
  static func readAll(from root: Root) -> Double {
    // Removing the force-unwrap eliminates the crash.
    let ignored = Root?(nil)!
    return 0
  }
}
Build output ``` (base) philipturner@m1-max-mbp bug % swiftc file.swift error: compile command failed due to signal 6 (use -v to see invocation) file.swift:13:9: warning: initialization of immutable value 'ignored' was never used; consider replacing with assignment to '_' or removing it let ignored = Root?(nil)! ~~~~^~~~~~~ _ Assertion failed: (OrigType == signature->getCanonicalTypeInContext(origType)), function initSwiftType, file AbstractionPattern.h, line 532. Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the project and the crash backtrace. Stack dump: 0. Program arguments: /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-07-07-a.xctoolchain/usr/bin/swift-frontend -frontend -c -primary-file file.swift -target arm64-apple-macosx12.0 -clang-target arm64-apple-macosx12.0 -Xllvm -aarch64-use-tbi -enable-objc-interop -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -color-diagnostics -new-driver-path /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-07-07-a.xctoolchain/usr/bin/swift-driver -resource-dir /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-07-07-a.xctoolchain/usr/lib/swift -module-name file -target-sdk-version 12.3.0 -o /var/folders/qn/86czb43d3pv03bfnxvb3x66h0000gn/T/TemporaryDirectory.3gswNE/file-1.o 1. Apple Swift version 5.5-dev (LLVM 341fda9ea021b38, Swift 753474e206d06d1) 2. 3. While evaluating request ExecuteSILPipelineRequest(Run pipelines { Mandatory Diagnostic Passes + Enabling Optimization Passes } on SIL for file.file) 4. While running pass #22 SILModuleTransform "Differentiation". 5. While processing // differentiability witness for static TestKeyPaths<>.readAll(from:) sil_differentiability_witness hidden [reverse] [parameters 0] [results 0] @$s4file12TestKeyPathsVAA16_Differentiation14DifferentiableRzSiRs_rlE7readAll4fromSdx_tFZ : $@convention(method) (@in_guaranteed Root, @thin TestKeyPaths.Type) -> Double { } on SIL function "@$s4file12TestKeyPathsVAA16_Differentiation14DifferentiableRzSiRs_rlE7readAll4fromSdx_tFZ". for 'readAll(from:)' (at file.swift:11:3) 6. While generating VJP for SIL function "@$s4file12TestKeyPathsVAA16_Differentiation14DifferentiableRzSiRs_rlE7readAll4fromSdx_tFZ". for 'readAll(from:)' (at file.swift:11:3) Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it): 0 swift-frontend 0x0000000108330914 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56 1 swift-frontend 0x000000010832fb74 llvm::sys::RunSignalHandlers() + 128 2 swift-frontend 0x0000000108330f6c SignalHandler(int) + 292 3 libsystem_platform.dylib 0x000000019cd194c4 _sigtramp + 56 4 libsystem_pthread.dylib 0x000000019cd01ee0 pthread_kill + 288 5 libsystem_c.dylib 0x000000019cc3c340 abort + 168 6 libsystem_c.dylib 0x000000019cc3b754 err + 0 7 swift-frontend 0x000000010839b890 swift::Lowering::AbstractionPattern::initSwiftType(swift::CanGenericSignature, swift::CanType, swift::Lowering::AbstractionPattern::Kind) (.cold.5) + 0 8 swift-frontend 0x0000000104596744 swift::Lowering::AbstractionPattern::initSwiftType(swift::CanGenericSignature, swift::CanType, swift::Lowering::AbstractionPattern::Kind) + 124 9 swift-frontend 0x0000000104aa5840 swift::autodiff::LinearMapInfo::getBranchingTraceEnumLoweredType(swift::SILBasicBlock*) const + 176 10 swift-frontend 0x0000000104aa4ff0 swift::autodiff::VJPCloner::Implementation::buildPredecessorEnumValue(swift::SILBuilder&, swift::SILBasicBlock*, swift::SILBasicBlock*, swift::SILValue) + 100 11 swift-frontend 0x0000000104aa4d18 swift::autodiff::VJPCloner::Implementation::createTrampolineBasicBlock(swift::TermInst*, swift::StructInst*, swift::SILBasicBlock*) + 392 12 swift-frontend 0x0000000104abb4e8 swift::autodiff::VJPCloner::Implementation::visitSwitchEnumTermInst(swift::SwitchEnumTermInst) + 256 13 swift-frontend 0x0000000104aa73b4 swift::SILCloner::visitBlocksDepthFirst(swift::SILBasicBlock*) + 1304 14 swift-frontend 0x0000000104aa6434 swift::SILCloner::cloneFunctionBody(swift::SILFunction*, swift::SILBasicBlock*, llvm::ArrayRef, bool) + 472 15 swift-frontend 0x0000000104aa5d40 swift::autodiff::VJPCloner::Implementation::run() + 1024 16 swift-frontend 0x0000000104aa64b0 swift::autodiff::VJPCloner::run() + 24 17 swift-frontend 0x0000000104b9d918 (anonymous namespace)::DifferentiationTransformer::canonicalizeDifferentiabilityWitness(swift::SILDifferentiabilityWitness*, swift::autodiff::DifferentiationInvoker, swift::IsSerialized_t) + 5144 18 swift-frontend 0x0000000104b9ba8c (anonymous namespace)::Differentiation::run() + 1088 19 swift-frontend 0x0000000104be2e18 swift::SILPassManager::runModulePass(unsigned int) + 632 20 swift-frontend 0x0000000104be951c swift::SILPassManager::execute() + 628 21 swift-frontend 0x0000000104bdfa24 swift::SILPassManager::executePassPipelinePlan(swift::SILPassPipelinePlan const&) + 68 22 swift-frontend 0x0000000104bdf9ac swift::ExecuteSILPipelineRequest::evaluate(swift::Evaluator&, swift::SILPipelineExecutionDescriptor) const + 68 23 swift-frontend 0x0000000104bfde04 swift::SimpleRequest (swift::SILPipelineExecutionDescriptor), (swift::RequestFlags)1>::evaluateRequest(swift::ExecuteSILPipelineRequest const&, swift::Evaluator&) + 28 24 swift-frontend 0x0000000104beb65c llvm::Expected swift::Evaluator::getResultUncached(swift::ExecuteSILPipelineRequest const&) + 252 25 swift-frontend 0x0000000104bdfc20 swift::executePassPipelinePlan(swift::SILModule*, swift::SILPassPipelinePlan const&, bool, swift::irgen::IRGenModule*) + 84 26 swift-frontend 0x0000000104bed7f0 swift::runSILDiagnosticPasses(swift::SILModule&) + 92 27 swift-frontend 0x00000001044c6510 swift::CompilerInstance::performSILProcessing(swift::SILModule*) + 68 28 swift-frontend 0x00000001043c36c0 performCompileStepsPostSILGen(swift::CompilerInstance&, std::__1::unique_ptr >, llvm::PointerUnion, swift::PrimarySpecificPaths const&, int&, swift::FrontendObserver*) + 620 29 swift-frontend 0x00000001043c319c bool llvm::function_ref::callback_fn(long, swift::CompilerInstance&) + 328 30 swift-frontend 0x00000001043b8d08 swift::performFrontend(llvm::ArrayRef, char const*, void*, swift::FrontendObserver*) + 4148 31 swift-frontend 0x0000000104324fcc main + 500 32 dyld 0x0000000110149088 start + 516 ```
jkshtj commented 6 months ago

Still crashes on 05/24 toolchain.