swiftlang / swift

The Swift Programming Language
https://swift.org
Apache License 2.0
67.57k stars 10.36k forks source link

[SR-15666] [AutoDiff] Assertion failure when differentiating through a map function involving keypaths #57945

Open BradLarson opened 2 years ago

BradLarson commented 2 years ago
Previous ID SR-15666
Radar None
Original Reporter @BradLarson
Type Bug
Additional Detail from JIRA | | | |------------------|-----------------| |Votes | 0 | |Component/s | Compiler | |Labels | Bug, AutoDiff | |Assignee | None | |Priority | Medium | md5: ae84dcea4936a75e0d58dcc65a61c26e

Issue Description:

The following simple file involving differentiation through a map function that uses keypaths triggers an assertion failure of "Assertion failed: (!NodePtr->isKnownSentinel()), function operator*, file ilist_iterator.h, line 138.":

import _Differentiation

public struct TestKeyPaths<Root, Value> {
    @noDerivative
    public var keyPaths: Set<KeyPath<Root, Value>> = []

    @differentiable(
        reverse
        where Root: Differentiable,
        Value: Differentiable
    )
    public func readAll(from root: Root) -> [Value] {
        return self.keyPaths.map { root[keyPath: $0] }
    }
}

To reproduce, place the above in a file and run `swiftc file.swift` to hit the assertion failure. This is present in current nightly snapshots and is not a recent regression.

My guess is that we're missing a diagnostic stating that the above isn't differentiable, in a similar manner to the missing diagnostic in SR-15201.

This is the full text of the assertion failure:

Assertion failed: (!NodePtr->isKnownSentinel()), function operator*, file ilist_iterator.h, line 138.
Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the project and the crash backtrace.
Stack dump:
0.  Program arguments: /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-12-23-a.xctoolchain/usr/bin/swift-frontend -frontend -c -primary-file /Users/bradlarson/Development/DifferentiableSwift/CompilerBugs/KeyPathMapCrasher/KeyPathMapCrasher/main.swift -emit-dependencies-path /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeyPathMapCrasher-aicdsywxxsfyfvaczyaewzrypmag/Build/Intermediates.noindex/KeyPathMapCrasher.build/Debug/KeyPathMapCrasher.build/Objects-normal/x86_64/main.d -emit-reference-dependencies-path /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeyPathMapCrasher-aicdsywxxsfyfvaczyaewzrypmag/Build/Intermediates.noindex/KeyPathMapCrasher.build/Debug/KeyPathMapCrasher.build/Objects-normal/x86_64/main.swiftdeps -serialize-diagnostics-path /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeyPathMapCrasher-aicdsywxxsfyfvaczyaewzrypmag/Build/Intermediates.noindex/KeyPathMapCrasher.build/Debug/KeyPathMapCrasher.build/Objects-normal/x86_64/main.dia -target x86_64-apple-macos11.3 -enable-objc-interop -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -I /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeyPathMapCrasher-aicdsywxxsfyfvaczyaewzrypmag/Build/Products/Debug -F /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeyPathMapCrasher-aicdsywxxsfyfvaczyaewzrypmag/Build/Products/Debug -enable-testing -g -module-cache-path /Users/bradlarson/Library/Developer/Xcode/DerivedData/ModuleCache.noindex -swift-version 5 -enforce-exclusivity=checked -Onone -D DEBUG -new-driver-path /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-12-23-a.xctoolchain/usr/bin/swift-driver -serialize-debugging-options -resource-dir /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-12-23-a.xctoolchain/usr/lib/swift -enable-anonymous-context-mangled-names -Xcc -I/Users/bradlarson/Library/Developer/Xcode/DerivedData/KeyPathMapCrasher-aicdsywxxsfyfvaczyaewzrypmag/Build/Intermediates.noindex/KeyPathMapCrasher.build/Debug/KeyPathMapCrasher.build/swift-overrides.hmap -Xcc -iquote -Xcc /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeyPathMapCrasher-aicdsywxxsfyfvaczyaewzrypmag/Build/Intermediates.noindex/KeyPathMapCrasher.build/Debug/KeyPathMapCrasher.build/KeyPathMapCrasher-generated-files.hmap -Xcc -I/Users/bradlarson/Library/Developer/Xcode/DerivedData/KeyPathMapCrasher-aicdsywxxsfyfvaczyaewzrypmag/Build/Intermediates.noindex/KeyPathMapCrasher.build/Debug/KeyPathMapCrasher.build/KeyPathMapCrasher-own-target-headers.hmap -Xcc -I/Users/bradlarson/Library/Developer/Xcode/DerivedData/KeyPathMapCrasher-aicdsywxxsfyfvaczyaewzrypmag/Build/Intermediates.noindex/KeyPathMapCrasher.build/Debug/KeyPathMapCrasher.build/KeyPathMapCrasher-all-target-headers.hmap -Xcc -iquote -Xcc /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeyPathMapCrasher-aicdsywxxsfyfvaczyaewzrypmag/Build/Intermediates.noindex/KeyPathMapCrasher.build/Debug/KeyPathMapCrasher.build/KeyPathMapCrasher-project-headers.hmap -Xcc -I/Users/bradlarson/Library/Developer/Xcode/DerivedData/KeyPathMapCrasher-aicdsywxxsfyfvaczyaewzrypmag/Build/Products/Debug/include -Xcc -I/Users/bradlarson/Library/Developer/Xcode/DerivedData/KeyPathMapCrasher-aicdsywxxsfyfvaczyaewzrypmag/Build/Intermediates.noindex/KeyPathMapCrasher.build/Debug/KeyPathMapCrasher.build/DerivedSources-normal/x86_64 -Xcc -I/Users/bradlarson/Library/Developer/Xcode/DerivedData/KeyPathMapCrasher-aicdsywxxsfyfvaczyaewzrypmag/Build/Intermediates.noindex/KeyPathMapCrasher.build/Debug/KeyPathMapCrasher.build/DerivedSources/x86_64 -Xcc -I/Users/bradlarson/Library/Developer/Xcode/DerivedData/KeyPathMapCrasher-aicdsywxxsfyfvaczyaewzrypmag/Build/Intermediates.noindex/KeyPathMapCrasher.build/Debug/KeyPathMapCrasher.build/DerivedSources -Xcc -DDEBUG=1 -Xcc -working-directory/Users/bradlarson/Development/DifferentiableSwift/CompilerBugs/KeyPathMapCrasher -module-name KeyPathMapCrasher -target-sdk-version 11.3 -o /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeyPathMapCrasher-aicdsywxxsfyfvaczyaewzrypmag/Build/Intermediates.noindex/KeyPathMapCrasher.build/Debug/KeyPathMapCrasher.build/Objects-normal/x86_64/main.o -index-store-path /Users/bradlarson/Library/Developer/Xcode/DerivedData/KeyPathMapCrasher-aicdsywxxsfyfvaczyaewzrypmag/Index/DataStore -index-system-modules
1.  Apple Swift version 5.6-dev (LLVM c8e0f2fe28693ea, Swift 68eb340a2b7690f)
2.  Compiling with the current language version
3.  While evaluating request ExecuteSILPipelineRequest(Run pipelines { Mandatory Diagnostic Passes + Enabling Optimization Passes } on SIL for KeyPathMapCrasher)
4.  While running pass #&#8203;70 SILModuleTransform "Differentiation".
5.  While processing // differentiability witness for TestKeyPaths.readAll(from:)
sil_differentiability_witness [serialized] [reverse] [parameters 0] [results 0] <Root, Value where Root : Differentiable, Value : Differentiable> @$s17KeyPathMapCrasher04TestA5PathsV7readAll4fromSayq_Gx_tF : $@convention(method) <Root, Value> (@in_guaranteed Root, @guaranteed TestKeyPaths<Root, Value>) -> @owned Array<Value> {
}

 on SIL function "@$s17KeyPathMapCrasher04TestA5PathsV7readAll4fromSayq_Gx_tF".
 for 'readAll(from:)' (at /Users/bradlarson/Development/DifferentiableSwift/CompilerBugs/KeyPathMapCrasher/KeyPathMapCrasher/main.swift:12:12)
6.  While generating VJP for SIL function "@$s17KeyPathMapCrasher04TestA5PathsV7readAll4fromSayq_Gx_tF".
 for 'readAll(from:)' (at /Users/bradlarson/Development/DifferentiableSwift/CompilerBugs/KeyPathMapCrasher/KeyPathMapCrasher/main.swift:12:12)
7.  While generating pullback for SIL function "@$s17KeyPathMapCrasher04TestA5PathsV7readAll4fromSayq_Gx_tF".
 for 'readAll(from:)' (at /Users/bradlarson/Development/DifferentiableSwift/CompilerBugs/KeyPathMapCrasher/KeyPathMapCrasher/main.swift:12:12)
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  swift-frontend           0x0000000107020127 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 39
1  swift-frontend           0x000000010701f345 llvm::sys::RunSignalHandlers() + 85
2  swift-frontend           0x0000000107020760 SignalHandler(int) + 288
3  libsystem_platform.dylib 0x00007fff20367d7d _sigtramp + 29
4  libsystem_platform.dylib 0x0000000000006040 _sigtramp + 18446603339975746272
5  libsystem_c.dylib        0x00007fff20277406 abort + 125
6  libsystem_c.dylib        0x00007fff202767d8 err + 0
7  swift-frontend           0x000000010725ba43 swift::autodiff::PullbackCloner::Implementation::run() (.cold.20) + 35
8  swift-frontend           0x0000000102961b95 swift::autodiff::PullbackCloner::Implementation::run() + 10213
9  swift-frontend           0x000000010295f382 swift::autodiff::PullbackCloner::run() + 18
10 swift-frontend           0x0000000102980818 swift::autodiff::VJPCloner::Implementation::run() + 1528
11 swift-frontend           0x0000000102980ed2 swift::autodiff::VJPCloner::run() + 18
12 swift-frontend           0x0000000102aa7c82 (anonymous namespace)::DifferentiationTransformer::canonicalizeDifferentiabilityWitness(swift::SILDifferentiabilityWitness*, swift::autodiff::DifferentiationInvoker, swift::IsSerialized_t) + 6930
13 swift-frontend           0x0000000102aa5768 (anonymous namespace)::Differentiation::run() + 1432
14 swift-frontend           0x0000000102b0a011 swift::SILPassManager::runModulePass(unsigned int) + 689
15 swift-frontend           0x0000000102b0f0ea swift::SILPassManager::execute() + 634
16 swift-frontend           0x0000000102b06bf8 swift::SILPassManager::executePassPipelinePlan(swift::SILPassPipelinePlan const&) + 72
17 swift-frontend           0x0000000102b06b81 swift::ExecuteSILPipelineRequest::evaluate(swift::Evaluator&, swift::SILPipelineExecutionDescriptor) const + 65
18 swift-frontend           0x0000000102b27b7d swift::SimpleRequest<swift::ExecuteSILPipelineRequest, std::__1::tuple<> (swift::SILPipelineExecutionDescriptor), (swift::RequestFlags)1>::evaluateRequest(swift::ExecuteSILPipelineRequest const&, swift::Evaluator&) + 29
19 swift-frontend           0x0000000102b11c91 llvm::Expected<swift::ExecuteSILPipelineRequest::OutputType> swift::Evaluator::getResultUncached<swift::ExecuteSILPipelineRequest>(swift::ExecuteSILPipelineRequest const&) + 241
20 swift-frontend           0x0000000102b06e32 swift::executePassPipelinePlan(swift::SILModule*, swift::SILPassPipelinePlan const&, bool, swift::irgen::IRGenModule*) + 82
21 swift-frontend           0x0000000102b14d27 swift::runSILDiagnosticPasses(swift::SILModule&) + 87
22 swift-frontend           0x00000001022801bc swift::CompilerInstance::performSILProcessing(swift::SILModule*) + 60
23 swift-frontend           0x000000010221907d performCompileStepsPostSILGen(swift::CompilerInstance&, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule> >, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, swift::PrimarySpecificPaths const&, int&, swift::FrontendObserver*) + 637
24 swift-frontend           0x0000000102218695 swift::performCompileStepsPostSema(swift::CompilerInstance&, int&, swift::FrontendObserver*) + 693
25 swift-frontend           0x000000010221aa3b swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 3803
26 swift-frontend           0x00000001021a9159 swift::mainEntry(int, char const**) + 553
27 libdyld.dylib            0x00007fff2033df3d start + 1
28 libdyld.dylib            0x0000000000000047 start + 18446603339975893259
philipturner commented 2 years ago

This reproducer can be narrowed down further. I don't think it's about whether the function is differentiable, because the reproducer works with [Float]. In that case, the function would be differentiable, but it would still crash.

import _Differentiation

@differentiable(reverse)
func readAll(from root: Float) -> [Float] {
  // [Int] can be any array type. It can also be initialized outside of the
  // function.
  return [Int]().map { _ in root }
}

The crash appeared between the 2021-01-27 and 2021-04-18 snapshots. On 2021-01-27, there seems to be a pre-existing bug that triggered before this one. To reproduce it on that toolchain, you must change @differentiable(reverse) to @differentiable.

Crash on 2021-04-18 toolchain ``` (base) philipturner@m1-max-mbp bug % swiftc file.swift error: compile command failed due to signal 6 (use -v to see invocation) Assertion failed: (!NodePtr->isKnownSentinel()), function operator*, file /Users/buildnode/jenkins/workspace/oss-swift-package-macos/llvm-project/llvm/include/llvm/ADT/ilist_iterator.h, line 138. Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the project and the crash backtrace. Stack dump: 0. Program arguments: /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-04-18-a.xctoolchain/usr/bin/swift-frontend -frontend -c -primary-file file.swift -target x86_64-apple-macosx12.0 -enable-objc-interop -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -color-diagnostics -new-driver-path /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-04-18-a.xctoolchain/usr/bin/swift-driver -resource-dir /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-04-18-a.xctoolchain/usr/lib/swift -module-name file -target-sdk-version 12.3.0 -o /var/folders/qn/86czb43d3pv03bfnxvb3x66h0000gn/T/TemporaryDirectory.dciRaZ/file-1.o 1. Apple Swift version 5.5-dev (LLVM fb5a91b4e892ed9, Swift f17142883421ec0) 2. While evaluating request ExecuteSILPipelineRequest(Run pipelines { Mandatory Diagnostic Passes + Enabling Optimization Passes } on SIL for file.file) 3. While running pass #31 SILModuleTransform "Differentiation". 4. While processing // differentiability witness for readAll(from:) sil_differentiability_witness hidden [reverse] [parameters 0] [results 0] @$s4file7readAll4fromSaySfGSf_tF : $@convention(thin) (Float) -> @owned Array { } on SIL function "@$s4file7readAll4fromSaySfGSf_tF". for 'readAll(from:)' (at file.swift:4:1) 5. While generating VJP for SIL function "@$s4file7readAll4fromSaySfGSf_tF". for 'readAll(from:)' (at file.swift:4:1) 6. While generating pullback for SIL function "@$s4file7readAll4fromSaySfGSf_tF". for 'readAll(from:)' (at file.swift:4:1) Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it): 0 swift-frontend 0x0000000109a38e87 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 39 1 swift-frontend 0x0000000109a38045 llvm::sys::RunSignalHandlers() + 85 2 swift-frontend 0x0000000109a394a6 SignalHandler(int) + 262 3 libsystem_platform.dylib 0x00007ff817e7edfd _sigtramp + 29 4 libsystem_platform.dylib 0x000000031ad0db00 _sigtramp + 18446603383514656032 5 libsystem_c.dylib 0x00007ff817db4d24 abort + 123 6 libsystem_c.dylib 0x00007ff817db40cb err + 0 7 swift-frontend 0x0000000109cb9e23 swift::autodiff::PullbackCloner::Implementation::run() (.cold.19) + 35 8 swift-frontend 0x000000010581113c swift::autodiff::PullbackCloner::Implementation::run() + 9724 9 swift-frontend 0x000000010580eb12 swift::autodiff::PullbackCloner::run() + 18 10 swift-frontend 0x000000010583041d swift::autodiff::VJPCloner::Implementation::run() + 1597 11 swift-frontend 0x0000000105830b12 swift::autodiff::VJPCloner::run() + 18 12 swift-frontend 0x0000000105967564 (anonymous namespace)::DifferentiationTransformer::canonicalizeDifferentiabilityWitness(swift::SILDifferentiabilityWitness*, swift::autodiff::DifferentiationInvoker, swift::IsSerialized_t) + 6804 13 swift-frontend 0x0000000105965059 (anonymous namespace)::Differentiation::run() + 1705 14 swift-frontend 0x00000001059b6f8e swift::SILPassManager::runModulePass(unsigned int) + 702 15 swift-frontend 0x00000001059bd4ca swift::SILPassManager::execute() + 666 16 swift-frontend 0x00000001059b37e8 swift::SILPassManager::executePassPipelinePlan(swift::SILPassPipelinePlan const&) + 72 17 swift-frontend 0x00000001059b3771 swift::ExecuteSILPipelineRequest::evaluate(swift::Evaluator&, swift::SILPipelineExecutionDescriptor) const + 65 18 swift-frontend 0x00000001059d85dd swift::SimpleRequest (swift::SILPipelineExecutionDescriptor), (swift::RequestFlags)1>::evaluateRequest(swift::ExecuteSILPipelineRequest const&, swift::Evaluator&) + 29 19 swift-frontend 0x00000001059bfcf0 llvm::Expected swift::Evaluator::getResultUncached(swift::ExecuteSILPipelineRequest const&) + 240 20 swift-frontend 0x00000001059b3a22 swift::executePassPipelinePlan(swift::SILModule*, swift::SILPassPipelinePlan const&, bool, swift::irgen::IRGenModule*) + 82 21 swift-frontend 0x00000001059c2047 swift::runSILDiagnosticPasses(swift::SILModule&) + 87 22 swift-frontend 0x000000010518156c swift::CompilerInstance::performSILProcessing(swift::SILModule*) + 60 23 swift-frontend 0x00000001050601a0 performCompileStepsPostSILGen(swift::CompilerInstance&, std::__1::unique_ptr >, llvm::PointerUnion, swift::PrimarySpecificPaths const&, int&, swift::FrontendObserver*) + 640 24 swift-frontend 0x000000010505fb9d performCompileStepsPostSema(swift::CompilerInstance&, int&, swift::FrontendObserver*) + 365 25 swift-frontend 0x0000000105054538 swift::performFrontend(llvm::ArrayRef, char const*, void*, swift::FrontendObserver*) + 5016 26 swift-frontend 0x0000000104fa4386 main + 566 27 dyld 0x0000000211cc151e start + 462 28 dyld 0x0000000000000005 start + 18446744064821030069 ```
Crash on 2021-01-27 toolchain ``` (base) philipturner@m1-max-mbp bug % swiftc file.swift Begin Error in Function: '$s4file7readAll4fromSaySfGSf_tFTJpSpSr' Error! Found a leaked owned value that was never consumed. Value: %3 = copy_value %0 : $Array.DifferentiableView End Error in Function: '$s4file7readAll4fromSaySfGSf_tFTJpSpSr' Found ownership error?! triggering standard assertion failure routine UNREACHABLE executed at /Users/buildnode/jenkins/workspace/oss-swift-package-macos/swift/lib/SIL/Verifier/LinearLifetimeCheckerPrivate.h:211! Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the project and the crash backtrace. Stack dump: 0. Program arguments: /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-01-27-a.xctoolchain/usr/bin/swift-frontend -frontend -c -primary-file file.swift -target arm64-apple-darwin21.4.0 -Xllvm -aarch64-use-tbi -enable-objc-interop -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -color-diagnostics -target-sdk-version 12.3 -module-name file -o /var/folders/qn/86czb43d3pv03bfnxvb3x66h0000gn/T/file-16ac19.o 1. Apple Swift version 5.4-dev (LLVM e2976fe639d1f50, Swift f9be289b9e04b3d) 2. While evaluating request ExecuteSILPipelineRequest(Run pipelines { Mandatory Diagnostic Passes + Enabling Optimization Passes } on SIL for file.file) 3. While running pass #31 SILModuleTransform "Differentiation". 4. While processing // differentiability witness for readAll(from:) sil_differentiability_witness hidden [parameters 0] [results 0] @$s4file7readAll4fromSaySfGSf_tF : $@convention(thin) (Float) -> @owned Array { } on SIL function "@$s4file7readAll4fromSaySfGSf_tF". for 'readAll(from:)' (at file.swift:4:1) 5. While generating VJP for SIL function "@$s4file7readAll4fromSaySfGSf_tF". for 'readAll(from:)' (at file.swift:4:1) 6. While verifying SIL function "@$s4file7readAll4fromSaySfGSf_tFTJpSpSr". for 'readAll(from:)' (at file.swift:4:1) 0 swift-frontend 0x00000001046039fc llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 52 1 swift-frontend 0x0000000104602c5c llvm::sys::RunSignalHandlers() + 128 2 swift-frontend 0x0000000104603fe4 SignalHandler(int) + 292 3 libsystem_platform.dylib 0x000000019cd194c4 _sigtramp + 56 4 libsystem_pthread.dylib 0x000000019cd01ee0 pthread_kill + 288 5 libsystem_c.dylib 0x000000019cc3c340 abort + 168 6 swift-frontend 0x00000001045831d8 llvm::install_out_of_memory_new_handler() + 0 7 swift-frontend 0x000000010131ed44 swift::LinearLifetimeChecker::ErrorBuilder::handleError(llvm::function_ref&&, bool) const + 612 8 swift-frontend 0x000000010131e208 swift::SILValueOwnershipChecker::checkValueWithoutLifetimeEndingUses(llvm::ArrayRef) + 500 9 swift-frontend 0x000000010131cd88 swift::SILValueOwnershipChecker::checkUses() + 248 10 swift-frontend 0x000000010131cab4 swift::SILValueOwnershipChecker::check() + 196 11 swift-frontend 0x000000010131e7e8 verifySILValueHelper(swift::SILFunction const*, swift::SILValue, swift::LinearLifetimeChecker::ErrorBuilder&, swift::DeadEndBlocks*, swift::ReborrowVerifier&) + 132 12 swift-frontend 0x000000010131e6d0 swift::SILValue::verifyOwnership(swift::DeadEndBlocks*) const + 256 13 swift-frontend 0x000000010133c1e0 (anonymous namespace)::SILVerifier::checkValueBaseOwnership(swift::ValueBase*) + 168 14 swift-frontend 0x000000010133dc24 (anonymous namespace)::SILVerifier::visitSILInstruction(swift::SILInstruction*) + 3140 15 swift-frontend 0x000000010132f810 swift::SILInstructionVisitor<(anonymous namespace)::SILVerifier, void>::visit(swift::SILInstruction*) + 28704 16 swift-frontend 0x0000000101327bf4 (anonymous namespace)::SILVerifier::visitSILBasicBlock(swift::SILBasicBlock*) + 1176 17 swift-frontend 0x00000001013263d0 (anonymous namespace)::SILVerifier::visitSILFunction(swift::SILFunction*) + 7844 18 swift-frontend 0x00000001013204f4 swift::SILFunction::verify(bool) const + 76 19 swift-frontend 0x0000000100dc9bf8 swift::autodiff::PullbackCloner::run() + 52 20 swift-frontend 0x0000000100de5df8 swift::autodiff::VJPCloner::Implementation::run() + 1268 21 swift-frontend 0x0000000100de63d4 swift::autodiff::VJPCloner::run() + 24 22 swift-frontend 0x0000000100eda58c (anonymous namespace)::DifferentiationTransformer::canonicalizeDifferentiabilityWitness(swift::SILDifferentiabilityWitness*, swift::autodiff::DifferentiationInvoker, swift::IsSerialized_t) + 5484 23 swift-frontend 0x0000000100ed863c (anonymous namespace)::Differentiation::run() + 940 24 swift-frontend 0x0000000100f1c7d4 swift::SILPassManager::runModulePass(unsigned int) + 516 25 swift-frontend 0x0000000100f21820 swift::SILPassManager::execute() + 628 26 swift-frontend 0x0000000100f19938 swift::SILPassManager::executePassPipelinePlan(swift::SILPassPipelinePlan const&) + 68 27 swift-frontend 0x0000000100f198dc swift::ExecuteSILPipelineRequest::evaluate(swift::Evaluator&, swift::SILPipelineExecutionDescriptor) const + 52 28 swift-frontend 0x0000000100f382c0 swift::SimpleRequest (swift::SILPipelineExecutionDescriptor), (swift::RequestFlags)1>::evaluateRequest(swift::ExecuteSILPipelineRequest const&, swift::Evaluator&) + 28 29 swift-frontend 0x0000000100f236f8 llvm::Expected swift::Evaluator::getResultUncached(swift::ExecuteSILPipelineRequest const&) + 252 30 swift-frontend 0x0000000100f19b3c swift::executePassPipelinePlan(swift::SILModule*, swift::SILPassPipelinePlan const&, bool, swift::irgen::IRGenModule*) + 84 31 swift-frontend 0x0000000100f251f4 swift::runSILDiagnosticPasses(swift::SILModule&) + 92 32 swift-frontend 0x0000000100a40028 swift::CompilerInstance::performSILProcessing(swift::SILModule*) + 68 33 swift-frontend 0x000000010094e798 performCompileStepsPostSILGen(swift::CompilerInstance&, std::__1::unique_ptr >, llvm::PointerUnion, swift::PrimarySpecificPaths const&, int&, swift::FrontendObserver*) + 716 34 swift-frontend 0x000000010094e24c bool llvm::function_ref::callback_fn(long, swift::CompilerInstance&) + 312 35 swift-frontend 0x00000001009431a0 swift::performFrontend(llvm::ArrayRef, char const*, void*, swift::FrontendObserver*) + 4292 36 swift-frontend 0x00000001008f0c88 main + 784 37 dyld 0x000000010bdad088 start + 516 :0: error: unable to execute command: Abort trap: 6 :0: error: compile command failed due to signal 6 (use -v to see invocation) ```

On the 2020-07-22 toolchain, it fails gracefully when using [Int](), saying it can't be differentiated because of unsupported control flow. Changing to [Float]() doesn't eliminate the error, so maybe this reproducer is non-differentiable in all forms. Thus, it could be about emitting a proper diagnostic for non-differentiable functions.

With [Int]() ``` (base) philipturner@m1-max-mbp bug % swiftc file.swift file.swift:3:2: error: function is not differentiable @differentiable ~^~~~~~~~~~~~~~ file.swift:4:6: note: when differentiating this function definition func readAll(from root: Float) -> [Float] { ^ file.swift:7:18: note: cannot differentiate unsupported control flow return [Int]().map { _ in root } ^ ```
With [Float]() ``` (base) philipturner@m1-max-mbp bug % swiftc file.swift file.swift:3:2: error: function is not differentiable @differentiable ~^~~~~~~~~~~~~~ file.swift:4:6: note: when differentiating this function definition func readAll(from root: Float) -> [Float] { ^ file.swift:7:18: note: cannot differentiate unsupported control flow return [Float]().map { _ in root } ^ ```
jkshtj commented 6 months ago

Still causes a crash on the 05/24 toolchain. Stack dump looks different.

Assertion failed: (index >= (getNumIndirectSILResults() + getNumIndirectSILErrorResults()) && index <= getNumSILArguments()), function getParamInfoForSILArg, file SILFunctionConventions.h, line 444.
Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the crash backtrace.
Stack dump:
0.  Program arguments: /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-05-01-a.xctoolchain/usr/bin/swift-frontend -frontend -c -primary-file /Users/kshitij/workspace/scratch/test.swift -target arm64-apple-macosx13.0 -Xllvm -aarch64-use-tbi -enable-objc-interop -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk -color-diagnostics -O -empty-abi-descriptor -Xcc -working-directory -Xcc /Users/kshitij/workspace/scratch -resource-dir /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-05-01-a.xctoolchain/usr/lib/swift -module-name test -target-sdk-version 14.2 -target-sdk-name macosx14.2 -external-plugin-path /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib/swift/host/plugins#/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/bin/swift-plugin-server -external-plugin-path /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/local/lib/swift/host/plugins#/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/bin/swift-plugin-server -plugin-path /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-05-01-a.xctoolchain/usr/lib/swift/host/plugins -plugin-path /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-05-01-a.xctoolchain/usr/local/lib/swift/host/plugins -enable-default-cmo -o /var/folders/5h/9l25ljkd47qct1zm4g4_dxz00000gq/T/TemporaryDirectory.0Bb8Yc/test-1.o
1.  Apple Swift version 6.0-dev (LLVM b66077aefd3be08, Swift 84d36181a762913)
2.  Compiling with effective version 5.10
3.  While evaluating request ExecuteSILPipelineRequest(Run pipelines { Mandatory Diagnostic Passes + Enabling Optimization Passes } on SIL for test)
4.  While running pass #145 SILModuleTransform "Differentiation".
5.  While processing // differentiability witness for TestKeyPaths.readAll(from:)
sil_differentiability_witness [serialized] [reverse] [parameters 0] [results 0] <Root, Value where Root : Differentiable, Value : Differentiable> @$s4test12TestKeyPathsV7readAll4fromSayq_Gx_tF : $@convention(method) <Root, Value> (@in_guaranteed Root, @guaranteed TestKeyPaths<Root, Value>) -> @owned Array<Value> {
}

 on SIL function "@$s4test12TestKeyPathsV7readAll4fromSayq_Gx_tF".
 for 'readAll(from:)' (at /Users/kshitij/workspace/scratch/test.swift:12:12)
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  swift-frontend           0x0000000109d9d600 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56
1  swift-frontend           0x0000000109d9bdcc llvm::sys::RunSignalHandlers() + 112
2  swift-frontend           0x0000000109d9dc48 SignalHandler(int) + 304
3  libsystem_platform.dylib 0x00000001a3286a24 _sigtramp + 56
4  libsystem_pthread.dylib  0x00000001a3257c28 pthread_kill + 288
5  libsystem_c.dylib        0x00000001a3165ae8 abort + 180
6  libsystem_c.dylib        0x00000001a3164e44 err + 0
7  swift-frontend           0x0000000109edb124 swift::SILFunctionConventions::getParamInfoForSILArg(unsigned int) const (.cold.3) + 0
8  swift-frontend           0x0000000105030b24 swift::SILFunctionConventions::getParamInfoForSILArg(unsigned int) const + 180
9  swift-frontend           0x00000001050f7c98 swift::autodiff::LinearMapInfo::getLinearMapType(swift::autodiff::ADContext&, swift::FullApplySite) + 476
10 swift-frontend           0x00000001050f6770 swift::autodiff::LinearMapInfo::generateDifferentiationDataStructures(swift::autodiff::ADContext&, swift::SILFunction*) + 1008
11 swift-frontend           0x00000001050f6eec swift::autodiff::LinearMapInfo::LinearMapInfo(swift::autodiff::ADContext&, swift::AutoDiffLinearMapKind, swift::SILFunction*, swift::SILFunction*, swift::AutoDiffConfig const&, swift::DifferentiableActivityInfo const&, swift::SILLoopInfo*) + 148
12 swift-frontend           0x0000000105117de4 swift::autodiff::VJPCloner::Implementation::Implementation(swift::autodiff::VJPCloner&, swift::autodiff::ADContext&, swift::SILDifferentiabilityWitness*, swift::SILFunction*, swift::autodiff::DifferentiationInvoker) + 616
13 swift-frontend           0x0000000105118ea0 swift::autodiff::VJPCloner::VJPCloner(swift::autodiff::ADContext&, swift::SILDifferentiabilityWitness*, swift::SILFunction*, swift::autodiff::DifferentiationInvoker) + 92
14 swift-frontend           0x00000001052752a4 (anonymous namespace)::DifferentiationTransformer::canonicalizeDifferentiabilityWitness(swift::SILDifferentiabilityWitness*, swift::autodiff::DifferentiationInvoker, swift::IsSerialized_t) + 6020
15 swift-frontend           0x0000000105273130 (anonymous namespace)::Differentiation::run() + 1060
16 swift-frontend           0x0000000105316a80 swift::SILPassManager::runModulePass(unsigned int) + 856
17 swift-frontend           0x0000000105318e40 swift::SILPassManager::execute() + 624
18 swift-frontend           0x00000001053136f8 swift::SILPassManager::executePassPipelinePlan(swift::SILPassPipelinePlan const&) + 72
19 swift-frontend           0x0000000105313678 swift::ExecuteSILPipelineRequest::evaluate(swift::Evaluator&, swift::SILPipelineExecutionDescriptor) const + 68
20 swift-frontend           0x000000010534f07c swift::SimpleRequest<swift::ExecuteSILPipelineRequest, std::__1::tuple<> (swift::SILPipelineExecutionDescriptor), (swift::RequestFlags)1>::evaluateRequest(swift::ExecuteSILPipelineRequest const&, swift::Evaluator&) + 28
21 swift-frontend           0x000000010532f2e4 swift::ExecuteSILPipelineRequest::OutputType swift::Evaluator::getResultUncached<swift::ExecuteSILPipelineRequest, swift::ExecuteSILPipelineRequest::OutputType swift::evaluateOrFatal<swift::ExecuteSILPipelineRequest>(swift::Evaluator&, swift::ExecuteSILPipelineRequest)::'lambda'()>(swift::ExecuteSILPipelineRequest const&, swift::ExecuteSILPipelineRequest::OutputType swift::evaluateOrFatal<swift::ExecuteSILPipelineRequest>(swift::Evaluator&, swift::ExecuteSILPipelineRequest)::'lambda'()) + 204
22 swift-frontend           0x00000001053138d4 swift::executePassPipelinePlan(swift::SILModule*, swift::SILPassPipelinePlan const&, bool, swift::irgen::IRGenModule*) + 64
23 swift-frontend           0x000000010533186c swift::runSILDiagnosticPasses(swift::SILModule&) + 192
24 swift-frontend           0x0000000104b73fa8 swift::CompilerInstance::performSILProcessing(swift::SILModule*) + 68
25 swift-frontend           0x0000000104938b44 performCompileStepsPostSILGen(swift::CompilerInstance&, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule>>, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, swift::PrimarySpecificPaths const&, int&, swift::FrontendObserver*) + 796
26 swift-frontend           0x0000000104938258 swift::performCompileStepsPostSema(swift::CompilerInstance&, int&, swift::FrontendObserver*) + 612
27 swift-frontend           0x00000001049494cc withSemanticAnalysis(swift::CompilerInstance&, swift::FrontendObserver*, llvm::function_ref<bool (swift::CompilerInstance&)>, bool) + 160
28 swift-frontend           0x000000010493a868 performCompile(swift::CompilerInstance&, int&, swift::FrontendObserver*) + 708
29 swift-frontend           0x00000001049397d4 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 2368
30 swift-frontend           0x00000001047549cc swift::mainEntry(int, char const**) + 3096
31 dyld                     0x00000001a2efff28 start + 2236