Open ec882e32-f2b6-4d2a-849c-98d6c7df0cfb opened 8 years ago
belkadan
commented
8 years ago I'm a little concerned that we'll lose what little coverage we have for the interpreter; right now there's a (non-buildbot) test mode that turns simple compile-and-run invocations into interpreter invocations. StdlibUnittest currently doesn't support that because it runs all tests out of process by default.
I should probably file a bug for this.
belkadan
commented
8 years ago Filed SR-1125 and SR-1126 (neither of which should block this effort).
swift-ci
commented
8 years ago Comment by Kyle Yoon (JIRA)
Is this task mostly changing
// RUN: %target-run %t/ArrayBridge > %t.txt
// RUN: FileCheck %s \< %t.txt
into
// RUN: %target-run-stdlib-swift | FileCheck %s
for the listed files?
Also, does it require you to re-build the toolchain and test every time?
Sorry for all the questions, again!
ec882e32-f2b6-4d2a-849c-98d6c7df0cfb
commented
8 years ago No need to apologize for clarifying the task! On the contrary, thanks for these follow-up questions.
The ideal end state of the listed tests would be the following:
// RUN: %target-run-simple-swiftA good example is test/1_stdlib/NSEnumeratorAPI.swift. Notice that that test does not use FileCheck at all. Instead, it uses the TestSuite construct from StdlibUnittest. This library is capable of making assertions that are not based on textual output, like FileCheck is. We want to avoid text output-based assertions.
swift-ci
commented
6 years ago Comment by Sammy Gutierrez (JIRA)
Here is the work I have for this ticket.
I was hoping someone could take a quick look over it before submitting the PR (should I ask for help in the forums instead?). I attempted to remove FileCheck from any files that I could in /test/stdlib but there were a few places where I did not know what to do:
Reflection_jit.swift - Could not get Reflection.swift tests to run with JIT (deleted this file for now)
SwiftObjectNSObject.swift - Calls SwiftObjectNSObject.m and checks for output.
belkadan
commented
6 years ago cc @airspeedswift
belkadan
commented
5 years ago Resetting assignee for all Starter Bugs not modified since 2018.
I think we lost this one since it wasn't a full PR. In the future, please feel free to make a PR even if it's still WIP.
Additional Detail from JIRA
| | | |------------------|-----------------| |Votes | 0 | |Component/s | Standard Library | |Labels | Improvement, StarterBug, Test | |Assignee | None | |Priority | Medium | md5: 5245d992633498d3acacd3655548dd7cIssue Description:
Swift's test/1_stdlib directory is meant to include tests for the Swift stdlib. Most of these are written using the testing framework designed for this purpose, StdlibUnittest. However, some use an ad-hoc system of printing strings to stdout, then checking the output via FileCheck.
The following tests should be written using StdlibUnittest (and no longer reference FileCheck):
test/1_stdlib/ArrayBridge.swift
test/1_stdlib/BitwiseOperationsType.swift
test/1_stdlib/BridgeNonVerbatim.swift
test/1_stdlib/Bridgeable.swift
test/1_stdlib/CGGeometry.swift
test/1_stdlib/Collection.swift
test/1_stdlib/CollectionOfOne.swift
test/1_stdlib/Float.swift
test/1_stdlib/FloatingPointIR.swift
test/1_stdlib/HeapBuffer.swift
test/1_stdlib/ImplicitlyUnwrappedOptional.swift
test/1_stdlib/KVO.swift
test/1_stdlib/LogicValue.swift
test/1_stdlib/Map.swift
test/1_stdlib/NSObject.swift
test/1_stdlib/NSUndoManager.swift
test/1_stdlib/NewArray.swift.gyb
test/1_stdlib/NewString.swift
test/1_stdlib/NewStringAppending.swift
test/1_stdlib/Nil.swift
test/1_stdlib/Reflection.swift
test/1_stdlib/Reflection_jit.swift
test/1_stdlib/Reflection_objc.swift
test/1_stdlib/SegmentAlignment.c
test/1_stdlib/SpriteKit.swift
test/1_stdlib/StringReallocation.swift
test/1_stdlib/TypeName.swift
test/1_stdlib/UIKit.swift
test/1_stdlib/UIViewControllerAdditions.swift
test/1_stdlib/VarArgs.swift
test/1_stdlib/Zip.swift
test/1_stdlib/alloc_rounding.swift
test/1_stdlib/integer_conversions.swift
test/1_stdlib/sort_integers.swift
test/1_stdlib/subString.swift