search

apple / swift-collections

Commonly used data structures for Swift
Apache License 2.0
3.55k stars 270 forks source link

Ship release 1.1.0 #345

Closed lorentey closed 3 months ago

lorentey commented 4 months ago

The last feature release of this package was also the very first stable release, and it happened more than two years ago. The following additions are long overdue:

Of these, _RopeModule will ship underscored for now, for use in swift-foundation. It is not going to be a part of the stable public API yet, and should not be used in production outside of Foundation. (I expect a public RopeModule will ship in version 1.2.0.)

All public additions have seen informal API "reviews" on the forums, although BitSet/BitArray have been extensively reworked since then, so they would probably deserve another chance for comments. (Or not -- this package is (un)fortunately not bound by any particular process for API additions, and to be frank, at this time the lack of a predictable release schedule is a far bigger problem than the problem of getting more eyes on these APIs. 😞 If we ship the wrong APIs, we'll simply deprecate and fix them in subsequent releases.)

vmanot commented 4 months ago

@lorentey thank you!

vanvoorden commented 4 months ago

@lorentey Sounds great! Are we targeting Q1 hopefully to ship? ~Do we have any RFCs up with the WIP benchmarks for TreeSet and TreeDictionary to show the O(log(n)) copy-on-write mutations?~ This is a great data structure… thanks for shipping this!

https://github.com/apple/swift-collections/blob/release/1.1/Documentation/Reviews/2022-10-31.ShareableHashedCollections/ShareableHashedCollections.md

Ahh… these were the benchmarks before ShareableSet became TreeSet?

narner commented 3 months ago

Awesome; thank you for shipping the new release!

vmanot commented 3 months ago

Hmm, don't think they shipped it yet.

vanvoorden commented 3 months ago

Any interest in shipping https://github.com/apple/swift-collections/pull/352 in 1.1.0? Anything more I can do to help?

vmanot commented 3 months ago

@lorentey what can we do to help release 1.1.0? A lot of folks have swift-collections as a dependency and we'd love to bump it up to 1.1.0.

lorentey commented 3 months ago

Release preparation work is progressing over in https://github.com/apple/swift-collections/pull/355, with full prerelease testing starting as soon as I run out of things on my checklist. I expect testing will take some amount of iteration as it tends to uncover random issues on specific compilers/platforms. The 1.1.0 tag will drop soon after testing is complete. I don't expect there will be an extended preview period on the forum, but I may post a candidate commit a day or two in advance.

(I think it won't be long now, but I can't promise any particular release date -- it depends on how many problems testing will uncover.)

lorentey commented 3 months ago

Candidate commit:

Diff: https://github.com/apple/swift-collections/compare/1.0.6...94cf62b3ba8d4bed62680a282d4c25f9c63c2efb

I'll skip Windows testing; I trust that the CMake build results on macOS & Linux indicate that Windows will pass too.

Testing progress:

macOS/AS

macOS/x86_64

Linux/Aarch64

Linux/x86_64

Windows 10/x86_64

(Note: I expect most of this matrix will remain untested in this tag.)

lorentey commented 3 months ago

Swift 5.9.2. failed to build the project on Xcode 15.2 for the watchOS simulator, due to XCTest no longer shipping i386 binaries.

ld: warning: ignoring file .../Contents/Developer/Platforms/WatchSimulator.platform/Developer/Library/Frameworks/XCTest.framework/XC
Test, file is universal (x86_64,arm64) but does not contain the i386 architecture: .../Contents/Developer/Platforms/WatchSimulator.p
latform/Developer/Library/Frameworks/XCTest.framework/XCTest
ld: warning: Could not find or use auto-linked library 'XCTestSwiftSupport'
Undefined symbols for architecture i386:
  "_$s6XCTest7XCTFail_4file4lineySS_s12StaticStringVSutF", referenced from:
      _$s23_CollectionsTestSupport13expectFailure_8trapping4file4lineySSyXK_Sbs12StaticStringVSutF in _CollectionsTestSupport.o
      _$s23_CollectionsTestSupport14_expectFailure__8trapping4file4lineySS_SSyXESbs12StaticStringVSutF05$s23_ab77Support14checkEquatable_6oracle10maxSamples
4file4lineySayxG_SbSi_SitXESiSgs12ij14VSutSQRzlFySnyR15GXEfU0_SSyXEfu_SiTf1ncnnn_n in _CollectionsTestSupport.o
      _$s23_CollectionsTestSupport14_expectFailure__8trapping4file4lineySS_SSyXESbs12StaticStringVSutF05$s23_ab77Support14checkEquatable_6oracle10maxSamples
4file4lineySayxG_SbSi_SitXESiSgs12ij14VSutSQRzlFySnyR16GXEfU0_SSyXEfu0_S2iTf1ncnnn_n in _CollectionsTestSupport.o
      _$s23_CollectionsTestSupport14_expectFailure__8trapping4file4lineySS_SSyXESbs12StaticStringVSutF05$s23_ab54Support10expectTrue__8trapping4file4lineySb
_SSyXKSbs12iJ16VSutFfA0_SSycfu_Tf1ncnnn_nTm in _CollectionsTestSupport.o
      _$s23_CollectionsTestSupport14_expectFailure__8trapping4file4lineySS_SSyXESbs12StaticStringVSutF05$s23_aB60Support12MinimalIndexV17_assertCompatible4w
ithyAC_tFSSyXEfu_Tf1ncnnn_nTm in _CollectionsTestSupport.o
      _$s23_CollectionsTestSupport14_expectFailure__8trapping4file4lineySS_SSyXESbs12StaticStringVSutF in _CollectionsTestSupport.o
      _$s23_CollectionsTestSupport12expectThrows__8trapping4file4line_yxyKXK_SSyXKSbs12StaticStringVSuys5Error_pXEtlF in _CollectionsTestSupport.o
      ...
  "_OBJC_CLASS_$_XCTestCase", referenced from:
      _$s23_CollectionsTestSupport010CollectionB4CaseCN in _CollectionsTestSupport.o
  "_OBJC_METACLASS_$_XCTestCase", referenced from:
      _OBJC_METACLASS_$__TtC23_CollectionsTestSupport18CollectionTestCase in _CollectionsTestSupport.o
  "__swift_FORCE_LOAD_$_XCTestSwiftSupport", referenced from:
      __swift_FORCE_LOAD_$_XCTestSwiftSupport_$_RopeModuleTests in Availability.o
      __swift_FORCE_LOAD_$_XCTestSwiftSupport_$_RopeModuleTests in SampleStrings.o
      __swift_FORCE_LOAD_$_XCTestSwiftSupport_$_RopeModuleTests in TestBigString.o
      __swift_FORCE_LOAD_$_XCTestSwiftSupport_$_RopeModuleTests in TestRope.o
      __swift_FORCE_LOAD_$_XCTestSwiftSupport_$__CollectionsTestSupport in _CollectionsTestSupport.o
     (maybe you meant: __swift_FORCE_LOAD_$_XCTestSwiftSupport_$_RopeModuleTests)
ld: symbol(s) not found for architecture i386

This is a problem with the generic/platform=watchOS Simulator destination specifier; it's not new and it isn't related to this project.

lorentey commented 3 months ago

A second run uncovered a trap in a benchmark case:

[20 run-benchmarks] ./Utils/run-benchmarks.sh run --max-size 128 /tmp/run-full-tests.sh.lv0rF/benchmark.results --cycles 3
  Failed in 82s. See /tmp/run-full-tests.sh.lv0rF/20.run-benchmarks.log for full console output.
    Array<Int> random insertions, reserving capacity
    Array<Int> removeLast
    Array<Int> removeFirst
    (584 more)
  Output file: /tmp/run-full-tests.sh.lv0rF/benchmark.results
  Appending to existing data (if any) for these tasks/sizes.

  Collecting data:
    1.2.4...8...16...32...64...128 -- 79.3s
    1./Utils/run-benchmarks.sh: line 11: 27359 Trace/BPT trap: 5       swift run --package-path "$srcroot" $flags benchmark "$@"
Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   benchmark                              0x10254d3a8 Swift runtime failure: Double value cannot be converted to Int because it is either infinite or NaN + 0 [inlined]
1   benchmark                              0x10254d3a8 Task.measure(size:input:options:) + 936 (Task.swift:121)
2   benchmark                              0x10254d274 static Tick.now.getter + 16 (Tick.swift:27) [inlined]
3   benchmark                              0x10254d274 static Time.since(_:) + 16 (<compiler-generated>:37) [inlined]
4   benchmark                              0x10254d274 Task.measure(size:input:options:) + 628 (Task.swift:110)
5   benchmark                              0x10251ae88 _ConcreteTask.measure(size:input:options:) + 276 (AnyTask.swift:119)
6   benchmark                              0x1025325e8 AnyTask.measure(size:input:options:) + 24 (AnyTask.swift:49) [inlined]
7   benchmark                              0x1025325e8 specialized Benchmark.measureOneCycle(tasks:sizes:options:delegate:) + 1512 (Benchmark+RunOptions.swift:40)
8   benchmark                              0x102533768 specialized Benchmark.run(options:delegate:) + 252 (Benchmark+RunOptions.swift:84)

This appears to be a problem in the benchmark environment! Huh

lorentey commented 3 months ago

The line in question is

        let remainingIterationEstimate =
          Int((remainingDuration / averageDuration).rounded(.up))

So presumably the average duration measured as zero. This is a real issue, but it's in swift-collections-benchmark, and it doesn't need to hold up the collections release.

lorentey commented 3 months ago

Ah, it's an old one: https://github.com/apple/swift-collections-benchmark/issues/12

lorentey commented 3 months ago

Development toolchains downloaded from swift.org do not allow building the package with xcodebuild. This is not a fault of the package, though.

lorentey commented 3 months ago

Tests have passed on every supported Swift version, which is a really good sign. (Architecture-specific problems may still be lurking, but working around language discrepancies was historically more difficult.)

lorentey commented 3 months ago

I'm running the Intel tests on macOS under Rosetta 2. I'll defer x86_64 tests on Linux for tomorrow, to run them on actual Intel hardware.

Once the current test jobs complete, I think I'll partially cherry pick the doc fixes in https://github.com/apple/swift-collections/pull/356 to release/1.1 -- it'll be a super late change, but the risks are really low. (I will not restart the testing process, beyond simply doing a quick single-platform pass verifying that the change does not introduce any surprise regressions.)

Beyond these, the last major item left is to run tests on arm64_32, which is (IIRC) the last remaining architecture actively supported by Swift that has 32-bit pointers.

lorentey commented 3 months ago

arm64_32 tests completed successfully -- things are still on track.

lorentey commented 3 months ago

Any interest in shipping https://github.com/apple/swift-collections/pull/352 in 1.1.0? Anything more I can do to help?

I'll add this one to the post-test batch, too!

lorentey commented 3 months ago

94cf62b3ba8d4bed62680a282d4c25f9c63c2efb is the current release candidate commit.

I am still waiting for some tests to complete, but I see no problems detected so far.

The next item on my list is going to be to draft 1.1.0 release notes.

lorentey commented 3 months ago

1.1.0 is out: https://github.com/apple/swift-collections/releases/tag/1.1.0