search

google / promises

Promises is a modern framework that provides a synchronization construct for Swift and Objective-C.
Apache License 2.0
3.81k stars 296 forks source link

Instructions for running benchmarks #12

Closed AndrewBarba closed 6 years ago

AndrewBarba commented 6 years ago

I maintain a not-so-popular Promise library for Swift, Bluebird.swift, which I'd like to benchmark against the same set of tests. Of the benchmarked libraries, I think my library (spoiler: directly based on Bluebird.js) is actually the most similar to the API in Promises.

AndrewBarba commented 6 years ago

Never mind, really easy for me to port over. Ran once, results if interested:

Test Suite 'Selected tests' started at 2018-02-02 17:41:09.382
Test Suite 'BluebirdTests.xctest' started at 2018-02-02 17:41:09.383
Test Suite 'PromiseThenPerformanceTests' started at 2018-02-02 17:41:09.383
Test Case '-[BluebirdTests.PromiseThenPerformanceTests testDispatchAsyncOnConcurrentQueue]' started.
Total time: 0.0158751011
Test Case '-[BluebirdTests.PromiseThenPerformanceTests testDispatchAsyncOnConcurrentQueue]' passed (0.037 seconds).
Test Case '-[BluebirdTests.PromiseThenPerformanceTests testDispatchAsyncOnSerialQueue]' started.
Average time: 0.0000108280
Test Case '-[BluebirdTests.PromiseThenPerformanceTests testDispatchAsyncOnSerialQueue]' passed (0.124 seconds).
Test Case '-[BluebirdTests.PromiseThenPerformanceTests testDoubleDispatchAsyncOnSerialQueue]' started.
Average time: 0.0000119200
Test Case '-[BluebirdTests.PromiseThenPerformanceTests testDoubleDispatchAsyncOnSerialQueue]' passed (0.120 seconds).
Test Case '-[BluebirdTests.PromiseThenPerformanceTests testDoubleThenOnSerialQueue]' started.
Average time: 0.0000337990
Test Case '-[BluebirdTests.PromiseThenPerformanceTests testDoubleThenOnSerialQueue]' passed (0.339 seconds).
Test Case '-[BluebirdTests.PromiseThenPerformanceTests testThenOnConcurrentQueue]' started.
Total time: 0.1026090384
Test Case '-[BluebirdTests.PromiseThenPerformanceTests testThenOnConcurrentQueue]' passed (0.186 seconds).
Test Case '-[BluebirdTests.PromiseThenPerformanceTests testThenOnSerialQueue]' started.
Average time: 0.0000181330
Test Case '-[BluebirdTests.PromiseThenPerformanceTests testThenOnSerialQueue]' passed (0.183 seconds).
Test Case '-[BluebirdTests.PromiseThenPerformanceTests testTripleDispatchAsyncOnSerialQueue]' started.
Average time: 0.0000118210
Test Case '-[BluebirdTests.PromiseThenPerformanceTests testTripleDispatchAsyncOnSerialQueue]' passed (0.120 seconds).
Test Case '-[BluebirdTests.PromiseThenPerformanceTests testTripleThenOnSerialQueue]' started.
Average time: 0.0000432970
Test Case '-[BluebirdTests.PromiseThenPerformanceTests testTripleThenOnSerialQueue]' passed (0.434 seconds).
Test Suite 'PromiseThenPerformanceTests' passed at 2018-02-02 17:41:10.931.
     Executed 8 tests, with 0 failures (0 unexpected) in 1.544 (1.548) seconds
Test Suite 'BluebirdTests.xctest' passed at 2018-02-02 17:41:10.931.
     Executed 8 tests, with 0 failures (0 unexpected) in 1.544 (1.549) seconds
Test Suite 'Selected tests' passed at 2018-02-02 17:41:10.932.
     Executed 8 tests, with 0 failures (0 unexpected) in 1.544 (1.550) seconds
Program ended with exit code: 0
shoumikhin commented 6 years ago

Thank you for the great question, Andrew!

We ran each test individually (i.e. comment out all other tests cases in a suite first) several times on an iPhone 6S with iOS 11.2.1, and then found the average. Hope that helps.

AndrewBarba commented 6 years ago

Thanks good to know, I just ran file all at once in a simulator so definitely not comparable as posted above