search

mongodb / stitch-ios-sdk

Apache License 2.0
42 stars 25 forks source link

STITCH-2737: Quality of life improvements to swift performance testing harness #162

Closed tkaye407 closed 5 years ago

tkaye407 commented 5 years ago

This PR Accomplishes The Following:

  1. Adds run_ios_performance_tests as an evergreen task (I expect this to be controversial but eventually needs to happen / I needed to test it)
  2. Refactors the scheme so that it can run the performance tests
  3. Removes TestParams and instead takes first from the GCC_PREPROCESSOR_DEFINITIONS, then the environment variables, and then from a set of defaults defined in SyncPerformanceTestUtils.swift
  4. Defines utility functions: generateDocuments(numDocs, docSize) in SyncPerformanceTestUtils.swift to mimic the functionality Adam used in the Android SDK
  5. Refactored code in the harness to make the main function less dense
  6. Corrected the failure handling procedure for the harness

Things still remaining:

  1. I would like to dig into the disk usage in this harness as well. It's working better than the Android Harness does but I think there is something slightly off with what we are measuring because it claims we are storing roughly 100 times as much data as we should be...
  2. Discuss how we want to handle evergreen with this.