search

3MFConsortium / lib3mf

lib3mf is an implementation of the 3D Manufacturing Format file standard
http://3mf.io
BSD 2-Clause "Simplified" License
228 stars 92 forks source link

Add Integration and Performance test to lib3mf build pipeline #361

Closed gangatp closed 3 months ago

gangatp commented 4 months ago

The repositories which contain the positive and negative test cases are :

  1. https://github.com/3MFConsortium/test_suites
  2. https://github.com/3MFConsortium/3mf-samples

Integration Tests : Run these test cases in the build pipeline optionally on every build or they should be triggered manually.

Performance Test: In the same build pipeline, run integration test cases and calculate elapsed time with the latest release/previous commit binaries to create a baseline and then run integration test with the built binaries to compare the elapsed time with the baseline. Need to make sure this produces reproducible performance results and not false positives.

Create two options in the build pipeline to run integration and performance tests manually.

vijaiaeroastro commented 4 months ago

@gangatp I haven't tried running these yet. I am curious to know if the compute resources from github actions be enough for this?