3MFConsortium / lib3mf

lib3mf is an implementation of the 3D Manufacturing Format file standard
http://3mf.io
BSD 2-Clause "Simplified" License
228 stars 92 forks source link

Integration tests #367

Closed vijaiaeroastro closed 3 months ago

vijaiaeroastro commented 3 months ago

This PR includes the integration tests to the existing build.yml script. This completes the issue #361.

It has 3 actions:

integration-tests-latest-commit
integration-tests-latest-release
integration-tests-last-two-releases

All of the integration tests dynamically choose the latest test performance suite from 3MFConsortium/test_suites. It picks the latest test suite zip file from the releases section automatically.

The latest commit action is triggered on every build. The other two have an if condition that checks if the current action is being run on a release event and only allows it in that scenario. The trigger condition can be commented/uncommented based on further discussions.

codecov[bot] commented 3 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 71.10%. Comparing base (6c92731) to head (658de28). Report is 3 commits behind head on develop.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #367 +/- ## ======================================== Coverage 71.10% 71.10% ======================================== Files 271 271 Lines 30307 30307 ======================================== Hits 21551 21551 Misses 8756 8756 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

vijaiaeroastro commented 3 months ago

@gangatp All of your requests have been incorporated. There is now an additional job that checks the last commit against the last release and an artifact is also produced with the comparison info as shown below.

image

A new environment variable is introduced in the build.yml file itself to turn integration tests on / off

image

The pipeline looks like this now

image

The last block of jobs with integration tests would not be run if its turned off. We can also have 4 separate variable for 4 different integration tests and have more fine grained control over turning them on / off.

Let me know if anymore changes are required in this.

vijaiaeroastro commented 3 months ago

@gangatp All of Cmake stuff is also now merged into this without conflicts. It can be merged into develop without issues :)

vijaiaeroastro commented 3 months ago
vijaiaeroastro commented 3 months ago

Thanks a lot for taking the time Pradeep. I have merged. Did you have conflicts with your branch ? The automated check from Github did not report any conflicts. I will check once more in develop branch.