GEOUNED-org / GEOUNED

A tool to convert CAD to CSG & CSG to CAD for Monte Carlo transport codes
European Union Public License 1.2
56 stars 30 forks source link

Adding regression tests #195

Open shimwell opened 4 months ago

shimwell commented 4 months ago

Description

I've produced CSG files for all the step files in the repo and added these to a folder in the tests folder. Now whenever we make a PR a regression test will run that compares all the CSG files converted during the existing CI tests with those in the regression test folder. This takes less than a second so we won't notice any CI time increase.

One of the common pains of regression tests is that from time to time we will find a bug and fix it which changes the CSG files in a good way. However this will fail the regression tests as the test just checks if the newly produced CSG files match the older files.

To help smooth out this annoyance I've include a script tests/update_regression_test_files.py that will remake all the regression test files.

Most of the files added in this PR are the regression test CSG files (208 files) however to help with the review the important files edited in this PR are

Note that while I was making this I had to do some messy Path manipulation to ensure each stp file in the testing/inputSTEP folder has its own folder with no naming overlaps. One thing I would like to do in the future is to rename the files in the testing/inputSTEP folder in a better way so that we don't have files with the same name even when they are in different subfolders. Also if I had the ability to specify different file names stems for the CSG files then that would also make the Paths more elegant and concise (which makes me keen on sperate export_csg methods for each MC code). So please excuse the long Path operations as this is something I can clean up in a following PR and I want to keep this one on topic.

Fixes issue

helps with issue #46

Checklist

shimwell commented 4 months ago

Looks like the files are not perfectly reproducible and I'm seeing lines in the regression test not matching the lines in the CI produced files

Errors like this are appearing, where the coefficients for the surfaces don't fully match but are very close as mentioned in this issue

  - surf 3 plane  6.4278761e-01 -7.6604444e-01  4.7675313e-14  9.4730000e+02
  ?                                                  -- ^
  + surf 3 plane  6.4278761e-01 -7.6604444e-01  4.7671611e-14  9.4730000e+02

this lack of reproducibility makes regression tests difficult so perhaps we should see if it is possible to make the files fully reproducible prior to the regression testing.

Or alternatively we can introduce some tolerance on numbers so they don't have to match perfectly

Any comments on the preferred direction would be appreciated

shimwell commented 4 months ago

I've added some extra code to the tests that will split each line up and check each token matches. if the token is a float then there is some tolerance on the checking (1e-6)