search

ggciag / mandyoc

MANDYOC is a finite element code written on top of the PETSc library to simulate thermo-chemical convection of the Earth's mantle
https://ggciag.github.io/mandyoc/
BSD 3-Clause "New" or "Revised" License
27 stars 5 forks source link

Test to check that the Mandyoc output result for a given model is equal to the expected result #27

Closed aguspesce closed 3 years ago

aguspesce commented 3 years ago

Create a test to check that the Mandyoc output result for the Crameri model is equal to the expected result.

It is a first approximation... I am thinking to use pytest to improve the code

Don't merge this PR without merge #31

Fix #20

aguspesce commented 3 years ago

Hi @rafaelmds. Thanks for your review and suggestions

It is a draft of my idea to test the code. All suggestions and ideas are welcome. Due to this test run in python, I am thinking to use pytest to create this kind of test because it has more tools to make the test. What do you think?

The folder order and name are symbolic, we can change it!

rafaelmds commented 3 years ago

I aggre with pytest, I think it will also make it easier to integrate with github actions

aguspesce commented 3 years ago

@rafaelmds ... Now the test is using pytest!! Can you run it in your pc? And give me your review. #

rafaelmds commented 3 years ago

WOW!

==== 22 passed in 5.38s ====

@aguspesce thanks for the effort on putting this to work! testing code is much cleaner and output formatting its very nice!

Will do my complete review soon, but I think its all to go merge!

rafaelmds commented 3 years ago

I also tested failed tests. I got (by changing denok value)

=== 7 failed, 15 passed in 4.12s ====

which I think its good result, since we can detect a small deviation in some values but not all together at same time.

aguspesce commented 3 years ago

@rafaelmds... Do you think that maybe we must add another model test?

rafaelmds commented 3 years ago

@victorsacek maybe we can add the instantaneous indentor benchmark.

aguspesce commented 3 years ago

@victorsacek, @rafaelmds and @jamisonassuncao ... Do you have other idea to test the code? From my part, I think that this test is ok! It is not automatically, but in the contribution guideline I put that it is mandatory to run the test in each code improvement.

What do you think? Is ready to merge to main?

rafaelmds commented 3 years ago

@aguspesce I was talking to @victorsacek about the "expected" results for the model. We can limit comparison to first output of results. Its enough to assert the results and also we gain in simulation time.

aguspesce commented 3 years ago

@rafaelmds and @victorsacek, The Crameri model doesn't take too much time to run and compare the two first time steps. It takes less than 1 minute. instead, I tried to make a test using a more complex model like continental rift, but it is impossible to use it because it needs 30 minutes to solve the first time step