search

AetherModel / Aether

This is the home of the Aether model of the thermosphere and ionosphere
GNU General Public License v3.0
19 stars 29 forks source link

[FEAT] Expand compilation tests #117

Open aburrell opened 1 year ago

aburrell commented 1 year ago

Is your feature request related to a problem? Please describe. Current CI only tests compilation on linux

Describe the solution you'd like Tests for

Describe alternatives you've considered Not covering all recent OS

Additional context These tests will also reduce the start up time for new users

aaronjridley commented 1 year ago

There are now a lot of tests in the test directory. Some of these could be run as standard tests that are run when pull requests happen. I don't know how to do this on github, though.

We would need to decide which tests are "important" and should be maintained long-term, and which were good for testing a feature and may not be critical.

aburrell commented 1 year ago

Can you tell me how to run the tests from the command line? If so, I can add that to the yaml. The best way is if there's a script that provides a success/failure and we just run that. At the moment we should just run all of them. Later we can prune down.

aaronjridley commented 1 year ago

In the tests directory, there are a bunch of sub-directories, each one has a test in it. Inside each of these directories is a shell script called something like "run_test.sh". You just have to run that. The PROBLEM is that there is no success or failure criteria at this point. All of the tests just make plots (assuming that aetherpy is installed in a given directory, even!), so they are not incredibly useful for automated testing - although the automated tests could just see if the code crashes or not (which is something..... just not much...)

So, I think that maybe one of the students could write something that maybe reads in the log file and does a comparison with another given log file and reports the difference between the two.

The problem with that is that we then have a ton of reference log files that will all need to be updated when something fundamental changes in the code.

aburrell commented 1 year ago

That's still a start. We definitely need something at the top level that runs the tests in each directory, though.