ufs-community / UFS_UTILS

Utilities for the NCEP models.
Other
21 stars 104 forks source link

Is there a way to mark some tests as particularly slow, and not run by default, so that we can avoid slowing down all CI workflows? #727

Open edwardhartnett opened 1 year ago

edwardhartnett commented 1 year ago

Wow, what a change in the testing landscape on the UFS_UTILS project! There is now much more test code.

I am currently working on the CI (see #719 ) and I wonder if there is a way of separating some slower tests (or slower-downloading test data files), and only running those tests on a few of the CI runs, instead of every CI run.

For example, the g2 and g2c project now have tests based on small test files in the repo, and larger test files downloaded from FTP. The CI system runs the FTP tests in only a few places, to speed tests everywhere else.

Perhaps this is not possible with UFS_UTILS - I just wanted to propose the idea to see if @GeorgeGayno-NOAA could think of anything...

GeorgeGayno-NOAA commented 1 year ago

Most of the time, each test takes only about 5 seconds or less to run. However, tests numbers 4 and 27 can occasionally take longer. Sometimes they can even time out. So, if we decide to mark some tests to not run by default, I would start with those two.

These tests also use the largest input data files, which can be slow to download from FTP.