The mgunit test suites for MMS are currently run in a single batch, using the mms_run_all_tests routine. But this is not ideal for automated testing using cron jobs. Some tests generate plots, and fail if they are run in a headless environment such as cron (even with set_plot,'z'). Some tests have runtimes on the order of hours while others just take a few minutes. Some are pure IDL and others do comparisons with Python results. Some features being tested are expected to be widely used outside the team, while other tests are using non-public data and require valid login credentials.
So it would be nice to regroup the tests into a few categories that could each be tested in an appropriate environment, with appropriate prioritization of test failures (e.g. a failed test using core features with public data should have a higher level of urgency than a test using private data for a feature that may or may not even be used anymore internally. I would suggest these categories to start:
Core routines, public data, IDL only, no plotting, short runtimes
Core routines, public data, IDL only, no plotting, longer runtimes (perhaps a single long test per category)
Core routines, public data, Python comparisons, no plotting
The mgunit test suites for MMS are currently run in a single batch, using the mms_run_all_tests routine. But this is not ideal for automated testing using cron jobs. Some tests generate plots, and fail if they are run in a headless environment such as cron (even with set_plot,'z'). Some tests have runtimes on the order of hours while others just take a few minutes. Some are pure IDL and others do comparisons with Python results. Some features being tested are expected to be widely used outside the team, while other tests are using non-public data and require valid login credentials.
So it would be nice to regroup the tests into a few categories that could each be tested in an appropriate environment, with appropriate prioritization of test failures (e.g. a failed test using core features with public data should have a higher level of urgency than a test using private data for a feature that may or may not even be used anymore internally. I would suggest these categories to start: