RFC: Fix Unit Tests

[simmsa]

Status

• Request for Comment
  • This pull request is open for comments and feedback. Please review the changes and provide any necessary input.

Summary of Changes

• Continuation of ebbflood branch
• Fix GitHub actions/workflow
  • Linux, macOS, and Windows Units are performing nominally
• Fix/update failing unit tests where possible
  • 100% (151 / 151) of test cases are passing
• Improve code style
• Update toolbox to version 0.4.1

Detail of Changes

This PR

Actions/Workflow

• Separate Unix (MacOS & Linux) and Windows unit tests into separate workflow files
• Use Anaconda to install Python and required packages
• Pass Anaconda version of python to MATLAB
• Add version matrix of MATLAB and Python
  • Python: 3.8 - 3.11
  • MATLAB: R2021b to R2023b
  • Exclude Python/MATLAB combinations that are not supported
    • 15 different combinations on each operating system
  • Exclude combinations that are error out when calling Python from MATLAB
    • Windows MATLAB > 2022a
    • MacOS MATLAB latest
• Add caching/artifact for hindcast data calls
  • This API is rate limited and the tests make successive requests causes intermittent failures in test_WPTO_point_multiloc and test_WPTO_point_multiparm.
  • Adds a cache_population job that runs a test with caching enable before the main job
    • Caches api call results in mhkit_webread_cache
    • main job reads in mhkit_webread_cache and uses the cache

Tests

• 151/151 of known good unit tests are passing
• Addresses issue #77

Fixed Tests

This PR fixes or adds the following tests:

• Loads_TestExtreme
  • [x] test_mler_coefficients (b40f427)
• QC_Test:
  • [x] test_check_corrupt (83dbc31)
  • [x] test_check_delta (83dbc31)
  • [x] test_check_increment (83dbc31)
  • [x] test_check_missing (83dbc31)
  • [x] test_check_timestamp (83dbc31)
• River_TestDevice
  • [x] test_circular (42c9a58)
    • This MHKiT-Python commit changes the value of this calculation, and this fix updates the assertion.
• River_TestResource
  • [x] test_exceedance_probability (cd0271d)
    • Setup Q as a struct with Discharge and time keys
• Tidal_TestIO:
  • [x] test_request_noaa_data (e2613e4)
• Tidal_TestResource:
  • [x] test_plot_phase_exceedance (30ead68)
• Wave_TestResourceMetrics:
  • [x] test_metrics_HsP (40cd75e)
  • [x] test_metrics_AH (40cd75e)
  • [x] test_metrics_CDIP1 (40cd75e)
  • [X] test_plot_elevation_timeseries (1b973e4)
  • [x] test_energy_flux_deep (40cd75e)
• Wave_TestResourceSpectrum
  • [x] test_surface_elevation_seed (a0cb0ef)
  • [x] test_surface_elevation_phasing (e2f6080)
  • [x] test_surface_elevation_moments (b0d93bb)

Added Tests

• [X] Python_Import
  • Add test to verify Python import mhkit performs nominally
  • Add test to verify Python import pandas performs nominally
  • Add test to verify Python import h5py performs nominally
  • Add test to verify operation of MHKiT Python per Test MHKiT-Python installation
• [X] Python_IO
  • Add Tests to verify MATLAB -> Python -> MATLAB serialization for different types
    • This is a WIP/Proof of Concept
    • Finishing this is out of scope for this PR. Future work could/should include expanding these tests

Warnings

• Fix warnings caused by importing python libraries that have functions with the same name as MATLAB functions (be70376)
  • Remove every call of py.importlib.import_module('numpy');
    • This imports all numpy functions into the global namespace and matplotlib gives the following warning for every function
      • [Warning: The name '<numpy function>' is already in use as a method name. This will become an error in a future release.]

Deprecated Tests

The following tests are not compatible with the latest versions of MHKiT-Python:

Note: Known failing tests use assertFail to force them to pass.

• Wave_TestResourceMetrics:
  • test_environmental_contour
    • Not compatible with latest version of MHKiT Python
  • test_plot_environmental_contour
    • Not compatible with latest version of MHKiT Python
  • test_plot_environmental_contour_multiyear
    • Not compatible with latest version of MHKiT Python

Code Coverage

• Add built in MATLAB code coverage (1f5f93a)
  • Addreses issue #67 and issue #86
  • Does not run automatically

Style

• Fix common spelling errors
• Trims whitespace in all files
• Convert files with dos line endings to unix line endings

Documentation

• Update README and other documentation to reflect changes from the ebbflood branch and this PR
  • [X] Add MATLAB/Python compatibility to README
  • [X] Update Testing Badges
    • Add Linux and macOS Unit Test Badge
    • Add Windows Unit Test Badge

ebbflood Branch #101

Features

• Power Quality Flicker Assessment Update (#105)
  • Update power quality flicker assessment workflow and calculations (077f525)
• Dolfyn Functionality: IO and Rotations (#79)
  • Implement Dolfyn IO functionality for classic Nortek data format (d29d5e3)
  • Add support for RDI file read (d29d5e3)
  • Initial development for NetCDF read and write functionality (d29d5e3)
  • Speed optimizations, bug fixes, and rotations for Dolfyn (d29d5e3, 4761a23, 71b0b38)
  • Addresses issue 66
• CDIP Data IO and Graphics (v2) [develop] (#83)
  • Implement CDIP data collection and plotting functions (2e91690, d723b3a, 9318f4f)
• Wave Updates (#70)
  • Add depth_regime, wave_length, and depth flags (6be98b5)
  • Implement updates to wave-related functions and tests (6be98b5, 2e91690)
• General Updates (#71)
  • Miscellaneous updates to spectrum, elevation, and IO paths (5d4cb8d, 8c28dcd, 722f42e)
• Addressing issue #59 Magnitude and Phase Function for Matlab (#76)
  • Implement magnitude and phase function for Matlab (c557c43, 219879e)
• Addressing issue #65 Tidal Ebb and Flood Plots for Matlab (#73)
  • Implement tidal ebb and flood plots for Matlab (c7f3992, 3916302, 3c9bd51)
• Move CI Unit Tests to GitHub Actions (#80)
  • Migrate CI unit tests to GitHub Actions (114c443, deedd7e, 45e15fd)

Bug Fixes

• Typo Fix in Examples (#103)
  • Correct typos in example files (7895020)
• Revert "Addressing issue #65 tidal ebb and flood plots for Matlab (#73)" (#84)
  • Revert the tidal ebb and flood plot changes (39f11dc)
• Revert "Revert "Addressing issue #65 tidal ebb and flood plots for Matlab (#73)" (#84)" (#85)
  • Revert the revert of tidal ebb and flood plot changes (85f5f5d)
• Revert "Run tests on my fork at the same time" (33da001)
  • Revert running tests on fork simultaneously (33da001)
• Fixing File Path Call in Tests (#81)
  • Correct file path calls in tests (ed0992d)
• Fixing Typo (#76)
  • Resolve typos (ffe58bb)
• WPTO for Matlab (#61)
  • Address issues related to WPTO for Matlab (114c443, deedd7e, 45e15fd)

Other

• Conditional Run of Hindcast Tests (#79)
  • Conditionally run hindcast tests (3697735)
• Updating Package (#83)
  • Package updates and error fixes (2e91690, d723b3a, 9318f4f)
• Finalize Workflow (#71)
  • Finalize workflow for testing and update IO paths (8c28dcd, 722f42e, 386ac97)
• Updating Power Example with Whole Workflow of Flicker (#105)
  • Update power example with complete flicker assessment workflow (077f525)
• Minor Changes (#105)
  • Miscellaneous minor changes in the power quality flicker assessment (077f525)

Caveats

Windows Unit Tests

Deactivation of selected Windows Unit Tests (MATLAB > 2022a) is necessary for the following reasons:

• Running MATLAB tests on a GitHub Actions Windows server fails due to errors using Python in MATLAB in the GitHub actions environment.
  • Running Python OutOfProcess yields Unable to launch Simple server: Unable to launch C:\Program Files\MATLAB\R2023b\interprocess\bin\win64\pycli\MATLABPyHost.exe
  • pyenv(ExecutionMode='OutOfProcess') allows access to Python libraries installed by the user and is required to use mhkit within MATLAB
  • Running Python InProcess yields Unable to launch Simple server: Unable to launch C:\Program Files\MATLAB\R2023b\interprocess\bin\win64\pycli\MATLABPyHost.exe

MacOS Unit Tests

Deactivation of selected MacOS Unit Tests for Python 3.11 and MATLAB latest is necessary for the following reasons:

• Throws the following error on every Python call from MATLAB (macos-latest, MATLAB latest, Python 3.11)

Python Error: ImportError:
      dlopen(/usr/local/miniconda/envs/TEST/lib/python3.11/site-packages/netCDF4/_netCDF4.cpython-311-darwin.so,
      0x0002): Symbol not found: (_SSL_get0_group_name)
        Referenced from: '/usr/local/miniconda/envs/TEST/lib/libcurl.4.dylib'
        Expected in:
        '/Users/runner/hostedtoolcache/MATLAB/2023.2.999/x64/MATLAB.app/bin/maci64/libssl.3.dylib'

[rpauly18]

I'll be adding comments as I go through the review -

1) Please open an issue on the MHKiT-Matlab repo (if you have not already) for the depreciated tests so we can assign someone to address the incompatibility 2) for the code coverage report, you say it does not run automatically - how do you trigger it?

[simmsa]

@rpauly18,

• I created Issue #109 to detail and track the current failing tests.
• 6b744e7 adds a "Development Notes" section to the README which includes details on generating code coverage reports.

[rpauly18]

@simmsa feel free to merge this if you are 100% done with any additional changes.

[simmsa]

@rpauly18 Thank you for the review and the merge!