Process testing triggered on PR

[PennyHow]

.github/process_test.yml performs pypromice processing tests on real AWS data from PROMICE and GC-Net stations. This is triggered on pull requests to the main branch of pypromice as a Github Action, and can also be manually triggered. Similar to.github/unit_test.yml #101, a pull request can not be accepted until this workflow successfully passes. In addition, output files are posted as a Github Action artifact after each run here, so they can be viewed and checked.

A few notes:

• This action only performs processing in a Python 3.8 environment (which is what we are currently using in our operational processing). We could run it on other Python versions too, if necessary

• I came across dependency problems in the pypromice set up on the Github Action VM - Bottleneck and netcdf4 are optional dependencies of xarray which we use in pypromice. I've added them in the setup.py so they are installed on pip install . instead of defining them in .github/process_test.yml

• Currently, this only processes data from a couple of stations that cover a range of different processing scenarios:

  1. KPC_U, one-boom (PROMICE) v2 station data
  2. JAR, one-boom (PROMICE) v3 station data
  3. CEN2, two-boom (GC-Net) station data I've only defined it to process L0 raw data. We could also have a test run with L0 tx data if we wish. I'm happy to include other stations that may be better at representing all station and data types

• We only perform getL3 processing for now in this action, but it may be that we want to add getL0tx and getBUFR in the future. Then we will have full testing coverage of all of our operational processing elements. For now, the unit testing covers the tx message retrieval and we have no testing of the BUFR file creation

[patrickjwright]

I agree that adding getL0tx and getBUFR will be important, but maybe not mandatory now, just to get this running and out the door. I left a few comments, but seems good to merge whenever you are happy.

[patrickjwright]

One other question. When exactly is this triggered? When you go to create a PR? Or when you try to merge the PR? And will both the unit test and the process test be triggered to run consecutively?

[PennyHow]

It should trigger when a PR is created, and it will block a PR from being merged if the tests fail. The unit and process test are triggered simultaneously.

At least, this is what I think should happen. It's hard to know if this is how it will work until it is merged with the main branch, but from reading around and testing on other branches, it should do! (fingers crossed!)

[PennyHow]

Now the two workflows are merged, I will test it with a branch that purposefully doesn't work. I'll open a PR, and the tests should fail and block a user from merging the PR.