Split integration tests and unit tests

[peterdudfield]

Detailed Description

There are both intergations tests and unit tests. It would be good to split these up, perhaps into different folders. Then we can always run the unit tests.

Context

• The integration tests need secrets, which I dont think we can give out openly

Possible Implementation

Split tests into unit and integrations, two different folders.

• update CI to run them seperately

[Jacqueline-J]

Hi Peter,

I've taken a look at this. This is the work I've done so far;

1. Test Organization: I split the test in the Satip/tests folder into unit and integration test. Take test_download.py as an example, I split the code into unit_test_download.py and integration_test_download, and stored them in two separate folders (1) unit_test and 2) integration_test). I did the same for the other 5 tests.
2. Workflow Updates: I updated the workflows.yaml file from .github/workflows/workflows.yaml. I replaced the Run pytest command with a pytest tests/unitTest command, and a pytest tests/integrationTest command, ensuring that both tests run separately.

I used the same specifications for both of them as set out in the original Run pytest command. Here's the updated workflow snippet:

- name: Run unit tests                  # ----- ADDED THIS LINE 
        run: |
          pytest tests/unitTest           # ----- ADDED THIS LINE 
          export EUMETSAT_USER_KEY="${{ secrets.EUMETSAT_USER_KEY }}"
          export EUMETSAT_USER_SECRET="${{ secrets.EUMETSAT_USER_SECRET }}"
          export PYTEST_COMMAND="pytest $PYTESTCOV $PYTESTXDIST -s --log-level=DEBUG tests/unit"   # ----- CHANGED THIS LINE 
          echo "Will be running this command: $PYTEST_COMMAND"
          eval $PYTEST_COMMAND

      - name: Show coverage
        run: coverage report -m

      - name: "Upload coverage to Codecov"
        uses: codecov/codecov-action@v2
        with:
          fail_ci_if_error: false

      - name: Run integration tests   #------- ADDED THIS LINE 
        run: |
          pytest tests/integrationTest  # ------ ADDED THIS LINE 
          export EUMETSAT_USER_KEY="${{ secrets.EUMETSAT_USER_KEY }}"
          export EUMETSAT_USER_SECRET="${{ secrets.EUMETSAT_USER_SECRET }}"
          export PYTEST_COMMAND="pytest $PYTESTCOV $PYTESTXDIST -s --log-level=DEBUG tests/integration"  # ----- CHANGED THIS LINE 
          echo "Will be running this command: $PYTEST_COMMAND"
          eval $PYTEST_COMMAND

      - name: Show coverage
        run: coverage report -m

      - name: "Upload coverage to Codecov"
        uses: codecov/codecov-action@v2
        with:
          fail_ci_if_error: false

Next Steps: I'm now at the stage where I need to test the code to ensure everything is running correctly. I'm unsure of how to go about doing that. Do you have any suggestions on how I can proceed or any guidance on testing these changes?

• I'll remove the annotations for the final implementation. Looking forward to your feedback! :)

[peterdudfield]

Thanks so much for doing this.

So the unit tests, you should be able to run, but the intergation test will need some secrets.

Do you want to do a PR, and then we can work out if its right?