Mock the erddap

[gmaze]

The problem

As mentioned in the last release PR, CI tests suffer from very long execution delays that we suspect to come from remote server connections.

We already created a mocked FTP server in #249 but we have similar issues with all remote server connections.

NB: Dealing with remote servers in CI tests has been a difficulty since the very beginning of argopy (#26 ). Up to this point the strategy was to catch all the possible request errors that were not due to argopy but to the remote server or the connection with a specific fixture

The solution

Here, we extend the mocked server approach to the Ifremer erddap. In other words, this PR will provide all the necessary fixture and data to mock all Ifremer erddap requests (with fsspec using the argopy httpstore). Hopefully (?) this will dramatically reduce the execution time of CI tests.

[gmaze]

Implementation 1

This add the following dependencies for CI environments:

• aiofiles
• aioresponses

The mocked http server is implemented with aioresponses. And for this to work we first need to download the expected erddap server raw response and save it on a local file that will be served by the mocked server. This is done out of the CI tests suite (typically on my laptop at this point) using aiofiles.

This approach raises the questions:

1. how to make sure all expected files used in the CI tests will stay sync with possible evolution of the erddap server ?
2. how to distribute and access these tests data ? should it be packaged with the software ? should it be on another repo (eg: https://github.com/euroargodev/argopy-data) ? if it stays in here, how do me make sure it's distributed with the pip/conda package ?

RESULTS: 👎 This solution based on aioresponses was not reliable and quite complicated to manage because of the subtleties between HEAD and GET requests. I also could not quite understand how it really worked and did not managed to face some weird stuff like 2 similar requets in a row, the 1st one succeed the 2nd returns a 404 🤬

[gmaze]

Implementation 2

I dig into the fsspec tests suite and gave a try to their implementation of a simple local server with http.server from here and used by the http store tests.

This is different from the 1st implementation above because here, we don't catch the erddap requests to replace them, but we create a local http server. Therefore, to try this I had to make the erddap data fetcher configurable with regard to the server used.

RESULTS: 👏 This solution seems to work smoothly ! Thanks fsspec for the inspiration

The mocked erddap server is used in:

• test_fetchers_data_erddap.py
• test_options.py
• test_stores.py

[gmaze]

will merge since this has achieved the PR purpose will open a new PR to implement the mocked http server approach to all other requests !