Ozone tests inconsistency - pytest --disable-recording not fixing cassette recording issues

[Milind220]

Note: I created this separate issue for this topic, even though this is actually what was being discussed in #130

Description The tests show me that two tests are failing. When I tried @lahdjirayhan 's suggestion of running the following: pytest tests/test_get_city_forecast.py tests/test_get_coordinate_air.py --disable-recording Then 16/17 tests failed for really strange reasons.

Screenshots of the tests failing are attached below.

To Reproduce Steps to reproduce the behavior:

1. Create a new conda virtual env with python 3.9
2. setup the development environment as detailed in the CONTRIBUTING file
3. run pytest in the root directory to see the two originally failing tests.
4. run pytest tests/test_get_city_forecast.py tests/test_get_coordinate_air.py --disable-recording to see 16/17 tests fail

Expected behavior

• The two failing tests in the beginning make sense as the cassettes need to be re recorded.
• However, the 16 failing tests when running pytest with --disable-recording is quite confusing. I expected all tests to pass with this.

Screenshots when I ran just pytest

when I ran pytest tests/test_get_city_forecast.py tests/test_get_coordinate_air.py --disable-recording

Environment

• MacOS
• Python 3.9.12
• Ozone installed in developed in editable mode as per CONTRIBUTING guide. Last commit was 57530e93d4526d515acb9d1876ad59e364fee647
• pytest run in dev branch
• Execution Environment: Terminal with conda environment activated

[lahdjirayhan]

Oh no, I forgot to add information about how to supply your own token 🤦🏼 , because eventually if we use --disable-recording, we're doing live requests (which will need our AQI token). I think I'll do some writeups right now to inform everyone (including myself) in the future about this; rather than writing the solution in this issue.

Hold my beer, I got this.

What we roughly need is as follows:

• Add information how to prepare your own token for testing purposes.
• Add information about when exactly to use --disable-recording vs relying on mock cassettes.
• Move information from tests/README into CONTRIBUTING, which is a more proper and unified place for information.

[lahdjirayhan]

For future visitors to this thread:

If you get vcr.errors.CannotOverwriteExistingCassetteException when running pytest, you probably need to use live connection (disable cassette recording) and see if the test passes. That exception is raised when the cassette file is not complete (there is a request in the test function that's not recorded in the cassette file).

If you get Exception: Your API token is invalid when running pytest --disable-recording, it's probably because you haven't supplied a working API token to the test suite. In turn, this will make the test suite uses a dummy token, which is obviously not a valid API token.

More about the test suite is explained in CONTRIBUTING.

[lahdjirayhan]

@Milind220 I just want to confirm, have you checked that the information provided in the PR is sufficient to solve the problem posed in this issue?

[Milind220]

@lahdjirayhan Yes, fixed it perfectly! The token was all that was needed.

EDIT: I no longer see the 'not provided a token' issue, but now a different test fails. I ran pytest with recordings disabled here.

I'm having a hard time deciphering what this means. Do I understand correctly that the purpose of this test is to check that incorrectly entered coordinates are handled smoothly? Where did 50.805778 come from?