Base and Weather Ingress

[EdwinB12]

Includes:

• BaseIngress for shared utilities. Class to be inherited by specific data ingress options.
• Readme describing what the BaseIngress is and how to use it to write custom DataIngress Classes.
• OpenWeatherDataIngress Class for ingressing weather data from OpenWeatherMap API.
• Basic tests for handling dt_from and dt_to. These are the variables determining what time range to pul data from.

TODO before merging :

• Tests for BaseClass methods. Will require a simple mocking of the get_data method.
• Tests for OpenWeatherDataIngress methods

TODO but perhaps different PR:

• Azure Functions Utility. Hopefully can be included into BaseIngress

[review-notebook-app[bot]]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[codecov-commenter]

Codecov Report

Attention: 18 lines in your changes are missing coverage. Please review.

Comparison is base (8ccc262) 89.12% compared to head (8a21db1) 89.16%.

:exclamation: Current head 8a21db1 differs from pull request most recent head 14e22ca. Consider uploading reports for the commit 14e22ca to get more accurate results

Files	Patch %	Lines
dtbase/ingress/ingress_weather.py	83.15%	16 Missing :warning:
dtbase/backend/api/sensor/routes.py	50.00%	1 Missing :warning:
dtbase/core/utils.py	87.50%	1 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #174 +/- ## ========================================== + Coverage 89.12% 89.16% +0.04% ========================================== Files 70 75 +5 Lines 4853 5039 +186 ========================================== + Hits 4325 4493 +168 - Misses 528 546 +18 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[mhauru]

TODO but perhaps different PR: Azure Functions Utility. Hopefully can be included into BaseIngress

Agreed, this can be a separate PR.

[EdwinB12]

@mhauru - I'm reading in some hardcoded responses for mocking purposes (such as MOCKED_CONST_OPENWEATHERMAP_HISTORICAL_URL_RESPONSE). They are quite large dictionaries. Have you got any strong opinions on where I should store these dictionaries:

• In test_ingress_weather.py
• In conftest.py (Have them as fixtures).
• In a seperate resources file (this is what I've currently implemented).

[mhauru]

The separate resource files sound good to me.

[mhauru]

One thing I was thinking of was that maybe there should be a separate pair of OpenWeatherMap sensors (historical and forecast) for every lat/long. That the lat and the long would be in the unique identifier of the sensor. Maybe this is for a separate PR though. I'm in general not fully happy with how we currently use environment variables, there may be a bit of an overhaul to do there, but it might make sense to do once things are running on Azure, since envvars are often the best way to handle parameters there.

[EdwinB12]

I guess we assume for now that a single entity would only use one lat and long for something liek weather. But agreed, I think it should be included in the sensor table.