Feat opendap

[danangmassandy]

This is for #11 TODO:

• [x] Unit tests
• [x] Investigate slowness accessing multiple files via opendap dmr++
• [x] Update Infra diagram and documentation on OpenDAP integration

Sample API calls:

• CBAM measurement Max Total Temperature: URL: http://localhost:8000/api/v1/measurement/?lat=-3.25&lon=30.4&attribute=Max%20Total%20Temperature&start_date=2019-11-01&end_date=2019-11-02 Response:

  {
  "2019-11-01": 24.6646728515625
  }

• Salient measurement Temperature Climatology http://localhost:8000/api/v1/measurement/?lat=-2.215&lon=29.125&attribute=Temperature%20Climatology&start_date=2024-03-13&end_date=2024-03-13 Response: salient_temp_clim.zip

[github-actions[bot]]

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines	Covered	Coverage	Threshold	Status
1315	1245	95%	0%	🟢

New Files

No new covered files...

Modified Files

No covered modified files...

updated for commit: bec1e2b by action🐍

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 96.40523% with 22 lines in your changes missing coverage. Please review.

Project coverage is 94.95%. Comparing base (28f72a3) to head (bec1e2b).

Files	Patch %	Lines
django_project/gap_api/api_views/measurement.py	78.87%	15 Missing :warning:
django_project/gap/ingestor/tahmo.py	66.66%	3 Missing :warning:
django_project/gap/utils/netcdf.py	97.70%	3 Missing :warning:
django_project/gap/models/measurement.py	92.30%	1 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #19 +/- ## ========================================== + Coverage 93.30% 94.95% +1.64% ========================================== Files 31 40 +9 Lines 612 1208 +596 ========================================== + Hits 571 1147 +576 - Misses 41 61 +20 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[danangmassandy]

Update: I'm using xArray instead of OpenDAP+pyDap. There is improvement when reading the files from s3 (minio). Previously using OpenDAP+pyDap to read 10 files of CBAM NetCDF, the API took 6-10seconds. Now, the API only take 1seconds. The other benefit is we can now fetch multiple variables at once.

The downside that I noticed is that the xArray will download parts of NetCDF files bigger than OpenDAP with DMR++. Note: I cannot generate DMR++ file for Salient dataset, so the previous API will have to download the file first into Hyrax file cache.