Issue/probablistic

[peterdudfield]

Pull Request

Description

• add column "properties" to ForecastValue that can store probablistic data
• update save method
• add migration
• update blend method
• need to pin pydantic=1.10.10 as there were some breaking changes - https://github.com/openclimatefix/nowcasting_datamodel/issues/205

https://github.com/openclimatefix/nowcasting_datamodel/issues/197

How Has This Been Tested?

• updated save test to check these values
• normal CI

Checklist:

• [x] My code follows OCF's coding style guidelines
• [x] I have performed a self-review of my own code
• [ ] I have made corresponding changes to the documentation
• [x] I have added tests that prove my fix is effective or that my feature works
• [x] I have checked my code and corrected any misspellings

[codecov[bot]]

Codecov Report

Merging #202 (f3b858c) into main (2e1d06e) will increase coverage by 0.09%. The diff coverage is 100.00%.

:exclamation: Current head f3b858c differs from pull request most recent head bbfa333. Consider uploading reports for the commit bbfa333 to get more accurate results

@@            Coverage Diff             @@
##             main     #202      +/-   ##
==========================================
+ Coverage   94.83%   94.92%   +0.09%     
==========================================
  Files          27       27              
  Lines        1627     1656      +29     
==========================================
+ Hits         1543     1572      +29     
  Misses         84       84              

Impacted Files	Coverage Δ
nowcasting_datamodel/fake.py	100.00% <ø> (ø)
nowcasting_datamodel/models/gsp.py	94.28% <ø> (-0.09%)	:arrow_down:
nowcasting_datamodel/models/metric.py	97.05% <ø> (-0.09%)	:arrow_down:
nowcasting_datamodel/models/convert.py	100.00% <100.00%> (ø)
nowcasting_datamodel/models/forecast.py	84.53% <100.00%> (+0.34%)	:arrow_up:
nowcasting_datamodel/read/blend/blend.py	96.22% <100.00%> (+2.89%)	:arrow_up:
nowcasting_datamodel/read/blend/utils.py	91.13% <100.00%> (+0.34%)	:arrow_up:
nowcasting_datamodel/save/update.py	99.01% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[peterdudfield]

I guess we're using a JSON column instead of columns per the fields in the JSON because they are expected to be inconsistent and numerous between different forecasts?

Yea, and for the moment I think we will store plevel_10 and plevel_90 but I think this could change quite soon, so this keeps it a bit more flexible.

Another way could be to do it with a join to another table, 1 to many and have this results, but I'm a bit caution of doing another big join