India Todo list

[peterdudfield]

Data Consumer

Info: Collect National wind and PV data from https://sldc.rajasthan.gov.in/rrvpnl/rajasthan-overview

https://github.com/openclimatefix/ruvnl-consumer-app

• [x] script to pull this data
• [x] Create app that runs collects this data every x mins (probably every 30 mins with triggered by airflow).
• [x] Would be nice if the app doesnt add any duplicate values in the database Examples, GSPConsumer, PVConsumer, nwp-consumer, but these might be too complicated

Forecast App

• Make forecast app, which saves fake forecasts to database
  • [x] Create new repo, Have a look at pv-site-production, pvnet_app. Could copy/ adapt one, but I would tempted to make a new one which is really simple
  • [x] Should be an app, running with click.
  • [x] Should have versioning, with bumpversion
  • [x] Should push a docker image
  • [x] Should run tests
  • [x] Can copy fake pv and wind algorithsm from fake API repo
  • [x] Should load a sites and run forecast for each one, and save to database
• [x] openclimatefix/india-forecast-app#11

Use pvsite-Datamodel

• [x] add asset_type so site can be wind or solar - https://github.com/openclimatefix/pv-site-datamodel/issues/94
• [x] do something with gsp and/or dno (decided just to leave it for now)
• [ ] Change name of repo, maybe quartz-site-datamodel - https://github.com/openclimatefix/pv-site-datamodel/issues/93

API

• [x] openclimatefix/india-api#6
• [x] openclimatefix/india-api#7
• [x] openclimatefix/india-api#8

Email

• [x] openclimatefix/india-api#16
• [x] openclimatefix/india-api#15

Infrastructure

• [x] Deploy API
• [x] Add RDS database
• [x] Setup airflow
• [x] Run database migrations
• [x] openclimatefix/ocf-infrastructure#407
• [x] openclimatefix/ocf-infrastructure#408

[jacobbieker]

Should this be imported into datapipes, like the current nowcasting_datamodel is? I'm primarily asking as I can't add India PV inputs to datapipes without changing nowcasting_datamodel to take more labels: https://github.com/openclimatefix/nowcasting_datamodel/blob/8fcc3ae61b50ca30815b910d9db2621ae6b2c33e/nowcasting_datamodel/models/pv.py#L20-L22

And I'm not sure whether that should be done here, or in the other one

[jacobbieker]

And if so, and nowcasting_datamodels is UK-specific, we might want to change how we name it

[peterdudfield]

Interesting, thanks @jacobbieker for flagging this.

Nowcasting_datamodel has a lot of GSP specific stuff in it, so I would be tempted to actually use pv-site-datamodel as a better repo to start with.

I'm a little bit suprised you need nowcasting_datamodel for ingesting PV India data? Can you share the code where this is happening? Is this for validating the data?

[jacobbieker]

Its this check here: https://github.com/openclimatefix/ocf_datapipes/blob/e86aabfe62f52a0bf18bd0a8c9cc1109e961e637/ocf_datapipes/config/model.py#L234-L243 that checks which provider the PV data is from in the config, this fails when not one of those ones listed here: https://github.com/openclimatefix/nowcasting_datamodel/blob/8fcc3ae61b50ca30815b910d9db2621ae6b2c33e/nowcasting_datamodel/models/pv.py#L20-L22

I could remove the check if we want? Would reduce dependencies and probably be fine

[jacobbieker]

I think for now, disabling the validator should work fine. We can always re-enable it if we want, or change how it works if we are splitting the datamodel into different repos for different regions/etc.

[peterdudfield]

Also if you are just importing those strings? you could move then to datapipes this means you can be more flexible and add more providers more easily

[jacobbieker]

Yeah, that might be the best way, I can do that

[peterdudfield]

Note we could also use pv-site-datamodel and expand this to wind-and-pv-site-datamodel. We would have to add a columns fo the site like:

• asset_type=wind or pv
• country

But this might save us a lot of duplicate code. @devsjc would be interested in your opinion?

[confusedmatrix]

I think we should also add a country field to the SiteSQL model in pvsite-datamodel to make it easy to query the for India specific sites from the forecast app. Currently, there is region - but this is perhaps more specific than country-level.

[peterdudfield]

I think we should also add a country field to the SiteSQL model in pvsite-datamodel to make it easy to query the for India specific sites from the forecast app. Currently, there is region - but this is perhaps more specific than country-level.

yea lets do it, default to 'uk'