openclimatefix / uk-pv-national-xg

National PV forecasting using Gradient Boosted Methods.
4 stars 3 forks source link

Tom/create app #14

Closed thomasarmstrong98 closed 1 year ago

thomasarmstrong98 commented 1 year ago

Pull Request

Description

Add model loading and saving from s3. Create command line app with click. Some further refactoring and removing unnecessary notebooks.

Fixes #

2 and #8

How Has This Been Tested?

Not sure whether to add s3 mock tests? Mock setup script runs successfully with GCP data. The configuration script still needs to be tested further, NOT READY to merge yet.

Checklist:

peterdudfield commented 1 year ago

I would be great to write test for this clikc function

This would be somehting more like a unit test rather than an intergration test. Intergration test would load data from live database, this could be written later

peterdudfield commented 1 year ago

P.s. I converted this to a draft so that so that we dont accidentally merge it

thomasarmstrong98 commented 1 year ago

I would be great to write test for this clikc function

* It could download s3 model, we can add AWS credentials into this repo for testing purposes

* The NWP and GSP can be mocked

This would be somehting more like a unit test rather than an intergration test. Intergration test would load data from live database, this could be written later

Yep, how do we want to add the AWS credentials for testing?

peterdudfield commented 1 year ago

I would be great to write test for this clikc function

* It could download s3 model, we can add AWS credentials into this repo for testing purposes

* The NWP and GSP can be mocked

This would be somehting more like a unit test rather than an intergration test. Intergration test would load data from live database, this could be written later

Yep, how do we want to add the AWS credentials for testing?

Would have to make some AWS credentials with the rights to read that bucket. Then add the AWS credientials to the Github repo.

Perhaps just first get the test running locally, and when can then add that later

thomasarmstrong98 commented 1 year ago

app.py produces predictions using live nwp/gsp data image

peterdudfield commented 1 year ago

app.py produces predictions using live nwp/gsp data image

Looks great - well done - looks like they are MW, so might just need a rename - we can sort that later

peterdudfield commented 1 year ago

app.py produces predictions using live nwp/gsp data image

Looks great - well done - looks like they are MW, so might just need a rename - we can sort that later

Need to round the start time to the lower 30 mins, so the target times are in 30 mins steps