search

cpmodel / FTT_StandAlone

Future Technology Transformation models
GNU General Public License v3.0
10 stars 1 forks source link

PR for Power Update for S0 #113

Open ldxib2 opened 5 months ago

Femkemilene commented 5 months ago

Hi Ian,

Can you make sure S1 and S2 are also updated? The model may crash if we accept this PR as is. There is an option for a draft pull request if you'd like eyes on an update and you need to do the finishing touches still.

Femke

From: Ian Burton @.> Sent: 10 April 2024 14:09 To: cpmodel/FTT_StandAlone @.> Cc: Nijsse, Femke @.>; Review requested @.> Subject: [cpmodel/FTT_StandAlone] PR for Power Update for S0 (PR #113)

CAUTION: This email originated from outside of the organisation. Do not click links or open attachments unless you recognise the sender and know the content is safe.

You can view, comment on, or merge this pull request online at:

https://github.com/cpmodel/FTT_StandAlone/pull/113

Commit Summary

File Changes

(8 fileshttps://github.com/cpmodel/FTT_StandAlone/pull/113/files)

Patch Links:

— Reply to this email directly, view it on GitHubhttps://github.com/cpmodel/FTT_StandAlone/pull/113, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGHDII5FC4JGBWYKO5RZUV3Y4U2XHAVCNFSM6AAAAABGANNETOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGIZTKNJWGEZDSMI. You are receiving this because your review was requested.Message ID: @.***>

Femkemilene commented 5 months ago

Can you show a comparison with some key variables between the old and new model? Is this in the direction we're expecting the change. The key differences I expect are:

Femkemilene commented 5 months ago

@ldxib2: why are solar prices in the US going down this suddenly (S0)? And why is the costs of offshore wind going up in the US and in China in the S2 scenarios? It should not be more expensive in these scenarios. Is something going wrong with the capacity factor calculations? There are some less pronounced oddities in the old version too.

ldxib2 commented 5 months ago

Issues with the S2 raise in price have been corrected. MEFI was being added in the LCOE script instead of subtracted. Hotfix was created to sort this in the main branch and here too. S2 graphs above are now obsolete. Cause of S0 drop in solar price is still to be determined

ldxib2 commented 5 months ago

After mcsc hotfix comparison outputs

S0_2022_CN_LCOE

S0_2022_CN_LCOE

S0_2024_CN_LCOE

S0_2024_CN_LCOE

S0_2022_US_LCOE

S0_2022_US_LCOE

S0_2024_US_LCOE

S0_2024_US_LCOE

S0_2022_MEWW

S0_2022_MEWW

S0_2024_MEWW

S0_2024_MEWW

S0_2022_global_shares

S0_2022_global_shares

S0_2024_global_shares

S0_2024_global_shares

Femkemilene commented 5 months ago

I see nothing worrying in these changes. Surprisingly little has changed after the model update. To make life easier for ourselves, we can choose to first figure out the differences between main and FORTRAN before merging the updated data.

pv-camecon commented 5 months ago

I'll investigate model differences this week. Suspects are:

Femkemilene commented 5 months ago

I've been comparing to 2022_baseline_update, which does have LBD in both CAPEX and OPEX. I also checked the capacity factors from the cost-supply curves: they're the same between the two model versions.

Ian compared the demand (very similar).

I found quite strong differences for MSSP for solar, but they can't explain the direction of difference. In SA, MSSP reaches a maximum of 11k $ per GWh, whereas in 2022_baseline it reaches 22k per GWh.

pv-camecon commented 4 weeks ago

Unless I am missing something, the current version committed does run correctly. I am getting a silent model crash due to issues in 2022 as storage cost becoming NaN implying most likely a div zero error has occurred. Can someone please test checking out a clean version of this and confirm the same issue.

Specifically I see the issue in regions 56, 62-71.

This is due to empty cells for onshore and offshore wind power in 2022 in the master files for all regions. Something went wrong there when bringing over the new data.

ldxib2 commented 4 weeks ago

I have just checked out this branch and I was able to run it even with the missing data. We don't have data for 2022 in this update as we stop at 2021. I checked the histend dates and they are fine too. Have you tried just filling the empty cells with random data, maybe copying from another masterfile just so there is something in there? I'm working on the belief it shouldn't change anything based on the 2021 histend

jp-camecon commented 4 weeks ago

I have just checked out this branch and I was able to run it even with the missing data. We don't have data for 2022 in this update as we stop at 2021. I checked the histend dates and they are fine too. Have you tried just filling the empty cells with random data, maybe copying from another masterfile just so there is something in there? I'm working on the belief it shouldn't change anything based on the 2021 histend

When you ran this through, were you able to view results through to 2050? When I download this branch as it stands, it runs through to completion, but the results are NaN after 2022

pv-camecon commented 4 weeks ago

The input function will read in the whole timeline and that's where the NaNs get included despite occurring after histend. Presumably, the variable is used somewhere before it is overwritten and this introduces NaNs everywhere else.

From: Jamie Pirie @.> Sent: Wednesday, September 4, 2024 16:21 To: cpmodel/FTT_StandAlone @.> Cc: Pim Vercoulen @.>; Review requested @.> Subject: Re: [cpmodel/FTT_StandAlone] PR for Power Update for S0 (PR #113)

I have just checked out this branch and I was able to run it even with the missing data. We don't have data for 2022 in this update as we stop at 2021. I checked the histend dates and they are fine too. Have you tried just filling the empty cells with random data, maybe copying from another masterfile just so there is something in there? I'm working on the belief it shouldn't change anything based on the 2021 histend

When you ran this through, were you able to view results through to 2050? When I download this branch as it stands, it runs through to completion, but the results are NaN after 2022

— Reply to this email directly, view it on GitHubhttps://github.com/cpmodel/FTT_StandAlone/pull/113#issuecomment-2329203906, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A45J7ETOXF7FCQTTWDNNFTLZU4JN3AVCNFSM6AAAAABGANNETOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRZGIYDGOJQGY. You are receiving this because your review was requested.Message ID: @.***>

jp-camecon commented 4 weeks ago

To follow up. Pim shared with me the earlier discussion that the issue I am having can be resolved by doing a manual update of the csv file by running SourceCode / support / convert_masterfiles_to_csv.py. I will review the output as it stands and if that is fine I will accept this update and then we look at the refresh of csv files once we have dealt with the rest of the FTT power update.

ldxib2 commented 4 weeks ago

I ran the convert_csv_files.py first too. I believe that this file was updated on one of the other branches, maybe explore_differences_fttp? That update was designed to check if the input files need updating automatically based on when the last changes to the masterfile were. If I am right then when we accept this update, then bring in the alignment changes with the generic sales PR then it should be brought in

jp-camecon commented 3 weeks ago

Overall, the version looks fine to me. There are a few calibration questions around uptake but we can resolve these in the new data update.

Only issue is I cannot approve the merge request due to the branch conflicts flagged by GitHub. Looks like these need to be handle via command line. Does anyone have any experience dealing with these conflicts like this?

ldxib2 commented 3 weeks ago

you can do it from the command line or in the Github desktop manager also, it will just open the 2 versions of the document that the conflicts are on and highlight for you the difference between the HEAD branch and the incoming update from the branch you are merging. Then you choose or edit for each one, it's quite straight forward in the desktop. However, I'm unsure why some of the things are registered as conflicts as they seem like any other update

jp-camecon commented 3 weeks ago

Can you

you can do it from the command line or in the Github desktop manager also, it will just open the 2 versions of the document that the conflicts are on and highlight for you the difference between the HEAD branch and the incoming update from the branch you are merging. Then you choose or edit for each one, it's quite straight forward in the desktop. However, I'm unsure why some of the things are registered as conflicts as they seem like any other update

Can you please then resolve these conflicts Ian so we can carry out the merge then?

ldxib2 commented 3 weeks ago

If you're happy for me to do that then sure! I will have a look tomorrow.

ldxib2 commented 2 weeks ago

Hi both, apologies for the delay. I wasn't able to resolve the conflicts with the main branch without actually merging and I assumed you final say on it and I'm not even sure I can as I'm not a registered reviewer. So I copied the current main branch to a new one, pr_merge_main_base, then merged it see the conflicts and resolve them. Here is a summary: Conflicts: get_exog - kept main version gitignore - kept main version (includes REXX) inititalise_csv - kept main version, no real changes ftt_lcoe_p - kept main version, changes in PR seem uneccessary and maybe older? and assuming these will be overwritten in alignment changes convert_csvs - taken incoming PR changes, appears more recent with extra conditions related to last edits of inputs VariableListing.csv - hard to see conflicts due to being csv, but appears to just be updated histend for MEWG

We can either make a new PR for this new branch that should go straight into main, or if you want to complete this PR you can assign me as a reviewer and I can redo the changes assuming you are ok with them?

Let me know your thoughts. Cheers!