search

RMI-PACTA / workflow.transition.monitor

This repository is used to create the docker image that runs the transition monitor platform.
Other
2 stars 0 forks source link

config: point `pa2024ch` to data prep with new weo 2023 scenarios #333

Closed jdhoffa closed 2 months ago

jdhoffa commented 2 months ago

The newly created data directory now contains APS and STEPS scenarios for WEO 2023 for the aviation, cement and steel sectors.

Importantly also, we had to update paths to the new (real) peer files, as those had to be re-run with those same latest scenarios.

With these scenarios missing, the executive summary couldn't render correctly.

Relates to: RMI-PACTA/workflow.data.preparation/pull/238 RMI-PACTA/workflow.data.preparation/pull/237 RMI-PACTA/workflow.data.preparation/pull/236 RMI-PACTA/pacta.scenario.data.preparation/pull/61 RMI-PACTA/pacta.scenario.data.preparation/pull/63

jdhoffa commented 2 months ago

Open question to @AlexAxthelm and @cjyetman do you think I should update the GENERAL config as well? I don't think it is strictly necessary to do so... BUT I also can't think fo a strong reason NOT to.

It might make life less confusing down the road to keep both initiatives with the same data version.

cjyetman commented 2 months ago

I think that we have said that whatever happens with WEO will happen everywhere (PA2024CH, GENERAL, and all 2023Q4 initiatives), and I expect that word has traveled around, so I think we should follow through on that.

Additionally, until this issue is resolved, I prefer that we don't get into using different 2023Q4 datasets for different initiatives: https://github.com/RMI-PACTA/workflow.data.preparation/issues/226

jdhoffa commented 2 months ago

Cool! done

jdhoffa commented 2 months ago

Going to wait for checks to pass and inspect the outputs prior to merging

cjyetman commented 2 months ago

presumably if #331 merges and this PR branch is updated, the tests will pass? so we're waiting on that, no?

jdhoffa commented 2 months ago

Indeed, depends on #331

I will re-trigger CI/CD once that is merged.

github-actions[bot] commented 2 months ago

Docker build status

commit_time git_sha project_code holdings_date language peer_group report summary download_command image
2024-07-17T09:30:50Z 0c40429499920129d00695140234cccf92ab4961, GENERAL 2022Q4 EN other az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports/rmi_pacta_2022q4_general-20240717T142911Z/EN/other/2/working_dir/50_Outputs/rmi_pacta_2022q4_general --destination rmi_pacta_2022q4_general-EN-other transitionmonitordockerregistry.azurecr.io/rmi_pacta_2022q4_general:20240717T142911Z
2024-07-17T09:30:50Z 0c40429499920129d00695140234cccf92ab4961, GENERAL 2023Q4 EN other az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports/rmi_pacta_2023q4_general-20240717T142911Z/EN/other/2/working_dir/50_Outputs/rmi_pacta_2023q4_general --destination rmi_pacta_2023q4_general-EN-other transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_general:20240717T142911Z
2024-07-17T09:30:50Z 0c40429499920129d00695140234cccf92ab4961, PA2024CH 2023Q4 DE bank az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/DE/bank/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-DE-bank transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z
2024-07-17T09:30:50Z 0c40429499920129d00695140234cccf92ab4961, PA2024CH 2023Q4 DE other az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/DE/other/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-DE-other transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z
2024-07-17T09:30:50Z 0c40429499920129d00695140234cccf92ab4961, PA2024CH 2023Q4 EN assetmanager az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/EN/assetmanager/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-EN-assetmanager transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z
2024-07-17T09:30:50Z 0c40429499920129d00695140234cccf92ab4961, PA2024CH 2023Q4 EN bank az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/EN/bank/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-EN-bank transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z
2024-07-17T09:30:50Z 0c40429499920129d00695140234cccf92ab4961, PA2024CH 2023Q4 EN insurance az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/EN/insurance/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-EN-insurance transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z
2024-07-17T09:30:50Z 0c40429499920129d00695140234cccf92ab4961, PA2024CH 2023Q4 EN other az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/EN/other/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-EN-other transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z
2024-07-17T09:30:50Z 0c40429499920129d00695140234cccf92ab4961, PA2024CH 2023Q4 EN pensionfund az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/EN/pensionfund/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-EN-pensionfund transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z
2024-07-17T09:30:50Z 0c40429499920129d00695140234cccf92ab4961, PA2024CH 2023Q4 FR bank az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/FR/bank/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-FR-bank transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z
2024-07-17T09:30:50Z 0c40429499920129d00695140234cccf92ab4961, PA2024CH 2023Q4 FR other az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/FR/other/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-FR-other transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z
2024-07-17T09:30:50Z 0c40429499920129d00695140234cccf92ab4961 ghcr.io/rmi-pacta/workflow.transition.monitor:pr333
cjyetman commented 2 months ago

🤔 https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports/rmi_pacta_2023q4_pa2024ch-20240702T122952Z/EN/assetmanager/1/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch/executive_summary/template.pdf

Screenshot 2024-07-02 at 15 22 32
jdhoffa commented 2 months ago

shit!

jdhoffa commented 2 months ago

Logs for this run can be found here: https://portal.azure.com/#view/Microsoft_Azure_Storage/ContainerMenuBlade/~/overview/storageAccountId/%2Fsubscriptions%2Ffeef729b-4584-44af-a0f9-4827075512f9%2FresourceGroups%2FRMI-SP-PACTA-DEV%2Fproviders%2FMicrosoft.Storage%2FstorageAccounts%2Fpactadatadev/path/ghactions-workflow-transition-monitor-results-full/etag/%220x8DC3D63D9990D41%22/defaultEncryptionScope/%24account-encryption-key/denyEncryptionScopeOverride~/false/defaultId//publicAccessVal/None

jdhoffa commented 2 months ago

error_messages.txt was helpful XD

2024-07-02 13:16:23.402782 ES: There was an error in prep/plot_scores(). Returning empty plot object.

2024-07-02 13:16:23.616256 ES: There was an error in prep/plot_scores(). Returning empty plot object.

Which is actually super bizarre since that isn't even the expected error message... https://github.com/RMI-PACTA/pacta.executive.summary/blob/14575e1433e229154b84049d9a6b9c406e9d8366/inst/extdata/PA2024CH_en_exec_summary/scorecard.Rmd#L129-L146

This might be a cache invalidation thing.

AlexAxthelm commented 2 months ago

error_messages.txt was helpful XD

2024-07-02 13:16:23.402782 ES: There was an error in prep/plot_scores(). Returning empty plot object.

2024-07-02 13:16:23.616256 ES: There was an error in prep/plot_scores(). Returning empty plot object.

Which is actually super bizarre since that isn't even the expected error message... https://github.com/RMI-PACTA/pacta.executive.summary/blob/14575e1433e229154b84049d9a6b9c406e9d8366/inst/extdata/PA2024CH_en_exec_summary/scorecard.Rmd#L129-L146

This might be a cache invalidation thing.

I think you might be looking at the wrong lines. The message you're getting is at line 354 & 377

jdhoffa commented 2 months ago

https://github.com/RMI-PACTA/pacta.executive.summary/blob/14575e1433e229154b84049d9a6b9c406e9d8366/inst/extdata/PA2024CH_en_exec_summary/scorecard.Rmd#L354

and

https://github.com/RMI-PACTA/pacta.executive.summary/blob/14575e1433e229154b84049d9a6b9c406e9d8366/inst/extdata/PA2024CH_en_exec_summary/scorecard.Rmd#L377

don't have anything interesting on them? Are we talking about different files?

jdhoffa commented 2 months ago

@MonikaFu FYI, the problem seems to be that some of the sectors yield NA scores When I run:

data_scores_b <- prep_scores(
      results_portfolio = results_portfolio,
      peers_results_aggregated = peers_results_aggregated,
      asset_class = "bonds",
      scenario_source = scenario_source
    )

I get:

Screenshot 2024-07-03 at 13 26 09

or

# A tibble: 14 × 5
   asset_class scope     entity         sector     score
   <chr>       <chr>     <chr>          <chr>      <chr>
 1 bonds       portfolio peers          NA         E    
 2 bonds       sector    peers          automotive C    
 3 bonds       sector    peers          aviation   NA   
 4 bonds       sector    peers          cement     NA   
 5 bonds       sector    peers          coal       E    
 6 bonds       sector    peers          gas        E    
 7 bonds       sector    peers          oil        E    
 8 bonds       sector    peers          power      E    
 9 bonds       sector    peers          steel      NA   
10 bonds       portfolio this_portfolio NA         A+   
11 bonds       sector    this_portfolio gas        A    
12 bonds       sector    this_portfolio oil        C    
13 bonds       sector    this_portfolio power      A+   
14 bonds       sector    this_portfolio steel      E

This seems to be what is causing the plot error.

jdhoffa commented 2 months ago

This is the error I get:

Error in `abort_if_invalid_values()` at pacta.executive.summary/R/plot_scores.R:227:3:
! Each value of `"sector"` must be one of these:
NA, power, automotive, coal, oil, gas, aviation, steel.
✖ You passed: cement.
Run `rlang::last_trace()` to see where the error occurred.
jdhoffa commented 2 months ago

Probably solved by https://github.com/RMI-PACTA/pacta.executive.summary/pull/348

jdhoffa commented 2 months ago

@AlexAxthelm @cjyetman @MonikaFu After some sleuthing locally, I have determined that the scorecard data still won't generate since the peer data does NOT contain the updated scenarios for the emission sectors.

What that means we need to re-run the peer data (might already have been done by @AlexAxthelm , need to make sure the latest run contains STEPS, APS and NZE for the aviation, cement and steel sectors) and then either:

This runs in parallel to a discussion we've already been having about updating our peer data in the CI/CD pipeline.

Converting this PR to draft until the above has been completed.

jdhoffa commented 2 months ago

Latest bump of data_share_path now incorporates changes from https://github.com/RMI-PACTA/pacta.data.preparation/pull/32

jdhoffa commented 2 months ago

For the record: after a call with @AlexAxthelm we realized that we need the appropriate versions of peer_results and user_data also configured appropriately in this PR.

To that end, I have asked him to target #333 with the #340 PR, and will merge it that together with this PR (I will update the PR title and description appropriately).

I will also wait until @MonikaFu runs and updates the user_data (and get that on Azure` before re-running CI/CD here.

After all the above is complete, we should have a good idea of if all IR and ES building runs well "in the wild". I would expect all plots to render correctly.

jdhoffa commented 2 months ago

I can't / don't know how to review the test results anymore, but otherwise seems reasonable

To test you can run this command:

az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/EN/bank/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-EN-bank

However, since I believe you were never able to get the az cli up and running? It is also possible to access the files from the Azure GUI.

From portal.azure.com, navigate to the pactadatadev > Storage Browser > Blob containers > ghactions-workflow-transition-monitor-results-reports-private and then find the file path as usual rmi_pacta_2023q4_pa2024ch-20240717T142912Z/EN/bank/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-EN-bank

I know it's not the most intuitive but definitely possible if you have the gumption 😉