JGCRI / gcamdata

The GCAM data system
https://jgcri.github.io/gcamdata/
Other
43 stars 26 forks source link

Missing logit.type column in logit table #897

Closed kdorheim closed 6 years ago

kdorheim commented 6 years ago

In the new data system any data frame containing logit information aka a column name with pattern logit.xxx needs to also contain the a logit.type column. The logit.type column was not required in the old data system.

However 41/82 logit tables in the new data system are missing the logit.type column. See table below where the logit.table column indicates T/F if the ds output is a logit table and the missing logit.type column indicates if the logit.type column is missing from the table.

There is an easy fix!! mutate(logit.type = X) must be added to the relevant chunks in the data system, where X reflects the appropriate logit.tpye from the old data system.

row name output missing logit.type logit.table
1 module_aglu_L202.an_input L202.Supplysector_in TRUE TRUE
2 module_aglu_L202.an_input L202.SubsectorAll_in TRUE TRUE
3 module_aglu_L202.an_input L202.Supplysector_an TRUE TRUE
4 module_aglu_L202.an_input L202.SubsectorAll_an TRUE TRUE
5 module_energy_L221.en_supply L221.Supplysector_en TRUE TRUE
6 module_energy_L221.en_supply L221.SubsectorLogit_en TRUE TRUE
7 module_energy_L222.en_transformation L222.Supplysector_en TRUE TRUE
8 module_energy_L222.en_transformation L222.SubsectorLogit_en TRUE TRUE
9 module_energy_L223.electricity L223.Supplysector_elec TRUE TRUE
10 module_energy_L223.electricity L223.SubsectorLogit_elec TRUE TRUE
11 module_energy_L225.hydrogen L225.Supplysector_h2 TRUE TRUE
12 module_energy_L225.hydrogen L225.SubsectorLogit_h2 TRUE TRUE
13 module_energy_L232.industry L232.Supplysector_ind TRUE TRUE
14 module_energy_L232.industry L232.SubsectorLogit_ind TRUE TRUE
15 module_energy_L2321.cement L2321.Supplysector_cement TRUE TRUE
16 module_energy_L2321.cement L2321.SubsectorLogit_cement TRUE TRUE
17 module_energy_L2322.Fert L2322.Supplysector_Fert TRUE TRUE
18 module_energy_L2322.Fert L2322.SubsectorLogit_Fert TRUE TRUE
19 module_energy_L252.transportation L252.Supplysector_trn TRUE TRUE
20 module_energy_L252.transportation L252.SubsectorLogit_trn TRUE TRUE
21 module_energy_L254.transportation_UCD L254.Supplysector_trn TRUE TRUE
22 module_energy_L254.transportation_UCD L254.tranSubsectorLogit TRUE TRUE
23 module_gcam.usa_L222.en_transformation_USA L222.Supplysector_en_USA TRUE TRUE
24 module_gcam.usa_L222.en_transformation_USA L222.SubsectorLogit_en_USA TRUE TRUE
25 module_gcam.usa_L223.electricity_USA L223.Supplysector_elec_FERC TRUE TRUE
26 module_gcam.usa_L223.electricity_USA L223.SubsectorLogit_elec_FERC TRUE TRUE
27 module_gcam.usa_L223.electricity_USA L223.Supplysector_elec_USA TRUE TRUE
28 module_gcam.usa_L223.electricity_USA L223.SubsectorLogit_elec_USA TRUE TRUE
29 module_gcam.usa_L2232.electricity_FERC_USA L2232.Supplysector_USAelec TRUE TRUE
30 module_gcam.usa_L2232.electricity_FERC_USA L2232.SubsectorLogit_USAelec TRUE TRUE
31 module_gcam.usa_L2232.electricity_FERC_USA L2232.Supplysector_elec_FERC TRUE TRUE
32 module_gcam.usa_L2232.electricity_FERC_USA L2232.SubsectorLogit_elec_FERC TRUE TRUE
33 module_gcam.usa_L2321.cement_USA L2321.Supplysector_cement_USA TRUE TRUE
34 module_gcam.usa_L2321.cement_USA L2321.SubsectorLogit_cement_USA TRUE TRUE
35 module_gcam.usa_L2322.Fert_USA L2322.SubsectorLogit_USAFert TRUE TRUE
36 module_gcam.usa_L2322.Fert_USA L2322.SubsectorLogit_Fert_USA TRUE TRUE
37 module_gcam.usa_L2322.Fert_USA L2322.Supplysector_Fert_USA TRUE TRUE
38 module_gcam.usa_L254.transportation_USA L254.Supplysector_trn_USA TRUE TRUE
39 module_gcam.usa_L254.transportation_USA L254.tranSubsectorLogit_USA TRUE TRUE
40 module_gcam.usa_L261.carbon_storage_USA L261.Supplysector_C_USA TRUE TRUE
41 module_gcam.usa_L261.carbon_storage_USA L261.SubsectorLogit_C_USA TRUE TRUE
kdorheim commented 6 years ago

Starting with rows 1 - 21.

kdorheim commented 6 years ago

@bpbond can you do 22-41? thanks!

kdorheim commented 6 years ago

FYI @bpbond

In the new data system the logit.type might have been dropped during the write_to_all_regions call to prevent that from happening by adding "logit.type" after the LEVEL2_DATA_NAMES[["SomeSectorSomething"]]

df %>%
write_to_all_regions(c(LEVEL2_DATA_NAMES[[["SomeSomething"]], "logit.type"), GCAM_region_names) ->
the_problem_df


If the logit.type needs to be added but it is not obvious what the logit.type is in the old data system, check get_logit_fn_tables call. The first get_logit_fn_tables argument should contain the logit.type.

For example L202.SectorLogitTables_in.

L202.SectorLogitTables_in <- get_logit_fn_tables( A_an_input_supplysector, names_SupplysectorLogitType,
    base.header="Supplysector_", include.equiv.table=T, write.all.regions=T )

> A_an_input_supplysector
         supplysector output.unit input.unit price.unit logit.exponent logit.type
1  FodderHerb_Residue          Mt         Mt   1975$/kg           -1.5         NA
2 Pasture_FodderGrass          Mt         Mt   1975$/kg           -1.5         NA
3           FeedCrops          Mt         Mt   1975$/kg           -1.5         NA
4    Scavenging_Other          Mt         Mt   1975$/kg           -1.5         NA
kdorheim commented 6 years ago

@bpbond when #898 is merged the number of files missing the logit.type column will decrease. See below of a list of the files that will still need to be changed.

index name output missing logit.type logit.table
1 module_gcam.usa_L222.en_transformation_USA L222.Supplysector_en_USA TRUE TRUE
2 module_gcam.usa_L223.electricity_USA L223.Supplysector_elec_FERC TRUE TRUE
3 module_gcam.usa_L223.electricity_USA L223.SubsectorLogit_elec_FERC TRUE TRUE
4 module_gcam.usa_L2232.electricity_FERC_USA L2232.Supplysector_USAelec TRUE TRUE
5 module_gcam.usa_L2232.electricity_FERC_USA L2232.SubsectorLogit_USAelec TRUE TRUE
6 module_gcam.usa_L2232.electricity_FERC_USA L2232.Supplysector_elec_FERC TRUE TRUE
7 module_gcam.usa_L2232.electricity_FERC_USA L2232.SubsectorLogit_elec_FERC TRUE TRUE
8 module_gcam.usa_L2322.Fert_USA L2322.SubsectorLogit_USAFert TRUE TRUE
9 module_gcam.usa_L261.carbon_storage_USA L261.Supplysector_C_USA TRUE TRUE
10 module_gcam.usa_L261.carbon_storage_USA L261.SubsectorLogit_C_USA TRUE TRUE