JGCRI / gcamwrapper

An R / Python wrapper to interactively run a GCAM scenario
https://jgcri.github.io/gcamwrapper/
Other
9 stars 3 forks source link

How should I use set_data() ? #29

Open nihongi opened 3 months ago

nihongi commented 3 months ago

I was trying to change the value of labor productivity, but I couldn't get the result I expected. For example, I changed the labor productivity of Africa_Eastern in 1975 to 0.003 and applied it with set_data(), but it was unchanged in GCAM. I tried a variety of query strings. In some cases, the value in GCAM was changed to something but not what I expected. Please tell me the correct way to use set_data().

C:\Users\nihon>python
Python 3.9.6 (tags/v3.9.6:db3ff76, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, platform
>>> os.add_dll_directory("C:/Program Files/RedHat/java-1.8.0-openjdk-1.8.0.372-1/bin")
<AddedDllDirectory('C:/Program Files/RedHat/java-1.8.0-openjdk-1.8.0.372-1/bin')>
>>> os.add_dll_directory("C:/Program Files/RedHat/java-1.8.0-openjdk-1.8.0.372-1/jre/bin/server")
<AddedDllDirectory('C:/Program Files/RedHat/java-1.8.0-openjdk-1.8.0.372-1/jre/bin/server')>
>>> os.add_dll_directory("C:/gcam-v6.0/gcam-core/exe")
<AddedDllDirectory('C:/gcam-v6.0/gcam-core/exe')>
>>> import gcamwrapper
>>> g = gcamwrapper.Gcam("configuration_ref.xml", "C:/gcam-v6.0/gcam-core/exe")
Running GCAM model code base version 6.0 revision gcam-v6.0
Configuration file:  configuration_ref.xml
Parsing input files...
Parsing ../input/gcamdata/xml/hector.xml scenario component.
Parsing ../input/gcamdata/xml/socioeconomics_gSSP2.xml scenario component.
Parsing ../input/gcamdata/xml/resources.xml scenario component.
Parsing ../input/gcamdata/xml/en_supply.xml scenario component.
Parsing ../input/gcamdata/xml/en_transformation.xml scenario component.
Parsing ../input/gcamdata/xml/electricity_water.xml scenario component.
Parsing ../input/gcamdata/xml/heat.xml scenario component.
Parsing ../input/gcamdata/xml/hydrogen.xml scenario component.
Parsing ../input/gcamdata/xml/en_distribution.xml scenario component.
Parsing ../input/gcamdata/xml/other_industry.xml scenario component.
Parsing ../input/gcamdata/xml/other_industry_incelas_gssp2.xml scenario component.
Parsing ../input/gcamdata/xml/iron_steel.xml scenario component.
Parsing ../input/gcamdata/xml/iron_steel_incelas_gssp2.xml scenario component.
Parsing ../input/gcamdata/xml/Off_road.xml scenario component.
Parsing ../input/gcamdata/xml/Off_road_incelas_gssp2.xml scenario component.
Parsing ../input/gcamdata/xml/chemical.xml scenario component.
Parsing ../input/gcamdata/xml/chemical_incelas_gssp2.xml scenario component.
Parsing ../input/gcamdata/xml/aluminum.xml scenario component.
Parsing ../input/gcamdata/xml/aluminum_incelas_gssp2.xml scenario component.
Parsing ../input/gcamdata/xml/cement.xml scenario component.
Parsing ../input/gcamdata/xml/cement_incelas_gssp2.xml scenario component.
Parsing ../input/gcamdata/xml/en_Fert.xml scenario component.
Parsing ../input/gcamdata/xml/HDDCDD_constdd_no_GCM.xml scenario component.
Parsing ../input/gcamdata/xml/building_det.xml scenario component.
Parsing ../input/gcamdata/xml/transportation_UCD_CORE.xml scenario component.
Parsing ../input/gcamdata/xml/Ccoef.xml scenario component.
Parsing ../input/gcamdata/xml/Cstorage.xml scenario component.
Parsing ../input/gcamdata/xml/ag_For_Past_bio_base_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/ag_cost_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/ag_prodchange_ref_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/resbio_input_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/an_input.xml scenario component.
Parsing ../input/gcamdata/xml/ag_Fert_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/land_input_1.xml scenario component.
Parsing ../input/gcamdata/xml/land_input_2.xml scenario component.
Parsing ../input/gcamdata/xml/land_input_3_IRR.xml scenario component.
Parsing ../input/gcamdata/xml/land_input_4_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/land_input_5_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/protected_land_input_2.xml scenario component.
Parsing ../input/gcamdata/xml/protected_land_input_3.xml scenario component.
Parsing ../input/gcamdata/xml/ag_an_demand_input.xml scenario component.
Parsing ../input/gcamdata/xml/bio_trade.xml scenario component.
Parsing ../input/gcamdata/xml/ag_trade.xml scenario component.
Parsing ../input/gcamdata/xml/unlimited_water_supply.xml scenario component.
Parsing ../input/gcamdata/xml/water_supply_constrained.xml scenario component.
Parsing ../input/gcamdata/xml/desalination.xml scenario component.
Parsing ../input/gcamdata/xml/water_td.xml scenario component.
Parsing ../input/gcamdata/xml/EFW_input_coefs.xml scenario component.
Parsing ../input/gcamdata/xml/EFW_irrigation.xml scenario component.
Parsing ../input/gcamdata/xml/EFW_manufacturing.xml scenario component.
Parsing ../input/gcamdata/xml/EFW_municipal.xml scenario component.
Parsing ../input/gcamdata/xml/ag_water_input_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/electricity_water_coefs.xml scenario component.
Parsing ../input/gcamdata/xml/water_demand_industry.xml scenario component.
Parsing ../input/gcamdata/xml/water_demand_livestock.xml scenario component.
Parsing ../input/gcamdata/xml/water_demand_municipal.xml scenario component.
Parsing ../input/gcamdata/xml/water_demand_primary.xml scenario component.
Parsing ../input/gcamdata/xml/ind_urb_processing_sectors.xml scenario component.
Parsing ../input/gcamdata/xml/all_energy_emissions.xml scenario component.
Parsing ../input/gcamdata/xml/all_fgas_emissions.xml scenario component.
Parsing ../input/gcamdata/xml/all_unmgd_emissions.xml scenario component.
Parsing ../input/gcamdata/xml/all_aglu_emissions_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/all_protected_unmgd_emissions.xml scenario component.
Parsing ../input/gcamdata/xml/emission_factor_controls.xml scenario component.
Parsing ../input/gcamdata/xml/all_energy_emissions_MAC.xml scenario component.
Parsing ../input/gcamdata/xml/all_fgas_emissions_MAC.xml scenario component.
Parsing ../input/gcamdata/xml/all_aglu_emissions_IRR_MGMT_MAC.xml scenario component.
Parsing ../input/gcamdata/xml/ind_urb_processing_sectors_MAC.xml scenario component.
Parsing ../input/gcamdata/xml/liquids_limits.xml scenario component.
Parsing ../input/gcamdata/xml/water_elec_liquids_limits.xml scenario component.
Parsing ../input/gcamdata/xml/negative_emissions_budget_gSSP2.xml scenario component.
Parsing ../input/gcamdata/xml/bio_externality.xml scenario component.
Parsing ../input/gcamdata/xml/onshore_wind.xml scenario component.
Parsing ../input/solution/cal_broyden_config.xml scenario component.
XML parsing complete.
Starting new scenario: Reference
Fri Apr 12 19:36:29 2024:WARNING:Hector::Logger::printLogHeader: hector version 2.5.0
Fri Apr 12 19:36:29 2024:WARNING:Hector::Core::setData: Disabling onelineocean
Fri Apr 12 19:36:29 2024:WARNING:Hector::Logger::printLogHeader: hector version 2.5.0
Fri Apr 12 19:36:29 2024:WARNING:Hector::Core::setData: Disabling onelineocean
Fri Apr 12 19:36:29 2024:WARNING:Hector::Core::prepareToRun: Disabling onelineocean
>>> labor_prod_query = gcamwrapper.get_query("socioeconomic", "labor_productivity")
>>> g.run_period_pre(5)
Starting a model run. Running period 4
Model run beginning.
Period 0: 1975
Model solved with last period's prices.
Period 1: 1990
Model solved normally. Iterations period 1: 242. Total iterations: 243
Period 2: 2005
Model solved normally. Iterations period 2: 214. Total iterations: 457
Period 3: 2010
Model solved normally. Iterations period 3: 170. Total iterations: 626
Period 4: 2015
Model solved normally. Iterations period 4: 221. Total iterations: 847
All model periods solved correctly.
Model run completed.
Period 5: 2020
>>> labor_prod_org = g.get_data(labor_prod_query)
>>> labor_prod_org.head()
           region  year  laborproductivity
0  Africa_Eastern  1975            0.00154
1  Africa_Eastern  1990            0.00154
2  Africa_Eastern  2005            0.01216
3  Africa_Eastern  2010            0.03926
4  Africa_Eastern  2015            0.02085
>>> labor_prod_change = labor_prod_org.copy()
>>> type(labor_prod_change)
<class 'pandas.core.frame.DataFrame'>
>>> labor_prod_change.at[0, 'laborproductivity'] = 0.003
>>> labor_prod_change.head()
           region  year  laborproductivity
0  Africa_Eastern  1975            0.00300
1  Africa_Eastern  1990            0.00154
2  Africa_Eastern  2005            0.01216
3  Africa_Eastern  2010            0.03926
4  Africa_Eastern  2015            0.02085
>>> g.set_data(labor_prod_change, labor_prod_query, region=["+", "="], year=["+", "="])
>>> labor_prod_out1 = g.get_data(labor_prod_qery)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'labor_prod_qery' is not defined
>>> labor_prod_out1 = g.get_data(labor_prod_query)
>>> labor_prod_out1.head()
           region  year  laborproductivity
0  Africa_Eastern  1975            0.00154
1  Africa_Eastern  1990            0.00154
2  Africa_Eastern  2005            0.01216
3  Africa_Eastern  2010            0.03926
4  Africa_Eastern  2015            0.02085
>>> g.set_data(labor_prod_change, labor_prod_query, region=["=", "Africa_Eastern"], year=["=", 1975])
>>> labor_prod_out2 = g.get_data(labor_prod_query)
>>> labor_prod_out2.head()
           region  year  laborproductivity
0  Africa_Eastern  1975            0.00154
1  Africa_Eastern  1990            0.00154
2  Africa_Eastern  2005            0.01216
3  Africa_Eastern  2010            0.03926
4  Africa_Eastern  2015            0.02085
>>> g.set_data(labor_prod_change, labor_prod_query, region=["=", "*"], year=["=", "*"])
>>> labor_prod_out3 = g.get_data(labor_prod_query)
>>> labor_prod_out3.head()
           region  year  laborproductivity
0  Africa_Eastern  1975            0.00154
1  Africa_Eastern  1990            0.00154
2  Africa_Eastern  2005            0.01216
3  Africa_Eastern  2010            0.03926
4  Africa_Eastern  2015            0.02085
>>> g.set_data(labor_prod_change, labor_prod_query, region=["-", "*"], year=["-", "*"])
>>> labor_prod_out4 = g.get_data(labor_prod_query)
>>> labor_prod_out4.head()
           region  year  laborproductivity
0  Africa_Eastern  1975            0.00577
1  Africa_Eastern  1990            0.00577
2  Africa_Eastern  2005            0.00577
3  Africa_Eastern  2010            0.00577
4  Africa_Eastern  2015            0.00577
>>>
nihongi commented 3 months ago

I solved the problem partially. When I set region=["-", "=", "Africa_Eastern"], year=["-", "=", 2020], it worked. Still I don't know how to use ["+", "="]. I think this can be used for setting multiple values ​​at the same time, but not successfull for me yet.

C:\Users\nihon>python
Python 3.9.6 (tags/v3.9.6:db3ff76, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, platform
>>> os.add_dll_directory("C:/Program Files/RedHat/java-1.8.0-openjdk-1.8.0.372-1/bin")
<AddedDllDirectory('C:/Program Files/RedHat/java-1.8.0-openjdk-1.8.0.372-1/bin')>
>>> os.add_dll_directory("C:/Program Files/RedHat/java-1.8.0-openjdk-1.8.0.372-1/jre/bin/server")
<AddedDllDirectory('C:/Program Files/RedHat/java-1.8.0-openjdk-1.8.0.372-1/jre/bin/server')>
>>> os.add_dll_directory("C:/gcam-v6.0/gcam-core/exe")
<AddedDllDirectory('C:/gcam-v6.0/gcam-core/exe')>
>>> import gcamwrapper
>>> g = gcamwrapper.Gcam("configuration_ref.xml", "C:/gcam-v6.0/gcam-core/exe")
Running GCAM model code base version 6.0 revision gcam-v6.0
Configuration file:  configuration_ref.xml
Parsing input files...
Parsing ../input/gcamdata/xml/hector.xml scenario component.
Parsing ../input/gcamdata/xml/socioeconomics_gSSP2.xml scenario component.
Parsing ../input/gcamdata/xml/resources.xml scenario component.
Parsing ../input/gcamdata/xml/en_supply.xml scenario component.
Parsing ../input/gcamdata/xml/en_transformation.xml scenario component.
Parsing ../input/gcamdata/xml/electricity_water.xml scenario component.
Parsing ../input/gcamdata/xml/heat.xml scenario component.
Parsing ../input/gcamdata/xml/hydrogen.xml scenario component.
Parsing ../input/gcamdata/xml/en_distribution.xml scenario component.
Parsing ../input/gcamdata/xml/other_industry.xml scenario component.
Parsing ../input/gcamdata/xml/other_industry_incelas_gssp2.xml scenario component.
Parsing ../input/gcamdata/xml/iron_steel.xml scenario component.
Parsing ../input/gcamdata/xml/iron_steel_incelas_gssp2.xml scenario component.
Parsing ../input/gcamdata/xml/Off_road.xml scenario component.
Parsing ../input/gcamdata/xml/Off_road_incelas_gssp2.xml scenario component.
Parsing ../input/gcamdata/xml/chemical.xml scenario component.
Parsing ../input/gcamdata/xml/chemical_incelas_gssp2.xml scenario component.
Parsing ../input/gcamdata/xml/aluminum.xml scenario component.
Parsing ../input/gcamdata/xml/aluminum_incelas_gssp2.xml scenario component.
Parsing ../input/gcamdata/xml/cement.xml scenario component.
Parsing ../input/gcamdata/xml/cement_incelas_gssp2.xml scenario component.
Parsing ../input/gcamdata/xml/en_Fert.xml scenario component.
Parsing ../input/gcamdata/xml/HDDCDD_constdd_no_GCM.xml scenario component.
Parsing ../input/gcamdata/xml/building_det.xml scenario component.
Parsing ../input/gcamdata/xml/transportation_UCD_CORE.xml scenario component.
Parsing ../input/gcamdata/xml/Ccoef.xml scenario component.
Parsing ../input/gcamdata/xml/Cstorage.xml scenario component.
Parsing ../input/gcamdata/xml/ag_For_Past_bio_base_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/ag_cost_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/ag_prodchange_ref_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/resbio_input_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/an_input.xml scenario component.
Parsing ../input/gcamdata/xml/ag_Fert_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/land_input_1.xml scenario component.
Parsing ../input/gcamdata/xml/land_input_2.xml scenario component.
Parsing ../input/gcamdata/xml/land_input_3_IRR.xml scenario component.
Parsing ../input/gcamdata/xml/land_input_4_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/land_input_5_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/protected_land_input_2.xml scenario component.
Parsing ../input/gcamdata/xml/protected_land_input_3.xml scenario component.
Parsing ../input/gcamdata/xml/ag_an_demand_input.xml scenario component.
Parsing ../input/gcamdata/xml/bio_trade.xml scenario component.
Parsing ../input/gcamdata/xml/ag_trade.xml scenario component.
Parsing ../input/gcamdata/xml/unlimited_water_supply.xml scenario component.
Parsing ../input/gcamdata/xml/water_supply_constrained.xml scenario component.
Parsing ../input/gcamdata/xml/desalination.xml scenario component.
Parsing ../input/gcamdata/xml/water_td.xml scenario component.
Parsing ../input/gcamdata/xml/EFW_input_coefs.xml scenario component.
Parsing ../input/gcamdata/xml/EFW_irrigation.xml scenario component.
Parsing ../input/gcamdata/xml/EFW_manufacturing.xml scenario component.
Parsing ../input/gcamdata/xml/EFW_municipal.xml scenario component.
Parsing ../input/gcamdata/xml/ag_water_input_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/electricity_water_coefs.xml scenario component.
Parsing ../input/gcamdata/xml/water_demand_industry.xml scenario component.
Parsing ../input/gcamdata/xml/water_demand_livestock.xml scenario component.
Parsing ../input/gcamdata/xml/water_demand_municipal.xml scenario component.
Parsing ../input/gcamdata/xml/water_demand_primary.xml scenario component.
Parsing ../input/gcamdata/xml/ind_urb_processing_sectors.xml scenario component.
Parsing ../input/gcamdata/xml/all_energy_emissions.xml scenario component.
Parsing ../input/gcamdata/xml/all_fgas_emissions.xml scenario component.
Parsing ../input/gcamdata/xml/all_unmgd_emissions.xml scenario component.
Parsing ../input/gcamdata/xml/all_aglu_emissions_IRR_MGMT.xml scenario component.
Parsing ../input/gcamdata/xml/all_protected_unmgd_emissions.xml scenario component.
Parsing ../input/gcamdata/xml/emission_factor_controls.xml scenario component.
Parsing ../input/gcamdata/xml/all_energy_emissions_MAC.xml scenario component.
Parsing ../input/gcamdata/xml/all_fgas_emissions_MAC.xml scenario component.
Parsing ../input/gcamdata/xml/all_aglu_emissions_IRR_MGMT_MAC.xml scenario component.
Parsing ../input/gcamdata/xml/ind_urb_processing_sectors_MAC.xml scenario component.
Parsing ../input/gcamdata/xml/liquids_limits.xml scenario component.
Parsing ../input/gcamdata/xml/water_elec_liquids_limits.xml scenario component.
Parsing ../input/gcamdata/xml/negative_emissions_budget_gSSP2.xml scenario component.
Parsing ../input/gcamdata/xml/bio_externality.xml scenario component.
Parsing ../input/gcamdata/xml/onshore_wind.xml scenario component.
Parsing ../input/solution/cal_broyden_config.xml scenario component.
XML parsing complete.
Starting new scenario: Reference
Mon Apr 15 16:59:24 2024:WARNING:Hector::Logger::printLogHeader: hector version 2.5.0
Mon Apr 15 16:59:24 2024:WARNING:Hector::Core::setData: Disabling onelineocean
Mon Apr 15 16:59:24 2024:WARNING:Hector::Logger::printLogHeader: hector version 2.5.0
Mon Apr 15 16:59:24 2024:WARNING:Hector::Core::setData: Disabling onelineocean
Mon Apr 15 16:59:24 2024:WARNING:Hector::Core::prepareToRun: Disabling onelineocean
>>> labor_prod_query = gcamwrapper.get_query("socioeconomic", "labor_productivity")
>>> g.run_period_pre(6)
Starting a model run. Running period 5
Model run beginning.
Period 0: 1975
Model solved with last period's prices.
Period 1: 1990
Model solved normally. Iterations period 1: 242. Total iterations: 243
Period 2: 2005
Model solved normally. Iterations period 2: 214. Total iterations: 457
Period 3: 2010
Model solved normally. Iterations period 3: 170. Total iterations: 626
Period 4: 2015
Model solved normally. Iterations period 4: 221. Total iterations: 847
Period 5: 2020
Model solved normally. Iterations period 5: 888. Total iterations: 1735
All model periods solved correctly.
Model run completed.
Period 6: 2025
>>> labor_prod_org = g.get_data(labor_prod_query)
>>> labor_prod_org.head(10)
           region  year  laborproductivity
0  Africa_Eastern  1975            0.00154
1  Africa_Eastern  1990            0.00154
2  Africa_Eastern  2005            0.01216
3  Africa_Eastern  2010            0.03926
4  Africa_Eastern  2015            0.02085
5  Africa_Eastern  2020            0.01952
6  Africa_Eastern  2025            0.02871
7  Africa_Eastern  2030            0.04133
8  Africa_Eastern  2035            0.04117
9  Africa_Eastern  2040            0.04335
>>> labor_prod_change = labor_prod_org[(labor_prod_org['region'] == "Africa_Eastern") & (labor_prod_org['year'] == 2020)].copy()
>>> labor_prod_change.at[5, 'laborproductivity'] = 0.04
>>> labor_prod_change.head(10)
           region  year  laborproductivity
5  Africa_Eastern  2020               0.04
>>> g.set_data(labor_prod_change, labor_prod_query, region=["-", "=", "Africa_Eastern"], year=["-", "=", 2020])
>>> labor_prod_new = g.get_data(labor_prod_query)
>>> labor_prod_new.compare(labor_prod_org)
  laborproductivity
               self    other
5              0.04  0.01952
>>> labor_prod_new.head(10)
           region  year  laborproductivity
0  Africa_Eastern  1975            0.00154
1  Africa_Eastern  1990            0.00154
2  Africa_Eastern  2005            0.01216
3  Africa_Eastern  2010            0.03926
4  Africa_Eastern  2015            0.02085
5  Africa_Eastern  2020            0.04000
6  Africa_Eastern  2025            0.02871
7  Africa_Eastern  2030            0.04133
8  Africa_Eastern  2035            0.04117
9  Africa_Eastern  2040            0.04335
>>>
whitenightZhang commented 1 month ago

@nihongi hi, did you compile gcamwrapper successfully using windows? Can I ask you some questions?

nihongi commented 1 month ago

@whitenightZhang Yes, I did successfully compile gcamwrapper on Windows. Any questions are welcome.