Electricity sector emissions are not showing up in state-level queries (GCAM-USA)

[chloe-fauvel]

Hi all,

The newest version of GCAM-USA was able to solve on UVA's Rivanna high-performance computer, but there is an issue of the queries not outputting state-level emissions for the electricity sector. I updated all query_*.xml files using the Main_queries.xml template under gcam-core/output/queries/ directory. For example, CO2_emissions_by_tech_states outputs electricity sector emissions for Global regions, but not state regions (e.g. VA). I am attaching example files below. query_CO2_emissions_by_tech_states.xml.txt xmldb_batch_CO2_emissions_by_tech_states.xml.txt

run_model.sh.txt

We believe there might have been a change to libraries used by the ModelInterface which I call in run_model.sh, which then runs the queries. Or do we need to do some additional processing due to a changed nesting structure for the electricity sector? Any help would be greatly appreciated, thank you!

[mbins]

Hi Chloe,

This is related to the new subsector nesting structure in the GCAM-USA 5.4 electricity sector. You simply need to add one more slash after *[@type='subsector'] in your xpath query (see below). This tells the xpath to look one additional nest down for the technology node, if needed.

<xPath buildList="true" dataName="emissions" group="false" sumAll="false">*[@type = 'sector' ]/*[@type='subsector']//*[@type='technology']//CO2/emissions/node()</xPath>

FYI, this change will likely apply to other generic "by tech" queries. We updated the GCAM-USA specific electricity queries to be compatible with the new subsector nesting structure in the electricity sector, but did not update the general queries like CO2 emissions by tech, inputs by tech, etc.

Best,

Matthew

[chloe-fauvel]

Hi Matthew,

Thank you for the quick response! This did indeed fix the issue and I now see electricity sector emissions. However, the formatting of the output files doesn't line up.

Looking at the same example of CO2 emissions by tech...The columns line up correctly for global emissions

But not for state emissions

It appears it just pushes back the values by one column

Would you have any advice on how to fix this? Thank you again.

[mbins]

Hi Chloe,

Can you send the exact query that you used? I can't currently replicate this issue on my end. Also, it seems from your screenshots and previous query file that you may be running each region individually and then concatenating the files afterwards. Is that right? If so, that could be part of the issue.

Best,

Matthew

[chloe-fauvel]

Hi Matthew,

I am not quite sure what you mean by running each region individually and then concatenating. Here is the query file we have used: query_CO2_emissions_by_tech_states.xml.txt

Due to the new electricity subsector nesting structure, we also tried this query file: query_CO2_emissions_by_tech_states_shree.xml.txt

And it gave us errors that resembled the following:

Before Function: 1628017127229
Completed 96.42857142857143% of batch queries.
About to perform query: collection()/scenario[ (@name='nodac_netzero_vcea_usa_global' and @date='2021-30-7T16:01:24-04:00')rld/region[ (@name='Argentina') ]/*[@type = 'sector' ]/*[@type='nesting-subsector']/*[@type='subsector']/
            CO2/emissions/node()
In Function: 1628017127230
Before Function: 1628017127242
java.util.concurrent.ExecutionException: java.lang.Exception: The query returned no results.Completed
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)97.61904761904762
% of batch queries.     at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at ModelInterface.ModelGUI2.BatchWindow$FutureQueryTask.get(BatchWindow.java:169)
        at ModelInterface.ModelGUI2.BatchWindow.doCsvOutput(BatchWindow.java:594)
About to perform query: collection()/scenario[ (@name='nodac_netzero_vcea_usa_global' and @date='2021-30-7T16:01:24-04:00')rld/region[ (@name='Colombia') ]/*[@type = 'sector' ]/*[@type='nesting-subsector']/*[@type='subsector']/
            CO2/emissions/node()        at ModelInterface.ModelGUI2.BatchWindow.access$100(BatchWindow.java:105)
        at ModelInterface.ModelGUI2.BatchWindow$3.run(BatchWindow.java:383)
In Function: 1628017127242Caused by: java.lang.Exception: The query returned no results.
        at ModelInterface.ModelGUI2.tables.ComboTableModel.buildTable(ComboTableModel.java:914)
        at ModelInterface.ModelGUI2.tables.ComboTableModel.<init>(ComboTableModel.java:806)
        at ModelInterface.ModelGUI2.BatchWindow$FutureQueryTask.call(BatchWindow.java:195)
        at ModelInterface.ModelGUI2.BatchWindow$FutureQueryTask.call(BatchWindow.java:135)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at ModelInterface.ModelGUI2.BatchWindow$FutureQueryTask.run(BatchWindow.java:181)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
java.util.concurrent.ExecutionException: java.lang.Exception: The query returned no results.
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at ModelInterface.ModelGUI2.BatchWindow$FutureQueryTask.get(BatchWindow.java:169)
        at ModelInterface.ModelGUI2.BatchWindow.doCsvOutput(BatchWindow.java:594)
        at ModelInterface.ModelGUI2.BatchWindow.access$100(BatchWindow.java:105)
        at ModelInterface.ModelGUI2.BatchWindow$3.run(BatchWindow.java:383)
Caused by: java.lang.Exception: The query returned no results.
        at ModelInterface.ModelGUI2.tables.ComboTableModel.buildTable(ComboTableModel.java:914)
        at ModelInterface.ModelGUI2.tables.ComboTableModel.<init>(ComboTableModel.java:806)
        at ModelInterface.ModelGUI2.BatchWindow$FutureQueryTask.call(BatchWindow.java:195)
        at ModelInterface.ModelGUI2.BatchWindow$FutureQueryTask.call(BatchWindow.java:135)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at ModelInterface.ModelGUI2.BatchWindow$FutureQueryTask.run(BatchWindow.java:181)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Completed java.util.concurrent.ExecutionException: java.lang.Exception: The query returned no results.
98.80952380952381% of batch queries.
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at ModelInterface.ModelGUI2.BatchWindow$FutureQueryTask.get(BatchWindow.java:169)
        at ModelInterface.ModelGUI2.BatchWindow.doCsvOutput(BatchWindow.java:594)
        at ModelInterface.ModelGUI2.BatchWindow.access$100(BatchWindow.java:105)
        at ModelInterface.ModelGUI2.BatchWindow$3.run(BatchWindow.java:383)
Caused by: java.lang.Exception: The query returned no results.
        at ModelInterface.ModelGUI2.tables.ComboTableModel.buildTable(ComboTableModel.java:914)
        at ModelInterface.ModelGUI2.tables.ComboTableModel.<init>(ComboTableModel.java:806)
        at ModelInterface.ModelGUI2.BatchWindow$FutureQueryTask.call(BatchWindow.java:195)
        at ModelInterface.ModelGUI2.BatchWindow$FutureQueryTask.call(BatchWindow.java:135)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at ModelInterface.ModelGUI2.BatchWindow$FutureQueryTask.run(BatchWindow.java:181)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Completed java.util.concurrent.ExecutionException: java.lang.Exception: The query returned no results.
100.0   at java.util.concurrent.FutureTask.report(FutureTask.java:122)% of batch queries.

        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at ModelInterface.ModelGUI2.BatchWindow$FutureQueryTask.get(BatchWindow.java:169)
        at ModelInterface.ModelGUI2.BatchWindow.doCsvOutput(BatchWindow.java:594)
        at ModelInterface.ModelGUI2.BatchWindow.access$100(BatchWindow.java:105)
        at ModelInterface.ModelGUI2.BatchWindow$3.run(BatchWindow.java:383)
Caused by: java.lang.Exception: The query returned no results.
        at ModelInterface.ModelGUI2.tables.ComboTableModel.buildTable(ComboTableModel.java:914)
        at ModelInterface.ModelGUI2.tables.ComboTableModel.<init>(ComboTableModel.java:806)
        at ModelInterface.ModelGUI2.BatchWindow$FutureQueryTask.call(BatchWindow.java:195)
        at ModelInterface.ModelGUI2.BatchWindow$FutureQueryTask.call(BatchWindow.java:135)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at ModelInterface.ModelGUI2.BatchWindow$FutureQueryTask.run(BatchWindow.java:181)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
WARNING; Batch queries finished with 84 errors.
INFO; Finished running batch file
exited exe_5

Thank you, Chloe

[mbins]

Hi Chloe,

I don't think the syntax in query_CO2_emissions_by_tech_states_shree.xml.txt is correct - if you look at elec_segments_water_USA.xml, you'll see that nesting-subsector is the higher nest, not the lower one. That query might work if you use subsector twice, although I haven't tried it.

By "running each region individually and then concatenating," I meant that you actually have a separate query for each region in query_CO2_emissions_by_tech_states.xml.txt, which are seemingly then combined (concatenated) into a single .csv file. It seems to me that this process of chaining together each of the individual queries in query_CO2_emissions_by_tech_states.xml.txt to a single .csv file may be leading to the mis-match in the way the columns are printed. For instance, I get one subsector column when I use the updated query I shared earlier on one of of the 32 core GCAM regions, two subsector columns when running on a state-level region, and one subsector column when running a core region and state-level region simultaneously. Listing all the regions within the same <aQuery> tag in your query file, or using the <all-regions/> tag instead of specifying each region individually, might resolve the issue with the column mismatch you're currently experiencing.

Best,

Matthew