JGCRI / gcamreader

Python package for reading GCAM output databases
Other
4 stars 5 forks source link

Problem about duplicate in results when query "elec gen by gen tech" #26

Closed YangchunB closed 3 years ago

YangchunB commented 3 years ago

When querying "elec gen by gen tech", the results will be duplicated and do not match the results of the model interface. I find the following problems have appeared in both wind and solar technology queries. I don't know how to avoid the following problems:

image

Here is the queries XML:

<?xml version="1.0" encoding="UTF-8"?>
<queries>
     <aQuery>
     <all-regions/>
               <supplyDemandQuery title="elec gen by gen tech">
                    <axis1 name="technology">technology</axis1>
                    <axis2 name="Year">physical-output[@vintage]</axis2>
                    <xPath buildList="true" dataName="output" group="false" sumAll="false">*[@type='sector' (:collapse:) and
               (@name='electricity' or @name='elect_td_bld' or @name='industrial energy use')]/
               *[@type='subsector' and (@name='wind')]/*[@type='technology' and not (@name='electricity' or @name='elect_td_bld')]/
               *[@type='output' and (@name='electricity' or @name='elect_td_bld')]/
               physical-output/node()</xPath>
                    <comments/>
                </supplyDemandQuery>
     </aQuery>
</queries>
pralitp commented 3 years ago

Indeed this is a bug which is pretty fundamental: The data needs to be aggregated. It would affect any results which is querying "vintaged" technologies and we have collapsed the vintaging structure (i.e. the dimension that needs to be aggregated).

This fix is simple and I'll put that in now. In the mean time you can do:

data = conn.runQuery(queries[0])
cols = data.columns
data_agg = data.groupby(cols.drop("value").to_list(), as_index=False).sum()

The data_agg should match the output from the Model Interface which was doing the aggregation.

YangchunB commented 3 years ago

Thank you for your information. It's very helpful to me.