search

JGCRI / gcam-core

GCAM -- The Global Change Analysis Model
http://jgcri.github.io/gcam-doc/
Other
275 stars 162 forks source link

Error reading linked GHG emissions markets #88

Closed robbieorvis closed 4 years ago

robbieorvis commented 4 years ago

Hello,

I'm trying to set a global carbon cap and have the carbon market prices applied to non-CO2 gases.

I set up the carbon market in its own file. I'm separately trying to read in the linked markets, following the instructions online, but am getting an error about unrecognized text strings when parsing ghgmarket.

I've never tried linking markets before, so probably doing something wrong. Any help would be great, thanks.

(also, like price and demand adjustments are using AR5 GWPs).

linked carbon cap.zip

pralitp commented 4 years ago

I think you are close. Looking at your XML files you are using the name="CO2" for two purposes:

However GCAM could only have one at a time, thus the messages about not being able to set linked GHG parameters into a ghg policy. I think what you are looking for is to have some other name for the constraint, typically we use something like GHG but you could use anything as long as you don't have naming clashes. So your XMLs would look like:

        <region name="USA">
            <ghgpolicy name="GHG">
                <market>global</market>
                    <constraint year="2025">8631</constraint>
                    <constraint year="2030">6473</constraint>
                    <constraint year="2035">4315</constraint>                   
                    <constraint year="2040">2158</constraint>

And the linked policy:

         <linked-ghg-policy name="CO2">
            <price-adjust fillout="1" year="1975">1</price-adjust>
            <demand-adjust fillout="1" year="1975">3.666666667</demand-adjust>
            <market>global</market>
            <linked-policy>GHG</linked-policy>
            <price-unit>1990$/tC</price-unit>
            <output-unit>MtC</output-unit>
         </linked-ghg-policy>

With this being said. There are some words of caution about directly pricing (i.e. price-adjust != 0) the non-CO2 gasses:

  1. The MAC curves will still be active by default so you will get emissions reductions in CH4_AGR even without pricing it directly.
  2. While we have in principal added high/low management technologies (read fertilizer application) in the Ag model we have not partitioned the non-CO2 emissions coefficients (i.e. the same for both) for CH4_AGR etc so it still doesn't really know how to "structurally" reduce emissions.

There is a bit more details about this in the documentation. Therefore I generally don't recommend doing it unless you will be explicitly putting in the emissions coefficients / turn off the MACs as appropriate.

robbieorvis commented 4 years ago

Thanks. This is very helpful.

A few clarifying questions.

1) When the constraint is set on "GHG", I assume the model is constraining all GHGs (not just CO2), and then pricing CO2, the non-CO2 gases, and then running them through the MAC curves, based on the constraint in CO2e. Is that correct? If not, can you explain how setting the constraint on "GHG" works compared to setting on "CO2"? If my assumption is correct and I'm trying to linearly decrease to zero, then the constraint needs to be calculated based on total GHGs, not just CO2 (as it is now).

2) In the linked policy file, how does GCAM know to link to the "GHG" policy? As I understand it, the name of the linked policy goes into "", which ties back to . If this is the case, then what is specified in ""? Or, is GCAM just taking the price of the CO2 market from the GHG policy and then applying the price to the MAC curves for the other gases?

3) Stepping back - my ultimate goal here is to reduce global GHG emissions to zero by 2050. If that can be done solely with the constraint, then no need for the linked policy here. But I had been thinking this may not be directly feasible, and instead trying to reduce CO2 to zero and then use the CO2 price to reduce emissions of non-CO2 gases as well. It may be that that approach is not ideal, however.

Thanks again!

pralitp commented 4 years ago

Maybe answering 2. will give some clarity on the rest of the questions:

To be clear GCAM doesn't know what a "GHG" is. Instead we are just using that name to create a market to bundle a set of gasses as we configure. It could just have easily been called something else such as "CO2e" etc.

When we create a constraint policy such as:

        <region name="USA">
            <ghgpolicy name="GHG">
                <market>global</market>
                    <constraint year="2025">8631</constraint>

It will create a market a market called "GHG" and for each period for which we have specified a constraint it will solve for a PriceG such that DemandG <= SupplyG. Where SupplyG = constraint. However we have not yet said how DemandG is calculated.

When we create a a linked GHG market such as:

         <linked-ghg-policy name="CO2">
            <price-adjust fillout="1" year="1975">1</price-adjust>
            <demand-adjust fillout="1" year="1975">3.666666667</demand-adjust>
            <market>global</market>
            <linked-policy>GHG</linked-policy>
            <price-unit>1990$/tC</price-unit>
            <output-unit>MtC</output-unit>
         </linked-ghg-policy>

We are tying the "CO2" market to the "GHG" market (as it is set as the value in the <linked-policy> tag) such that: Pricei = PriceG Price-adjusti DemandG = SUM(DemandI Demand-adjustI) For each gas i linked to G in your linked GHG policy XML file.

And finally just for completeness GCAM computes the demand for gasses such as CO2 or CH4 etc by calculating emissions specified through the Ag/energy system with tags like:

<CO2 name="CO2" />
<Non-CO2 name="CH4">
    <emiss-coef>0.1737528684</emiss-coef>
    <output-driver/>
    <mac-control name="Natural Gas and Oil Systems">
        <mac-reduction tax="0">0.27</mac-reduction>
        <market-name>CO2</market-name>
        <mac-reduction tax="5">0.286</mac-reduction>
        <mac-reduction tax="10">0.298</mac-reduction>

Stepping back - my ultimate goal here is to reduce global GHG emissions to zero by 2050. If that can be done solely with the constraint, then no need for the linked policy here. But I had been thinking this may not be directly feasible, and instead trying to reduce CO2 to zero and then use the CO2 price to reduce emissions of non-CO2 gases as well. It may be that that approach is not ideal, however.

From that perspective, yes you should update the values in your constraint to be reflective of the GWP weighted sum of all emissions you want to include and decline that to zero in 2050. Including all of the gasses and not just CO2 will result in CO2 essentially goes negative in 2050 to allow emissions from other gasses that can't be reduced or are too costly.

robbieorvis commented 4 years ago

Okay, thanks. I think I've got this figured out now, thanks to this explanation.

One follow-up. I'm now trying to setup a market for HFC gases only, and I think I've got that figured out using the steps above but only linking the HFC gases to the market.

This seems to be working to drive down HFCs (so long as there is sufficient room to decrease gases from the existing HFCs with MACCs), but I'm also seeing a significant decrease in emissions of other gases as well in later years (CO2, CH4, etc...), even though they aren't linked specifically. My linked-policy name is still "GHG" but I don't think that should matter. Is it possible that the cost of the reductions is driving down energy consumption? If so, what is the relationship between MAC curves for non-CO2 and energy consumption in terms of price feedbacks? If not, do you have any ideas why this is happening?

Thanks.

pkyle commented 4 years ago

It's hard to say without seeing the specific numbers, but my guess would be that the policy (presumably a constraint on HFCs) is maxing out the relevant MAC curves and still above the target, at which point the only way to reduce the HFC emissions is to reduce the underlying activity. The reduction in the activity levels then decreases emissions of non-target gases. If this is the case, then you should see high prices of sectors that produce HFCs (e.g., "industry"), and the policy/reference change in non-target GHGs should be greatest in these sectors.

robbieorvis commented 4 years ago

@pkyle thanks. Indeed I'm seeing residential cooling and industrial energy use decrease significantly (e.g. 25%), so it does appear as if that's what's happening. I hadn't considered that the model might just reduce demand to meet the constraint; I thought it would only work within the bounds of the MACs or else not solve.

How do the preloaded MACs scale in future years beyond 2020 or 2030 (when the original EPA dataset ended)? Also, any chance that the new EPA dataset released this fall will be incorporated anytime in the near future? In general, seems like much higher potential emissions abatement in the newer study with significant cost and reduction potential implications.

ssmithClimate commented 4 years ago

@robbieorvis If you have price-adjust not equal to 0 (see comment below), then there is also an economic feedback to these technologies that will reduce their emissions. If you set price-adjust=0, then there will be no economic feedback, and emissions will only reduce due to MAC curves.

pkyle commented 4 years ago

This all sounds correct. Note that if you do set price-adjust to 0, then there isn't any guarantee that the model will be able to solve the HFC constraint. The MAC curves all have end points, beyond which any further price increases don't translate to reduced emissions rates. These are written in the all_fgas_emissions.xml file. Many of the technologies therein do have maximum mac-reduction of ~98%, which should work well for a constraint policy, but several (e.g., semiconductor manufacturing emissions of C2F6) have pretty low reduction percentages at all carbon prices. In this case in particular, the semi-conductor manufacturing industry is not modeled separately from the industrial sector as a whole. As such, even if price-adjust is set to 1, an increase in the HFC emissions price can only reduce emissions (beyond the levels indicated in the MAC curve) by reducing the output from industry as a whole.

robbieorvis commented 4 years ago

Thanks @pkyle and @ssmithClimate.

I'm wondering if I still may be doing something wrong. I've attached the two files I'm using for this specific policy, which is limiting HFCs as discussed above.

I'm not imposing a very strict constraint, and yet the model is failing to solve, which makes me think something is off in these files.

Two things that would be helpful to know:

What units should the constraint be set in? I am under the impression this should be the same units that I'm converting to in in the linked policy file, which in this case would be MtCO2. In the attached linked file, the conversions are all for converting from tC to MtCO2e using GWPs from AR5 (I am also aware that not all of the gases in the attached have MAC, but I don't think that matters if there is sufficient "room" to decrease emissions using the MACs for the covered gases)

Is there any other conversion I am missing in either of these files?

Thanks. HFC_cap.zip

ssmithClimate commented 4 years ago

Two quick points. The GHG Macs are set-up to use carbon prices as their input units. You can use a different price unit (say $/tHFCxxxeq), but then you would need to add a price conversion value into the MAC curves, so its most straightforward just to keep the market operating in terms of carbon prices.

I should amend my comment above slightly, if you don't want to actually keep price-adjust = 0, you will need to set a value to price-adjust appropriate to convert carbon prices to prices in terms of tonnes of the specific emission species. (You'll see this is the case for CH4 and N2O in the linked policy file distributed with core GCAM. Note that price-adjust = 0 for fluoridated gases in that file, so that was set up assuring that F-gases would only be reduced through MAC curves.)

pkyle commented 4 years ago

Well I just learned something--thanks @ssmithClimate ! To turn on the endogenous price response for these nonCO2 GHGs, the price-adjust should be set equal to the hundred-year GWP multiplied by 12/44. Just setting it to 1 won't return the correct response. (I know, this doesn't address the issue of the model not solving when it's set to 0...will try to get to that later!)

rjplevin commented 4 years ago

It would be super helpful if these nuggets of knowledge were copied into the model documentation as they were revealed!

pkyle commented 4 years ago

@rjplevin I totally agree; make a PR! https://github.com/JGCRI/gcam-doc.git @robbieorvis The plot does thicken, somewhat...here's the issue. If any of you out there have experience with this, feel free to chime in b/c this is all new to me! Here is some XML of a relevant technology, from the debug file:

                            <emiss-coef>14.9737</emiss-coef>
                            <input-emissions>0</input-emissions>
                            <control-adjusted-emiss-coef>14.9737</control-adjusted-emiss-coef>
                            <output-driver></output-driver>
                            <mac-control name="Refrigeration and Air Conditioning">
                                <reduction>0</reduction>
                                <mac-reduction tax="0">0</mac-reduction>
                                <mac-reduction tax="10">0.0328219</mac-reduction>
                                <mac-reduction tax="25">0.361951</mac-reduction>
                                <mac-reduction tax="50">0.71957</mac-reduction>
                                <mac-reduction tax="100">0.900471</mac-reduction>
                                <mac-reduction tax="200">0.977864</mac-reduction>
                                <tech-change year="1975" fillout="1">0</tech-change>
                                <market-name>CO2</market-name>
                                <no-zero-cost-reductions>0</no-zero-cost-reductions>
                                <tech-change>0</tech-change>
                            </mac-control>

First, note that emiss-coef is equal to control-adjusted-emiss-coef. What that indicates is that the MAC control function isn't applied, and the reason why is that the MAC function itself is applied to the market-name of CO2. And, since your market is actually called HFC, the MACs aren't adjusting the emissions at all. Going back to all_fgas_emissions.xml, the market-name of CO2 is actually specified here, which is good. And that can be traced back to a value in constants.R. I'm throwing in the towel on this issue for the day, but what I'd recommend is, at least for the purposes of this exercise which is only focused on abating HFCs, just replace the following value in constants.R: emissions.MAC_MARKET <- "HFC" # Default market that MAC curves will look for

ssmithClimate commented 4 years ago

If you're using a market name other than CO2, you do need to specify this in the mac-curve objects, as noted in the GCAM docs.

Looks like emissions.MAC_MARKET does, indeed, change this variable - but that changes it for all MAC's, and you'll need to make sure you're not activating MAC curves that you don't want to. So you will want to do a separate csv->xml conversion just for the HFCs with this market set to what you need it to be.

pkyle commented 4 years ago

Oops, thanks @ssmithClimate ! @robbieorvis please disregard the fix I offered above. The way I recommend to do this is not to change the emissions.MAC_MARKET object in constants.R, but rather to make a new object in constants.R: emissions.HFC_MAC_MARKET <- "HFC" And then in zchunk_L252.MACC.R, in line 203 (in the code block that creates the object L252.MAC_higwp), replace emissions.MAC_MARKET with emissions.HFC_MAC_MARKET. Note that this will make the MAC functions apply to all F gas emissions, including e.g. SF6 that weren't included in the policy file you sent.

robbieorvis commented 4 years ago

Thanks @ssmithClimate and @pkyle. I think I've finally got it figured out.

Using a combination of @rjplevin's pygcam and custom mods to input files, I'm overwriting the market names for the covered chemicals (those that are HFCs AND have MAC curves) to limit reductions to those gases. In calculating the constraint, I'm also going to have to limit it to those covered gases. Then, ensuring I set price-adjust to 0 and demand-adjust to the GWP for the covered gases in the linked-policy file.

So, if we were to improve the documentation for how to setup a market for non-CO2 gases, the steps would be:

1) Create a policy tax or constraint file. Per @pralitp's comments above, you cannot leave the name of the market as CO2 if you intend to use the linked-policy file, because that creates an issue. So, rename the market to something like GHG, HFC, etc... The constraint value should be in MtCO2e, not MtC (or rather, the constraint value should be based on how you setup the demand-adjust values in the linked file - if you are converting to MtCO2e, then the constraint should be in MtCO2e)

2) Create a linked-ghg policy file. Only include the gases you want covered by the constraint. If you want GCAM to only use MAC curves (i.e. prevent GCAM from reducing energy demand as a way to meet the constraint), set price-adjust to 0. If you want to allow this behavior, set price-adjust to the GWP*12/44. The linked-policy value should be the name of the market created in step 1.

3) In the file with the MAC curve for the relevant gases, ensure that the market name for the gases you want covered matches the market name in the the constraint/tax and linked-ghg policy files. Note that if the same market is specified for gases you don't want included, GCAM will still allow those MACs to contribute to emissions reductions. In other words, you need a unique market name, and that market name should only be used in the MACs for gases you actually want covered.

4) If the model fails to solve and you've set all this up correctly, odds are the constraint is too stringent and the model cannot find enough reductions within the MACs.

@ssmithClimate and @pkyle please feel free to correct any of that as you see fit.

I really appreciate that you both have been so helpful here.

I do want to add one more comment, which is to say that while I know some of this is "documented" (e.g. when you tell me it's in the docs and link to the page), the issue is that the existing documentation is often still extremely unclear to anyone who is not intimately familiar with the inner workings of GCAM. This is not limited to this issue in particular, but more broadly to a lot of the documentation for GCAM. While that documentation is indeed helpful, I would encourage you to get feedback on future doc updates from external model users to help you validate how clear and useful the documentation is (I would be happy to volunteer for this). It would likely save a lot of time addressing issue threads like this one if the documentation were clearer and basically a step-by-step guide for users. That being said, I know you all have a lot on your plates and limited time and that you can't do everything at once.

Thanks again for your help.

I'll leave this open in case you want to make any tweaks to the steps I outlined above, but I think this is otherwise resolved.

robbieorvis commented 4 years ago

I finally got this to work, so will be closing this issue. Steps below:

1) Setup emissions constraint XML file (I used MtCO2e as the unit here).

2) Setup linked emissions XML file. I set the price-adjust value to 0 for all entries and the demand-adjust value to the GWP. I only included the gases I wanted covered in the markets. I named the linked market "HFC" in both the constraint and linked emissions market

3) Setup an updated all_fgas_emissions.xml file, with updated market names for the covered chemicals. Here the markets for the covered gases needed to reference HFC, but the market(s) for the non-covered gases had to be something different, otherwise the model could reduce those gases as a way to meet the constraint. I kept non-covered gases in the CO2 market and covered gases in the HFC market.

4) Update abatement potential for covered gases. I noticed that several regions (e.g. India and Indonesia) with quite significant emissions of HFCs in later years, particularly for residential and commercial cooling, had zero abatement potential programmed into GCAM. In practice, however, these countries are party to HFC reduction commitments, such as the Kigali Agreement, so this omission puts an artificial constraint on those countries' ability to reduce HFCs. I carried over the reduction potential/cost data from other regions for certain chemicals and end uses to allow for reductions. While this may overstate potential reductions from certain regions, it is certainly better than not including any potential reductions whatsoever. Note that this was tricky to do because it appears that within the GCAM Data System, the determination on which MACs to exclude is handled internally within the R files based on whether or not a region has emissions of a certain pollutant in the base year, and not any kind of flag on an input CSV (which would have been much easier).

Cermouth commented 2 years ago

Hi @robbieorvis , thanks for your discussion here. I also met with the issue of "Linked market CO2 in USA could not be linked to GHG." but am having trouble with it. Would you like to share the files that finally worked? Very grateful for your help!

moulai commented 2 years ago

I also encountered the problem mentioned by @Cermouth. I would greatly appreciate any help with resolving this issue!

Hi @robbieorvis , thanks for your discussion here. I also met with the issue of "Linked market CO2 in USA could not be linked to GHG." but am having trouble with it. Would you like to share the files that finally worked? Very grateful for your help!

robbieorvis commented 2 years ago

It's been quite a while since I did this, but I believe this set of files is what I got to work. The comment above from 1/3/2020 outlines the procedure used for these files and why I needed to do each of these. GCAM_China_GHG_Markets.zip

moulai commented 2 years ago

It's been quite a while since I did this, but I believe this set of files is what I got to work. The comment above from 1/3/2020 outlines the procedure used for these files and why I needed to do each of these. GCAM_China_GHG_Markets.zip

Thank you for your help! I have found a bug in my settings file: I didn't add the ghgpolicy field in Cap.xml, so it couldn't find the corresponding market. When I fixed this, the model worked fine.

Cermouth commented 2 years ago

Thanks for your help!