Add a new Ag Technology in GCAM.

[Wentemi]

Hi , I need help adding a new agriculture technology in GCAM v5.3. My ultimate goal is to incorporate a technology option called "Corn_NelsonR_IRR_tew", which becomes available after the calibration year I included the following lines in the land_input_5_IRR_MGMT.xml file :

<region name="USA">
         <LandAllocatorRoot name="root">
            <LandNode name="AgroForestLand_NelsonR">
               <LandNode name="AgroForest_NonPasture_NelsonR">
                  <LandNode name="CropLand_NelsonR">
                     <LandNode name="Corn_NelsonR">
                        <LandNode name="Corn_NelsonR_IRR">
                           <absolute-cost-logit>
                              <logit-exponent fillout="1" year="1975">2.5</logit-exponent>
                           </absolute-cost-logit>
------------------------- Truncated ---------------------------------
                           <LandLeaf name="Corn_NelsonR_IRR_tew">
                              <land-use-history>
                                 <allocation year="1700">0</allocation>
                                 <allocation year="1750">0</allocation>
                                 <allocation year="1800">0</allocation>
                                 <allocation year="1850">0</allocation>
                                 <allocation year="1900">0</allocation>
                                 <allocation year="1950">0</allocation>
                                 <allocation year="1975">0</allocation>
                                 <above-ground-carbon-density>0.605</above-ground-carbon-density>
                                 <below-ground-carbon-density>9</below-ground-carbon-density>
                              </land-use-history>
                              <landAllocation year="1975">0</landAllocation>
                              <landAllocation year="1990">0</landAllocation>
                              <landAllocation year="2005">0</landAllocation>
                              <landAllocation year="2010">0</landAllocation>
                              <landAllocation year="2015">0</landAllocation>
                              <minAboveGroundCDensity>0</minAboveGroundCDensity>
                              <minBelowGroundCDensity>0</minBelowGroundCDensity>
                              <land-carbon-densities>
                                 <above-ground-carbon-density>0.605</above-ground-carbon-density>
                                 <below-ground-carbon-density>9</below-ground-carbon-density>
                                 <mature-age fillout="1" year="1975">1</mature-age>
                              </land-carbon-densities>
                             </LandLeaf>
                        </LandNode>

In the ag_For_Past_bio_base_IRR_MGMT.xml file, I included the following lines :

<AgProductionTechnology name="Corn_NelsonR_IRR_tew">
                  <period year="1975">
                     <share-weight>1</share-weight>
                     <CalDataOutput>
                        <calOutputValue>0</calOutputValue>
                     </CalDataOutput>
                     <harvests-per-year>1</harvests-per-year>
                  </period>
                  <period year="1990">
                     <share-weight>1</share-weight>
                     <CalDataOutput>
                        <calOutputValue>0</calOutputValue>
                     </CalDataOutput>
                     <harvests-per-year>1</harvests-per-year>
                  </period>
                  <period year="2005">
                     <share-weight>1</share-weight>
                     <CalDataOutput>
                        <calOutputValue>0</calOutputValue>
                     </CalDataOutput>
                     <harvests-per-year>1</harvests-per-year>
                  </period>
                  <period year="2010">
                     <share-weight>1</share-weight>
                     <CalDataOutput>
                        <calOutputValue>0</calOutputValue>
                     </CalDataOutput>
                     <harvests-per-year>1</harvests-per-year>
                  </period>
                  <period year="2015">
                     <share-weight>1</share-weight>
                     <CalDataOutput>
                        <calOutputValue>0</calOutputValue>
                     </CalDataOutput>
                     <harvests-per-year>1.032963</harvests-per-year>
                  </period>
                  <period year="2020">
                     <harvests-per-year>1.032963</harvests-per-year>
                  </period>
------------- Truncated -----------------------------
                  <period year="2100">
                     <harvests-per-year>1.032963</harvests-per-year>
                  </period>
               </AgProductionTechnology>

In the ag_Fert_IRR_MGMT.xml file I included the following lines

<AgProductionTechnology name="Corn_NelsonR_IRR_tew">
                  <period year="1975">
                     <share-weight>1</share-weight>
                     <CalDataOutput>
                        <calOutputValue>0</calOutputValue>
                     </CalDataOutput>
                     <harvests-per-year>1</harvests-per-year>
                  </period>
                  <period year="1990">
                     <share-weight>1</share-weight>
                     <CalDataOutput>
                        <calOutputValue>0</calOutputValue>
                     </CalDataOutput>
                     <harvests-per-year>1</harvests-per-year>
                  </period>
                  <period year="2005">
                     <share-weight>1</share-weight>
                     <CalDataOutput>
                        <calOutputValue>0</calOutputValue>
                     </CalDataOutput>
                     <harvests-per-year>1</harvests-per-year>
                  </period>
                  <period year="2010">
                     <share-weight>1</share-weight>
                     <CalDataOutput>
                        <calOutputValue>0</calOutputValue>
                     </CalDataOutput>
                     <harvests-per-year>1</harvests-per-year>
                  </period>
                  <period year="2015">
                     <share-weight>1</share-weight>
                     <CalDataOutput>
                        <calOutputValue>0</calOutputValue>
                     </CalDataOutput>
                     <harvests-per-year>1.032963</harvests-per-year>
                  </period>
                  <period year="2020">
                     <harvests-per-year>1.032963</harvests-per-year>
                  </period>
                  <period year="2025">
                     <harvests-per-year>1.032963</harvests-per-year>
                  </period>
-------------- Truncated --------------------------
                <period year="2100">
                     <harvests-per-year>1.032963</harvests-per-year>
                  </period>
               </AgProductionTechnology>

In the ag_prodchange_ref_IRR_MGMT.xml file I include the following lines :

<AgProductionTechnology name="Corn_NelsonR_IRR_tew">
                  <period year="2020">
                     <agProdChange>0.0042</agProdChange>
                  </period>
                  <period year="2025">
                     <agProdChange>0.0041</agProdChange>
                  </period>
---------------- Truncated-----------------------------------------
                  <period year="2100">
                     <agProdChange>0.0013</agProdChange>
                  </period>
               </AgProductionTechnology>

I also made similar changes to the ag_water_input_IRR_MGMT.xml and ag_cost_IRR_MGMT.xml files.

When I run these files it looks like the technology is read into GCAM. However, the issue is that the new technology option is allotted 0 (see picture below).

I have tried the following options :

1. by including <isNewTechnology fillout="1" year="2020">1</isNewTechnology> tag in the land_input_5_IRR_MGMT.xml file.
2. add a very small land allocation <landAllocation year="2015">0.0001</landAllocation> All my attempts above did not work?
   Any ideas and help to fix this problem will be appreciated.

[realxinzhao]

Did you add production output in ag_For_Past_bio_base_IRR_MGMT.xml for the technology? Once you added in initial land allocation, you want to assign a corresponding production as well so that the initial yield is implied.

[pkyle]

This looks to be all correct, and 95% of what needs to be done--make sure to read in the tag in the land input file, leave the base-year land allocation at 0, and then in the ag_For_Past_bio_base_IRR_MGMT.xml file, you'll need to read in a yield. Because the land allocation and production are zero in the base years, the model can't calculate a yield on its own, so it's just defaulting to zero and the profit-rate is zero and it gets no share. In order to calculate the profit-rate, the model needs a yield. See what's done for bioenergy crops in ag_For_Past_bio_base_IRR_MGMT.xml and copy that structure. The other thing that ought to be read in is "ghost shares", or the share of the market that this element would get if it had the same profit-rate as the dominant thing it's competing against. Again see what's done for bioenergy in the land allocator (search for the string "ghost"; the parameters are ghost-unnormalized-share and is-ghost-share-relative), and write back here if anything isn't clear. Bioenergy is a little bit different because it's introduced competing against the crops, whereas this new technology seems like it's just entering at the lowest level in the competition.

[Wentemi]

Hi @pkyle and @realxinzhao, Your suggestions help tremendously, Thank you.

[Swati0201]

Hey, @pkyle and @realxinzhao, I want to add different rice varieties that may have the same yield but lower emissions and less water consumption. Do you know if the same files and tags will work in this situation?

[realxinzhao]

@pkyle has done something similar: adding C4 or Tree identifiers at the technology levels but not the market. E.g., you can see CornC4 being produced at region X basin X irrigation X technology levels while at the regional level, there is only one Corn market and one producer price for that (it then consumed domestically or being exported). So you can do something similar to differentiate rice at the tech levels but they all have the same market price (Rice).

I would suggest you testing this for one place first, e.g., adding a new rice variety in India IndusR basin. If you look at ag_For_Past_bio_base_IRR_MGMT.xml, e.g., India has 19 basins producing rice and the first one is IndusR (<AgSupplySubsector name="Rice_IndusR">). There are 4 techs in a basin and I guess what you want to do is split the current ones into 2 or more (e.g., high or low Soil Carbon). You need to make sure the total production won't change.

Apparently, that's step1 above for adding production. You will need to do the same to add/split land area, future yield growth, production cost, fertilizer inputs, residual biomass (straw for biomass), and emission factors. Here is how I find the xmls that need the change. In the XML folder in gcamdata (assuming you have run driver), find xmls with "C4_" identifier (so they will be the ones new identifiers are needed):

$ grep -rl "C4_" ag_cost_IRR_MGMT.xml ag_Fert_IRR_MGMT.xml ag_For_Past_bio_base_IRR_MGMT.xml ag_prodchange_ref_IRR_MGMT.xml ag_prodchange_ssp1_IRR_MGMT.xml ag_prodchange_ssp2_IRR_MGMT.xml ag_prodchange_ssp3_IRR_MGMT.xml ag_prodchange_ssp4_IRR_MGMT.xml ag_prodchange_ssp5_IRR_MGMT.xml ag_water_input_IRR_MGMT.xml all_aglu_emissions_IRR_MGMT.xml all_aglu_emissions_IRR_MGMT_MAC.xml land_input_4_IRR_MGMT.xml land_input_5_IRR_MGMT.xml MACC_TC_SSP1.xml MACC_TC_SSP2.xml MACC_TC_SSP5.xml prune_empty_ag.xml resbio_input_IRR_MGMT.xml

Not all these are used in your configuration. Key ones:

1. ag_cost_IRR_MGMT.xml: production cost
2. ag_Fert_IRR_MGMT.xml: fertilizer input-output coefficient
3. ag_For_Past_bio_base_IRR_MGMT.xml: production
4. ag_prodchange_ref_IRR_MGMT.xml: annual growth of yield
5. ag_water_input_IRR_MGMT.xml: water input input- output coefficient
6. all_aglu_emissions_IRR_MGMT.xml: emission factors
7. all_aglu_emissions_IRR_MGMT_MAC.xml: marginal abatement cost curve
8. land_input_4_IRR_MGMT.xml: logit exponent governing the tech transformation
9. land_input_5_IRR_MGMT.xml: land area
10. prune_empty_ag.xml: an be removed in configuration with no solution impacts
11. resbio_input_IRR_MGMT.xml: residual biomass assumptions

For all these, you can use existing assumptions if no better assumptions. If adjusting these for one place (e.g., IndusR) works, you can scale it up by tracing back in gcamdata code.

[Swati0201]

Thankyou @realxinzhao for your reply . Just one more doubt: I want this new variety as a low-emission technology in the future and not in the base year. The new variety is rather a different cultivation and management practice than a seed variety. I want the traditional variety to gradually phase out. So how do we deal with the emission factors, which are kept constant from the base year until 2100?

[pkyle]

Emissions factors of any technology can be set in any year. The tag name is emiss-coef and the unit is usually kg of gas (e.g., CH4) per kg of crop produced. While we normally read in input-emissions (Tg/yr) in the base years, and the model uses that to calculate an emissions factor that is passed to future years, one can also set an emissions factor exogenously and it will over-ride the default behavior. e.g.

                    <AgProductionTechnology name="Rice_IndusR_IRR_lo" nocreate="1">
                        <period year="2025">
                            <Non-CO2 name="CH4_AGR">
                                <emiss-coef>0.001</emiss-coef>
                                <output-driver/>
                            </Non-CO2>
                        </period>
                    </AgProductionTechnology>

[realxinzhao]

I want the traditional variety to gradually phase out. @Swati0201 You probably want to think about the driver of the phasing-out behavior as well, e.g, yield, cost, policy, preference, etc.

[Swati0201]

Thank you @pkyle and @realxinzhao.

[Swati0201]

Hi @pkyle @realxinzhao , While adding rice4 (arbitory name) at Indus basin level in ag_cost_IRR_MGMT.xml (very first try), I am facing this error (image). I assure you that all the tags are properly closed. Any suggestions, please?

[realxinzhao]

@Swati0201 I haven't seen this error before. Probably check if this also happens in other xmls or just this one, and do a syntax check. And try restarting XML Marker or copying it to other places.

[pkyle]

I also don't know and don't see anything obvious in the image; the file is ~150 MB which is typical for an aglu file but which might cause problems for XML marker. Perhaps try just a text editor.

[Swati0201]

Thankyou for the suggestion @pkyle and @realxinzhao . I have successfully added RiceDSR as a subsector under Rice in the supply side. a) I am now trying to make both Rice and RiceDSR compete and a partial phase-out of Rice as Rice DSR has a higher yield, lower cost of production and low emissions. The production shares in the base year are 99.99% for Rice and 0.01% for RiceDSR. Here I want to understand the role of logit as I tried changing it in A_agSupplySubsector.csv as well as forcing it in module_aglu_L2012.ag_For_Past_bio_input_irr_mgmt, and it does not seem to work (For, e.g. giving Rice as -12 and RiceDSR as -3). Also, on the documentation page, the logit exponent is only discussed for the demand side. Note that I want the final production to be the same, say 100 units, which was produced by Rice in the reference scenario, and now, by adding new tech Rice DSR, 80-90% of production gradually shifts towards it, and only 5-10% is left in normal Rice. I have made the nonLandVariblecost in ag_cost_IRR_MGMT of RiceDSR to be 50% less than Rice. But still in the model interface under query ag tech variable cost i see the cost of RiceDSR is higher than Rice. Why is that so? how to make the new technology profitable so the model picks it up for future years

[realxinzhao]

@Swati0201

Parameters in A_agSupplySubsector.csv indeed shouldn't affect anything. The competition could be adjusted through land changes.

GCAM calculates rental profit = (price - nonLandVariablecost - WaterFertizercosts)*yield This profit is calculated for both Rice and RiceDSR and the land competes based on their relative rental profits.

In land_input_3_IRR.xml there is a relative cost logit of 1.75 governing the flexibility of the land share change wrt to the relative rental profit changes. In short, increasing that value could probably help as it will imply that higher rental profit increases in RiceDSR (when you lower the cost) relative to Rice would encourage higher corresponding land share.

However, this would affect other crops as well since all crops are in this same nest (I think still true in your case).

In addition, you might consider some share-weight changes on the land side in land_input_4_IRR_MGMT.xml, similar to the ghost-unnormalized-share used in biomass there I guess. There might be a different tag name though? @pkyle.

[Swati0201]

Hi @realxinzhao, Thank you for the reply. I have tried changing the land node logit from this CSV, but this doesn't seem to have much effect.

And to put in the value in the above mentioned equation price for both rice and RiceDSR are same. I am using variable cost from ag_cost_IRR_MGMT. in which the cost for DSR is less than Rice. And i am taking input cost from ag_Fert_IRR_MGMT. The yield if more for DSR in the input csv Muller Yield (aglu>LDS folder). Still the new tech is not picking up over rice. So if you please help.

[realxinzhao]

@Swati0201 Note that only changes in future relative rental profit between the two affect future land shares of the two. That is, if the cost of RiceDSR is becoming lower, yield is becoming higher, relative to Rice over time, it's land share should increase relative to Rice as well. And this would be more responsive when the land logit exponent is larger. (there crop prices should be the same, right?)

Also the parameter I suggested was indeed one level up for CropLand which is probably 1.75 (both A_LandNode_logit.csv and A_LandNode_logit_irr.csv ) in your version (there will be some default parameter changes in the upcoming version). Just note that (1) if you change this in csv, it will be applied to all regions & basins (2) this will affect the competition of other crops as well.

BTW, the one you modified was for the competition of irrigation vs. rainfed for RiceSDR. However, there could be an option if you want to add additional RiceSDR technologies directly under Rice technologies. E.g., currently, rice production has 4 technologies (IRR &RDF x hi&lo). If RiceSDR technologies are added to there e.g., 8 Rice technologies in total, you can use that parameter to control the competition between Rice and RiceSDR more directly (but together with irrigation options).

But in your current framework, modifying crop-specific logit share-weights might also help (but less due to market competition)

[pkyle]

Just scanning this issue, the thing that I see that's likely causing problems is that the RiceDSR technology/node is being calibrated, but its base-year share is 0.01%. In the energy system this could be OK because the calibration-derived share-weights can be exogenously over-ridden in future years. But in AgLU we don't have exogenous control over the "profit-scaler" (parameter that corresponds to the share-weight, in the land allocator), and a node with 0.01% market share will have an extremely low profit-scaler which will mean its land share is always going to be at or near zero. What I'd recommend instead is to not calibrate it, and to introduce it as a new technology in similar fashion to biomass crops. Two new parameters will need to be assigned in the land allocator to bring it in: ghost-unnormalized-share and is-ghost-share-relative. The way they are applied in land_input_4_IRR_MGMT.xml shows how the new node should be brought in, and the same parameters can also be found in land_input_5_IRR_MGMT.xml which defines the competition between the new technologies of the new node (if there are any; e.g. RiceDSR_IRR vs. RiceDSR_RFD). As for specific values to assign to the upper-level node's ghost share, some experimentation will be needed; note that there's no stock effects or capital carryover in the land allocator, so any new land use type should probably be phased in over time. I think assigning a 1 means it will get the profit-scaler of the currently dominant node in the nest.

[realxinzhao]

Thanks, Page! It seems there won't be a tag/variable similar to ghost-unnormalized-share once the crop is introduced in the base year. I guess another option might be land constraints (https://jgcri.github.io/gcam-doc/policies_examples.html#land-constraint).

[Swati0201]

Thank you so much for your reply, @pkyle and @realxinzhao. This brings me so much clarity as I was trying many things, but the land shares were not changing. Thank you again, I just wanted to clarify one thing: this new technology, RiceDSR, has to compete with only traditional rice and has to phase it out (in short, it is a new way of growing rice). So, if you can bring a little bit more clarity to my concepts, biomass is added as another crop (sector and not subsector), which has its demand in the energy sector and its market and it equally competes with all other 16 crops for land. While in my case, I don't want any market or new crop on the demand side, ultimately, it has to feed in the rice market. So, I am a bit confused here at what point this will be different than biomass, and at what point the completion just remains between rice and RiceDSR with no effect on any other crop.

[pkyle]

Right, so this is a bit different than biomass, which is its own category under cropland. Here you'll be within Rice, and adding new nodes, leaves, and ag production technologies. So I guess the XML will look something like this, on the land input side (just focusing on the application of these parameters that are used to bring in new land use types):

                <LandNode name="AgroForestLand_IndusR">
                    <LandNode name="AgroForest_NonPasture_IndusR">
                        <LandNode name="CropLand_IndusR">
                            <LandNode name="Rice_IndusR">
                                <LandNode name="Rice_IndusR_DSR_IRR">
                                    <ghost-unnormalized-share year="2025">0.1</ghost-unnormalized-share>
                                    <ghost-unnormalized-share year="2030">0.2</ghost-unnormalized-share>
                                    <is-ghost-share-relative>1</is-ghost-share-relative>
                                    <LandLeaf name="Rice_IndusR_DSR_IRR_hi">
                                        <ghost-unnormalized-share year="2025">0.5</ghost-unnormalized-share>

[Swati0201]

Thank you @pkyle . This is helpful

[Swati0201]

Hi @pkyle @realxinzhao . When I try to create final xml, the tables i have created are not included. I tried finding solution here https://github.com/JGCRI/gcamdata/wiki/XML-Conversion , but wasn't successful. So can you please guide.

I have also tried setting this to TRUE in test_xml_conv_utils.R