search

APSIMInitiative / ReferencePanel

2 stars 2 forks source link

RP Meeting - 1/6/21 #88

Open sarahcleary opened 3 years ago

sarahcleary commented 3 years ago

Review of Chickpea Model - https://github.com/APSIMInitiative/ApsimX/issues/158

@AllanPeake to present.

@APSIMInitiative/reference-panel - Please let me know if you have suggestions for additional attendees.

sarahcleary commented 3 years ago

@APSIMInitiative/steering-committee - FYI. If you would like to attend, please let me know and I'll forward the meeting invitation.

yashvirchauhan commented 3 years ago

Will the presentation be available in advance?

sarahcleary commented 3 years ago

I'm currently putting together information for developers information for RP/reviewers which will include timing expectations.
I will be asking for the presenter to forward their presentation for review a week in advance of the meeting as well as putting the model "under review" at the same time.

sarahcleary commented 3 years ago

As per request from @peter-devoil - invited @jwhish and @uqdrodri

uqghamme commented 3 years ago

Hi Sarah

If possible I wouldn’t mind sitting in on chickpea model review

Regards

Graeme Hammer

Professor in Crop Science Centre for Crop Science, Queensland Alliance for Agriculture and Food Innovation The University of Queensland, Brisbane, Qld 4072, Australia Tel: +61 7 3346 9463 Mob: 0412 189 757 Email: @.**@.> Web: http://www.qaafi.uq.edu.au/ https://orcid.org/0000-0002-1180-7374

From: sarahcleary @.> Sent: Tuesday, 11 May 2021 9:23 AM To: APSIMInitiative/ReferencePanel @.> Cc: Graeme Hammer @.>; Team mention @.> Subject: Re: [APSIMInitiative/ReferencePanel] RP Meeting - 1/6/21 (#88)

I'm currently putting together information for developers information for RP/reviewers which will include timing expectations. I will be asking for the presenter to forward their presentation for review a week in advance of the meeting as well as putting the model "under review" at the same time.

— You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHubhttps://github.com/APSIMInitiative/ReferencePanel/issues/88#issuecomment-837486609, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANMBBGEKBGZY6FB77XZTV4LTNBTEPANCNFSM44S3EMRA.

sarahcleary commented 3 years ago

Information for Developer

Information for Reviewers

sno036 commented 3 years ago

@AllanPeake, @hut104 some early comments.

The validation graphs are much better than most! Exception is HI (harvest index I assume?) but I gather that is not unusual with the crop models?

Section 4.1 has some sort of issue (documentation unless there is something wrong with the supplied coordinates) with the map

Section 5.1 is really useful but could do with some interpretation/narrative of the results against the experimental trends.

What should be section 5.2 (detailed dynamics) is not showing in the documentation

The test needs a grazing simulation to make sure that the stock eat the things they should and don't eat the things they should not.

It looks like the validation dataset is from a limited geographic range. Should add some geographic transects ranging across climates where you would and would not expect the crop to thrive.

Also need to see a good example simulation before it is ready for release. This should not be the chickpea version of "Continuous Wheat" but should include a simple simulation and then a series of more complex set ups, probably including some experiments. Add some explanatory memos so that it begins to act as a tutorial. See AgPasture for one that is not fantastic but heading in the right direction - this is something that we are trying to get all models in release to do better. These examples might include skip row (if applicable to chickpea) and grazing (again if applicable - but note that still need the grazing in the validation simulation to continually test that the interaction with Stock is as it should be).

AllanPeake commented 3 years ago

Hi All, here is the presentation I will be giving next week. We will discuss Val's comments (and any others) after the presentation. Chickpea Peake Reference Panel Presentation May 2021.pptx

sarahcleary commented 3 years ago

@APSIMInitiative/reference-panel - any further comments for @AllanPeake prior to the meeting/presentation tomorrow?

sarahcleary commented 3 years ago

@jwhish; @uqdrodri - please note @AllanPeake's presentation provided

uqdrodri commented 3 years ago

Sarah I’ve got to do some training at this time though will try to follow up the seminar with my left eye,…

d

From: sarahcleary @.> Date: Monday, 31 May 2021 at 9:26 am To: APSIMInitiative/ReferencePanel @.> Cc: Daniel Rodriguez @.>, Mention @.> Subject: Re: [APSIMInitiative/ReferencePanel] RP Meeting - 1/6/21 (#88)

@jwhishhttps://github.com/jwhish; @uqdrodrihttps://github.com/uqdrodri - please note @AllanPeakehttps://github.com/AllanPeake's presentation provided

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/APSIMInitiative/ReferencePanel/issues/88#issuecomment-851077894, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMHY7N2MUOJDR2EAHKJ7HTTQLCS3ANCNFSM44S3EMRA.

sarahcleary commented 3 years ago

Apologies from @MarkLieffering and @Keith-Pembleton

yashvirchauhan commented 3 years ago

Questions for Alan Peake: I would appreciate if Alan could enlighten me and others on the following points. In chickpea flowering and pod set is strongly modulated by soil water and we now have evidence for this. Is the APSIM Chickpea (Next Gen) model able to handle soil water effect on flowering and pod set well? In one of the GRDC updates you mentioned that the model is able to predict phenology well in drier years but not in wetter years. Is it correct? What do you think of the paper by Chauhan et al. (2019) in Scientific Reports being relevant to improving phenology prediction by the model? Is the model able to account for frost impacts on yield (both positive and negatives)? Is the model able to account for temperature threshold of 15oC for pod set? Have you been able to collect data or get reference from literature that demonstrates that 15oC threshold for pod set is real as it appears strange for a crop with a base temperature of oC to have such a high threshold. At Kingaroy chickpea can set pod at much lower temperatures than 15oC

jbrider commented 3 years ago

@AllanPeake, @hut104 some early comments after going through the simulations file not the documentation.

[Phenology].StartBudSoilTT, StartBudding_PhotoPeriod, PoddingDAS and EndJuv_PhotoPeriod all have the same memo. If intentional then would help to say where they are all referenced from.

Nodule - many of the constants have empty memo's - there are a few instances for constants in other organs as well.

Not chickpea specific [Root].RootShape doesn't have any description on the right when clicked - RootshapeCylindre (correct spelling?)

StorageNDemandFunction that is implemented in generic organ (NDemands.Storage) is implemented the same way every time - referencing an NitrogenDemandSwitch that is always at the base level of the organ and he 2nd parameter is always maximumNConc. Can the NitrogenDemandSwitch be moved to be inside the StorageNDemandFunction as it does not appear to be referenced from anywhere else. The documentation refers to it as The demand for N is reduced by a factor specified by the NitrogenDemandSwitch but it is always used as a switch - either 1 or 0. NitrogenDemandSwitch also seems to have an empty memo by default.

AvailableDMRetranslocation in GenericOrgan calls RetranslocateNitrogen.CalculateBiomass - which in most instances is implemented by RetranslocateNonStructural.cs. RetranslocateNitrogen seems to be doing more than just Nitrogen? RetranslocateNonStructural also has nothing displayed in the presenter when selected.

GenericOrgan.DMSupply.Fixation has a default reference to PhotoSynthesis - and is usually implemented using a constant function that returns 0. Should GenericOrgan fixing dry matter?

Missing Charts in the chickpea.apsimx file: Validation.2019PulseAdaptation.Emerald19.BiomassByCvv - Genesis90 chart is not in the factors list Validation.2019_PulseAdaptation.GT_HatTrick_WU2019.YieldByCvv.HatTrickIrr - incorrect filter Validation.2019_PulseAdaptation.GT_HatTrick_WU2019.ProfileWater.ProfileWater_Genesis90 - cultivar not in this experiment Validation.SE Queensland.FloweringDate - no observed data maybe? Validation.SE Queensland.Lawes1991 - large number of charts with repetitions/missing/bad configuration Validation.SE Queensland.Lawes1995.Stress1 - shouldn't be there? Validation.SE Queensland.Bremar2003.SW_Layers.Function - wrong name? Validation.SE Queensland.Bremar2003.SW_Layers1 - different to SW_Layers. Validation.ICRISAT.ICRISAT_18_19BM8.Transpiration Validation.ICRISAT.ICRISAT_18_19BM8.PhotoperBranch

Charts Referencing incorrect variables Every earlyVEGBranch and lateVEGBrach chart looks to have bad references. Also check EarlyVeg and LateVeg charts for similar issue. Validation.2019_PulseAdaptation.Greenethorpe2019.NStress.Predicted: Selected Y field name 'Lentil.Leaf.Fn' Validation.2019_PulseAdaptation.GT_HatTrick_WU2019.DMSupplyDemand.Demand: Selected Y field name 'Lentil.Arbitrator.DM.TotalPlantDemand' Validation.2019PulseAdaptation.Narrabri19.DMSupplyDemand.Demand: Selected Y field name 'Lentil.Arbitrator.DM.TotalPlantDemand' Validation.SE Queensland.Bremar2003.rowspace.Predicted: Selected Y field name 'Chickpea.SowingData.RowSpacing' Validation.ICRISAT.ICRISAT_18_19BM8.rowspace.Predicted: Selected Y field name 'Chickpea.SowingData.RowSpacing'

Charts that don't look to be configured correctly Validation.2019_PulseAdaptation.Greenethorpe2019.DMSupplyDemand Validation.2019_PulseAdaptation.Greenethorpe2019.DMDemands Validation.2019_PulseAdaptation.Greenethorpe2019.Partitioning Validation.2019_PulseAdaptation.Greenethorpe2019.LeafPartitioning Validation.2019_PulseAdaptation.Greenethorpe2019.PartitioningSenesced Validation.2019_PulseAdaptation.Greenethorpe2019.StemPartitioning Validation.SE Queensland.Bremar2003.OrganDemand Validation.ICRISAT.Leaf Mass Validation.ICRISAT.Stem Mass Validation.ICRISAT.Dead Leaf Mass Validation.ICRISAT.ICRISAT_18_19BM8. - many charts near the bottom

Repeated Charts? Validation.2019PulseAdaptation.Narrabri19.BiomassByCvv1 Validation.ICRISAT.ICRISAT_18_19BM8.SW_Layers1 Validation.ICRISAT.ICRISAT_18_19BM8.SW_Layers2

jbrider commented 3 years ago

I have created an issue #6511 for a problem in the Phenology documentation where the OnEvent functions after the phases in Phenology are not being documented properly.

Leaf.Photosynthesis.FN references Leaf.Fn - as does FW reference Fw - not chickpea specific but case sensitive names shouldn't be there should they? There are also graphs nodes underneath these that I don't think should be there?

sno036 commented 3 years ago

Thanks for the presentation @AllanPeake! As with all models, they are works in progress and it is great to have a legume model in release. Following the presentation and discussion, I think it would be sensible to add some cautions for users added to the early part of the documentation are:

yashvirchauhan commented 3 years ago

Effect of soil water on chickpea phenology Rainfedandirigated_phenology_yash

uqdrodri commented 3 years ago

Sorry I missed the discussion while in a different meeting.

It if was not mentioned by others: I reckon that a collaboration with Yash will be important to move this forward. So maybe building a cross agency team should be the next step.

d

From: Yash Chauhan @.> Date: Tuesday, 1 June 2021 at 10:54 am To: APSIMInitiative/ReferencePanel @.> Cc: Daniel Rodriguez @.>, Mention @.> Subject: Re: [APSIMInitiative/ReferencePanel] RP Meeting - 1/6/21 (#88)

Effect of soil water on chickpea phenology [Image removed by sender. Rainfedandirigated_phenology_yash]https://user-images.githubusercontent.com/42857551/120252355-a35d1900-c2c7-11eb-93f1-8fe49654b2b7.jpg

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/APSIMInitiative/ReferencePanel/issues/88#issuecomment-851734368, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMHY7IVY4T2AH5ROLY5YKTTQQVS7ANCNFSM44S3EMRA.

sarahcleary commented 3 years ago

2021-06 Minutes

Attendees:

@LouisAK; @hol353; ; @peter-devoil; @yashvirchauhan; @sarahcleary; @jbrider; @HamishBrownPFR; @hut104; @sarchontoulis; @sno036; @EnliWang; @kchenu; @JulianneLilley; @Fernanda-Dreccer; @uqghamme; @Keith-Pembleton; @jwhish; @yashvirchauhan; @LouisAK; @allanpeake @sme016

Apologies/Not in attendance:

@MarkLieffering

Tuesday 1st June 2021 - 9:30 am AEST

Meeting recorded – which can be accessed on Teams link for meeting attendees. Review of Chickpea Model - https://github.com/APSIMInitiative/ApsimX/issues/158 @AllanPeake ran through the powerpoint – Chickpea Peake Reference Panel Presentation May 2021 Chickpea.Peake.Reference.Panel.Presentation.May.2021.pptx @allanpeake - worked through issues raised prior to the meeting (on this issue)

Discussion can be found on recording. Please email apsim@csiro.au if you wish to have access.

Work required for Model to be accepted into Release

Fix errors identified by @jbrider's - https://github.com/APSIMInitiative/ReferencePanel/issues/88#issuecomment-851117688 Add cautions for user as per @sno036's - https://github.com/APSIMInitiative/ReferencePanel/issues/88#issuecomment-851732497 Possible integration of Stock and Pest&Disease tests - to be discussed at next AI RP Meeting Include plausibility tests - range of environments and management (geographical range/temperature)

Discussion for next AI RP Meeting

Requirement for crop models to include integration of stock and/or pest&disease examples Include a requirement for broad scale simulation - to check for 'silly' stuff - biomass and yield.
Potential solution - should there be a standardised test? Making it easier for crop modellers How to show the data behind the functions in the Autodocumentation, e.g A's photoperiod response slide.

yashvirchauhan commented 3 years ago

Looks all the points I made have been overlooked in an attempt to push through the model. Am I right?

jbrider commented 3 years ago

@yashvirchauhan, I asked several times if people were happy for it go into release once the additional simulations were added and fixes to charts as noted by Sarah above. I'm sorry if I wasn't clear enough during that process - but if you don't think the model should go into release then you need to express clearly which part of your concerns were not addressed adequately during the meeting.

@AllanPeake addressed the soilwater issue verbally - were you not happy with his response?

I must admit I can't remember the response about frost?

His presentation and the Nextgen documentation both address the 15oC pod set - you didn't address it during the question time - did you have further concerns about it?

yashvirchauhan commented 3 years ago

I had mentioned that soil water modulate chickpea phenology (photograph I included provide that evidence) and have even developed an equation that captures that effect. The new model apparently does not address this issue at all. Not sure if that equation was tried to predict flowering time of chickpea. Also as highlighted by Peter Carberry in conversation with Jeremy whether, the old phenology data collected by him should be used to validate the phenology model of the new model. At least these areas should have been highlighted as areas for future work. The frost effect is not adequately covered while it accounts for substantial variation in chickpea yield as has also been published by Jeremy Whish several years ago. However, frosts before flowering can be beneficial too as they may reduce canopy to reduce water use. I also mention that 15oC is not a verifiable threshold for chickpea (no one has produced data it is just said as a guess work) then why modify the model to capture that effect. I am more passionate that science should be correct as much as colleagues like you will look into software part of it. In a nutshell more basic scientific work needs to be done which should be mentioned in the minutes for record purposes and also prioritise future model improvements. It is as important as integrating the model with the stock model even though stock may not like to eat chickpea (rabbits and Kangaroos don't eat) because of acid on its leaves.

yashvirchauhan commented 3 years ago

Just as I was highlighting the need to work on frost and temperature in the near future in the comment earlier, GRDC came up with the tender call: EOI: Prediction tools to quantify yield loss from frost and heat events in wheat, barley, canola, chickpeas, and lentils This EOI presents our chance to fix issues related to subzero and high temperature temperature in both chickpea and lentil crops and improve models around this issue. This is also our chance to make models practical tools for growers and their advisors to use. Any interest in collaboration to put up a joint proposal? I will be interested in contributing to this effort on chickpea and lentil. Phenology prediction will be an important part of the whole tender as phenological stages determine the sensitivity to frosts and heat stress. Cheers, Yash

uqdrodri commented 3 years ago

That’s the spirit Yash!!, happy to have a chat

D

From: Yash Chauhan @.> Date: Tuesday, 1 June 2021 at 3:29 pm To: APSIMInitiative/ReferencePanel @.> Cc: Daniel Rodriguez @.>, Mention @.> Subject: Re: [APSIMInitiative/ReferencePanel] RP Meeting - 1/6/21 (#88)

Just as I was highlighting the need to work on frost and temperature in the near future in the comment earlier, GRDC came up with the tender call: EOI: Prediction tools to quantify yield loss from frost and heat events in wheat, barley, canola, chickpeas, and lentils This EOI presents our chance to fix issues related to subzero and high temperature temperature in both chickpea and lentil crops and improve models around this issue. This is also our chance to make models practical tools for growers and their advisors to use. Any interest in collaboration to put up a joint proposal? I will be interested in contributing to this effort on chickpea and lentil. Phenology prediction will be an important part of the whole tender as phenological stages determine the sensitivity to frosts and heat stress. Cheers, Yash

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/APSIMInitiative/ReferencePanel/issues/88#issuecomment-851821324, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMHY7JE67QGVILVOCYNUPLTQRVZ7ANCNFSM44S3EMRA.

jbrider commented 3 years ago

@yashvirchauhan Sarah has uploaded the video from the review and it is available from the chat history for AI RP Discussion - Review of ChickPea Model https://csiroau-my.sharepoint.com/:v:/g/personal/cle11c_csiro_au/EaysLAmtBqlMjRpNCQG29kAB3mlXTG7uArG5JThmejgNYA?e=zRSqrr

Allan went through the podding/phenology solution using the 15o avg from 00:11:50 He commented on the soilwater issue at 1:03:00 in response to your initial github comments - which included an acknowledgment that there was no frost response implemented.

jbrider commented 3 years ago

@AllanPeake @hut104 After watching the review again, I have found a couple of issues that we hadn't initially captured.

Jeremy raised a question about original data from previously published papers - primarily a paper by Peter Carberry. Yash said he had that data and Neil said that there may be existing simulations using that data. It would be good to have that in there.

Jeremy also raised an issue about stage/phase naming to keep inline with existing naming - in particular Late Vegetative being renamed to Floral Initiation.

Keith-Pembleton commented 3 years ago

Just caught up with the recording after I had to drop off the meeting. @AllanPeake well done with this model. I did have a question around transpirational cooling that you answered towards the end so now I only have one question/comment. You made a comment around the important in highlighting the uncertainties in the model to guide future development (and the funding for future development effort). Could the sensibility testing of response relationships that @uqghamme proposed be a set up as a good/efficient way of highlighting that uncertainty to users of the model (particularly those who might pick it up to use without reading the documentation fully)? @LouisAK already gave USQ's yes vote to into release pending the required revisions but for what its worth I also agree.

AllanPeake commented 3 years ago

Thanks Keith. Sensibility testing is one place uncertainty could be highlighted, but it’s role is probably more to ensure the model gives realistic outputs across a range of environmental or management extremes.

I would say that any time we present on model outputs (whether in papers, conference talks or industry presentations), we should be open about model uncertainties so that people are aware of them.

AllanPeake commented 3 years ago

The chickpea model has now been re-submitted for release. The following comments relate to changes requested to the model:

uqghamme commented 3 years ago

It is great to see all this However, there is an absence of explanatory text in the plausibility/sensibility section where there are just some graphs and very little context explanation A few more words about what is expected (qualitative responses) and what model actually does would be a good idea I think this should become some sort of exemplar for a section like this in the documentation

yashvirchauhan commented 3 years ago

ChickpeamodelvalidationYash

Hi all Simulation of flowering using APSIM NextGen chickpea model and the old 7.10 model with soil water effect being accounted for. For your information.