Open hol430 opened 5 years ago
Comment from @yashvirchauhan: I have been using Soiltemp in APSIM classic for quite a while. Tthe major difference in soil temperature simulated by Soiltemp and that simulated by soilN module is when soil is covered by the crop canopy. Soil N based simulated temperature is much higher (>2-5oC) indicating it is not able correctly simulate soil temp when radiation reaching soil is reduced - if I correctly remember. Soil temperature is used in predicting aflatoxin risk in peanuts and the values predicted by soil temp were more realistic.
@sarchontoulis to provide Validation data
@hol353 I have some data that I would be keen to have used in the soil validation and I think will be very usefull.
These two sets should provide a wide range of soil temperature profiles and fluxes
I am happy to pull datasets and simulations together if you want to include them
Yep that would be great. The more validation data we have the better.
Sounds good to me. The WWEPP validation is looking pretty good at the moment.
@sno036 - Are you actively working on this? I may have a bit of time in the next few weeks to have a go with it, if it's a priority.
No one at AgResearch is working on SoilTemp at the moment. Yes, it is a priority. It seems to be working - just needs some tests and validation data.
OK. I was thinking along the lines of also refactoring the code so it fits a bit better with ApsimX conventions. It currently reflects a bit of its Fortran->VB->C# history in the variable naming and handling of arrays. Or should that be retained a while longer for comparison purposes?
I think the only reason we haven't refactored it already is that we wanted to wait for tests, to be certain that we didn't accidentally change its behaviour while making the cosmetic changes.
I think we refactor SoilTemp2. We are going to validate it anyway. Go for it!
@hol353 - A quick questions about the model: One of the "outputs" is named Thr_profile, but I'm a bit unclear what it is, and would like to provide a better name. It appears to be the soil temperature profile at 5:00 A.M., but I can find no documentation or comments to verify that. Another "output" property provides daily minimum temperatures through the profile, so is this thing needed (or used)?
I've got no idea. @sno036 ?
@hol353 @sno036 - A couple of the grower groups in the Monaro and NSW Southern Highlands have established a network of soil moisture probes (see https://www.soilmoistureprobes.com.au/ for a list of locations. I have access to the full set of data, which the groups regard as public, as funding for the probes came from federal grants. The data include soil temperature records for many of the sites. Should I look into using these data for validation of soil water and temperature models? Note that these are pasture sites, not cropping sites. Probe depths are normally 10, 20, 40, 60, 80, 100 cm.
The SoilTemperature model in the Prototypes folder currently doesn't work as expected because the Report model is configured with reporting variables like:
[Soil].SoilTemperature.AverageSoilTemp[1] as SoilTemperature@1 [Soil].SoilTemperature.AverageSoilTemp[2] as SoilTemperature@25 [Soil].SoilTemperature.AverageSoilTemp[3] as SoilTemperature@75
The "@" character, however, is not parsed as part of the output alias. Would there be any objections if I allowed the"@" character to appear in alias names? Is this particular case, it seems to be a useful notation.
No objection - go for it.
Thanks for this and for the refactoring. I don't think I'll be back to the doco and validation for a while. The first (and perhaps primary) validation I was intending was a comparison against the analytical solution.
@zur003 - if you pull from my SoilTemperature branch you will see "soil temp.docx" which contains the original documentation.
See also the analytical solution from @rcichota in this zip file. SoilTemperature-AnalyticalSolution.zip
I started working on this using an old publication of his and ran into difficulties. Rogerio's reply was as below but I've not been able to get back to this since then (end of March):
Hi Val, I had a good look at the formulas, they all seem right, but as we suspected the devil was in the details. In a nutshell rounding errors. The actual explanation is way more complex (or there is a collection of small ones). One thing that I believe makes the problem much bigger is the format of the equations, which use frequency and phase constant instead of periods and lags. Once I switch the way to look at the functions I realise that I several small errors compounded to make those weird graphs. Because of the small numbers it doesn’t seem to make much of a difference if the frequency is 0.0000727 or 0.000072722… but it makes all the difference! While at it, I re-wrote the equations to use period and lag, they are in the word doc attached and both functions with the parameters better explained (I hope) in the excel spreadsheet. Hope this helps, and happy to run through this and show the effect of rounding in the calculations. Cheers
@sno036 - I'm not sure where to find your "SoilTemperature" Branch. Is that that SoilTemperatureValidation branch of this repository? If so, I don't see a "soil temp.docx" file, although I do see an encrypted file "ObservedFACTS.7z" which might contain it.
@zur003 - the branch is at https://github.com/sno036/ApsimX/tree/SoilTemperature/Prototypes/SoilTemperature
Here is the document - if GitHob lets be add it! soil temp.docx
Here is the zip file (which I had not added to the branch I see) [SoilTemperature-AnalyticalSolution.zip] (https://github.com/APSIMInitiative/ApsimX/files/4679904/SoilTemperature-AnalyticalSolution.zip)
Great that this is going ahead, happy to help if needed with the analytical functions, etc. Only problem is time, as the end of financial year looms... Here at PFR we do have datasets for soil temperature under cropping, we can certainly put them together when we come to that stage.
@rcichota, @sno036 - I'm just trying to clarify what has been done and what needs to be done. @sno036 indicated above that she would like to use the analytical solution as validation for the APSIM model, and @rcichota has implemented the analytical solution in Excel. Is that correct? Is there more to be done with the analytical approach, or is it just a matter of taking values from the spreadsheet (perhaps extending it to use more layers/nodes) to use as validation?
Thanks @zur003! My intention for the analytical solution was to test the numerics and also if the soil layering that most people use (many centimetres even at the surface) with SoilWater are causing any issues with the quality of the solution. This would be under highly idealised conditions but is a necessary step - the advantage is that it is only testing the soil temperature solution (not the water, crop, ...). I am happy to take this up later on (after the end of this financial year) but if you have the capacity then:
set up the soil water model such that the profile is uniform (all the same water content, bulk density, texture, carbon) with no crop and there are no changes in water content with time. I can't exactly remember how I did this last time but I would try setting it up with the soil at LL15 (are going to need a few different levels of this for testing) and if that didn't work then at SAT with Ksat and swcon = 0
need a couple of artificial weather files at latitude = 0 degrees, no rainfall, no radiation, and with air temperature of varying amplitude and average as from the spreadsheet
I would start with soil layers of about 1 cm each (where are pretty confident that the solution will work) but be set up to expand these out to more normal layering (can probably use the layering component for these).
With the above then are about (I think - I may have forgotten something) to use the spreadsheet outputs are validation/testing data for the idealised setting for various depths in the soil. I would also like to understand how well the model copes with initial soil temperatures.
After that I would test things that should affect damping depth (bulk density, carbon, water content, texture ...).
In your email you mentioned the "NGF" data? I know you have limited time. It would be great to get these into the test set. As you already (I assume) have a good understanding of that data it would be great to get them into a simulation as we might lose that information while you go on long service leave.
Make sense at all?
Hi Val,
That all makes sense, but I do have a question for you. What is the relationship of SoilTemp and SoilTemp2 in ASPIMClassic? At first glance, SoilTemp2 appears to be a VB port of SoilTemp’s Fortran, but the two models give rather different results. Is this by design?
Cheers,
Eric Zurcher
Hi Val, That all makes sense, but I do have a question for you. What is the relationship of SoilTemp and SoilTemp2 in ASPIMClassic? At first glance, SoilTemp2 appears to be a VB port of SoilTemp’s Fortran, but the two models give rather different results. Is this by design? Cheers, Eric Zurcher
HI Eric, The impetus for porting from the Fortran was that we wanted to add more detail to the boundary conditions (a atmospheric stability correction). John Hargreaves came to NZ for a stint and did the port. He first did a straight translation and the VB and Fortran versions have the same results. Then the stability corrections were added and they did change the results.. Hope this helps.
Thanks, Val. That too makes sense. I just wanted to be sure I understood how we got to the current point.
Eric Zurcher Software Developer | Grazplan Project | CSIRO Eric.Zurcher@csiro.aumailto:Eric.Zurcher@csiro.au | 02 6246 5143 | 0415 631 421
From: Val Snow notifications@github.com Sent: Friday, 5 June 2020 1:29 PM To: APSIMInitiative/ApsimX ApsimX@noreply.github.com Cc: Zurcher, Eric (A&F, Black Mountain) Eric.Zurcher@csiro.au; Mention mention@noreply.github.com Subject: Re: [APSIMInitiative/ApsimX] Port the old apsim SoilTemp2 model (#4080)
Hi Val, That all makes sense, but I do have a question for you. What is the relationship of SoilTemp and SoilTemp2 in ASPIMClassic? At first glance, SoilTemp2 appears to be a VB port of SoilTemp’s Fortran, but the two models give rather different results. Is this by design? Cheers, Eric Zurcher
HI Eric, The impetus for porting from the Fortran was that we wanted to add more detail to the boundary conditions (a atmospheric stability correction). John Hargreaves came to NZ for a stint and did the port. He first did a straight translation and the VB and Fortran versions have the same results. Then the stability corrections were added and they did change the results.. Hope this helps.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/APSIMInitiative/ApsimX/issues/4080#issuecomment-639236747, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AC2UHLFOHAU5LZIP5PR4MDDRVBRBRANCNFSM4IK57UPQ.
@sarahcleary - to add as the top comment. SoilTemp2 is a numerical solution of soil temperature changes. It is a port of the original model in Classic and is based on the solution in Campbell's "Soil Physics with Basic". There have been various reports that this model performed better than the older CERES soil temperature model. The proposal is that once SoilTemp2 is ported, the CERES model would be removed.
I have created an 'Experiment' in SoilTemperature.apsimx that is the beginning of a test of the model against the analytical solution in the spreadsheet (which I've committed) as specified by @sno036 above.
The simulation reads dummy weather data from AnalyticalWeather.xlsx.
The soil physical node has a uniform profile specified.
The water balance component has been replaced by a manager script (DummyWaterBalance) that ensures water content (a user specified parameter) does not change during the simulation.
A 'LayerStructure' component can be added to the soil to get 1cm layers. There is a 'LayerStructure' component in the Wagga simulation that can be copied/pasted.
@peter-devoil - this models needs SWIM first. Getting closer on that - Sep/Oct probably and then can think about this.
Note #7417 about initialisation needs
@sno036 @hol353 @HamishBrownPFR When reviewing the soil temperature model, I suggested that we could extract soil texture information from the currently available digital soil mapping products, rather than using a default texture for all simulations. Attached is an R script I have developed to extract soil sand, silt, and clay content within Australia from the latest Soil and Landscape Grid of Australia DSM product, and from ISRIC SoilGrids for other parts of the world. The script reports these values in the native mapped depth intervals of the GlobalSoilMap project (0-5, 5-15, 15-30, 30-60, 60-100, and 100-200 cm).
Although it may be too labour intensive to incorporate this into APSIM Next Gen at the moment, I believe this code could be useful for obtaining a more realistic initialisation of soil texture, and consequently, the soil temperature model. I am sharing it here for anyone who might find it helpful, though not sure how widely this can reach.
I'm also wondering if you will see value e.g., to link this in the documentation for the time being so that if users would like to use the code to initialise their soil texture they can easily find it.
https://github.com/Chloe0317/dsm/blob/main/SoilTextureExtraction.R
Thanks for this @Chloe0317. Yes, there is merit in your suggestion. There are quite a few data sources that could be used to in-fill a lot of soil parameters, not just particle size information. Can you create a separate GitHub Issue for this because your suggestion probably shouldn't hold up the release of the Soil Temperature model and we don't want to lose your idea?
Thanks @hol353, I'll open a new issue as I agree that this should not affect the release of the soil temperature model and rather, serve as a complementary tool and can be applied more widely for other parameters.
Interesting and useful, but there may be a need to ground truth this information. Knowing soil texture is not much difficult and should be done if reliability of soil temperature being predicted is important. Else, this may be another source of error. Estimating soil texture by hand | Agriculture and Foodhttps://www.agric.wa.gov.au/soil-constraints/soil-texture-estimating-hand#:~:text=Add%20enough%20water%20to%20make,the%20soil%20is%20very%20sandy.
From: Chloe0317 @.> Sent: Monday, 5 August 2024 11:04 AM To: APSIMInitiative/ApsimX @.> Cc: Yash Chauhan @.>; Mention @.> Subject: Re: [APSIMInitiative/ApsimX] Port the old apsim SoilTemp2 model (#4080)
Thanks @hol353, I'll open a new issue as I agree that this should not affect the release of the soil temperature model and rather, serve as a complementary tool and can be applied more widely for other parameters. — Reply to this email
Thanks @hol353https://urldefense.com/v3/__https://github.com/hol353__;!!PUY2jUP3Fp7oEg!AyjhrJ4HEyV_csmCQmUJIzxnbYGtPO91Z9y3Hnk5titfm2BmvG2esN22Ibe1uT_Zivk8bUH69B47PEFC-xxpRPx0SU0JUQgJirc$, I'll open a new issue as I agree that this should not affect the release of the soil temperature model and rather, serve as a complementary tool and can be applied more widely for other parameters.
— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/APSIMInitiative/ApsimX/issues/4080*issuecomment-2267990448__;Iw!!PUY2jUP3Fp7oEg!AyjhrJ4HEyV_csmCQmUJIzxnbYGtPO91Z9y3Hnk5titfm2BmvG2esN22Ibe1uT_Zivk8bUH69B47PEFC-xxpRPx0SU0JdEQV2zo$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AKG7IT7XGLSND3DZN53WHT3ZP3FQ5AVCNFSM6AAAAABL3Y2J3CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRXHE4TANBUHA__;!!PUY2jUP3Fp7oEg!AyjhrJ4HEyV_csmCQmUJIzxnbYGtPO91Z9y3Hnk5titfm2BmvG2esN22Ibe1uT_Zivk8bUH69B47PEFC-xxpRPx0SU0JwGpiSVA$. You are receiving this because you were mentioned.Message ID: @.***>
Hi Yash,
I have started a new issue as per Dean's suggestion as there are potentially more applications for it. I agree using mapped soil parameters can be another source of error, though they will probably still provide more value when certain attributes are un-measured and present a quantitative way of model initialisation. The mapped products have respective uncertainty assessments such as prediction confidence intervals that can be accessed and for SoilGrids the actual soil profile datasets used to produce the maps can also be accessed from the ISRIC portal. One can thus account for such uncertainty and their propagation in the modelling process through statistical means. Other mapped soil parameters available include e.g., DUL and CLL, newly mapped in 2021. I think their prediction accuracies were quite reasonable and will be useful for regional/national scale modelling.
SoilTemp2 is a numerical solution of soil temperature changes. It is a port of the original model in Classic and is based on the solution in Campbell's "Soil Physics with Basic". There have been various reports that this model performed better than the older CERES soil temperature model. The proposal is that once SoilTemp2 is ported, the CERES model would be removed.