search

APSIMInitiative / ApsimX

ApsimX is the next generation of APSIM
http://www.apsim.info
Other
132 stars 161 forks source link

Port the old apsim SoilTemp2 model #4080

Open hol430 opened 5 years ago

hol430 commented 5 years ago

SoilTemp2 is a numerical solution of soil temperature changes. It is a port of the original model in Classic and is based on the solution in Campbell's "Soil Physics with Basic". There have been various reports that this model performed better than the older CERES soil temperature model. The proposal is that once SoilTemp2 is ported, the CERES model would be removed.

sarahcleary commented 5 years ago

Comment from @yashvirchauhan: I have been using Soiltemp in APSIM classic for quite a while. Tthe major difference in soil temperature simulated by Soiltemp and that simulated by soilN module is when soil is covered by the crop canopy. Soil N based simulated temperature is much higher (>2-5oC) indicating it is not able correctly simulate soil temp when radiation reaching soil is reduced - if I correctly remember. Soil temperature is used in predicting aflatoxin risk in peanuts and the values predicted by soil temp were more realistic.

sarahcleary commented 5 years ago

@sarchontoulis to provide Validation data

HamishBrownPFR commented 4 years ago

@hol353 I have some data that I would be keen to have used in the soil validation and I think will be very usefull.

  1. Soil temperature at 2, 15, 30, 90, 120, 150 and 180 cm, heat flux at 15cm, and crop radiation interception over entire crop duration's under barley, wheat, oats, fodder beet and potato with 6 treatments (and 4 reps for each crop) giving differences in canopy cover patterns
  2. Soil temperature at 7.5, 22.5, 37.5 and 52.5 cm in lysimeters with 0, 30 and 50% stones in the top soil layer in factorial combination with gravels at 30 or 60 cm.

These two sets should provide a wide range of soil temperature profiles and fluxes

  1. Soil temperature at 1.5 cm depth under wheat and barley crops for a range of sowing dates. It is going to be important to have the surface temperature well validated for soil temp so we can look to use it for driving early crop development.

I am happy to pull datasets and simulations together if you want to include them

hol353 commented 4 years ago

Yep that would be great. The more validation data we have the better.

sno036 commented 4 years ago

Sounds good to me. The WWEPP validation is looking pretty good at the moment.

zur003 commented 4 years ago

@sno036 - Are you actively working on this? I may have a bit of time in the next few weeks to have a go with it, if it's a priority.

hol353 commented 4 years ago

No one at AgResearch is working on SoilTemp at the moment. Yes, it is a priority. It seems to be working - just needs some tests and validation data.

zur003 commented 4 years ago

OK. I was thinking along the lines of also refactoring the code so it fits a bit better with ApsimX conventions. It currently reflects a bit of its Fortran->VB->C# history in the variable naming and handling of arrays. Or should that be retained a while longer for comparison purposes?

hol430 commented 4 years ago

I think the only reason we haven't refactored it already is that we wanted to wait for tests, to be certain that we didn't accidentally change its behaviour while making the cosmetic changes.

hol353 commented 4 years ago

I think we refactor SoilTemp2. We are going to validate it anyway. Go for it!

zur003 commented 4 years ago

@hol353 - A quick questions about the model: One of the "outputs" is named Thr_profile, but I'm a bit unclear what it is, and would like to provide a better name. It appears to be the soil temperature profile at 5:00 A.M., but I can find no documentation or comments to verify that. Another "output" property provides daily minimum temperatures through the profile, so is this thing needed (or used)?

hol353 commented 4 years ago

I've got no idea. @sno036 ?

zur003 commented 4 years ago

@hol353 @sno036 - A couple of the grower groups in the Monaro and NSW Southern Highlands have established a network of soil moisture probes (see https://www.soilmoistureprobes.com.au/ for a list of locations. I have access to the full set of data, which the groups regard as public, as funding for the probes came from federal grants. The data include soil temperature records for many of the sites. Should I look into using these data for validation of soil water and temperature models? Note that these are pasture sites, not cropping sites. Probe depths are normally 10, 20, 40, 60, 80, 100 cm.

zur003 commented 4 years ago

The SoilTemperature model in the Prototypes folder currently doesn't work as expected because the Report model is configured with reporting variables like:

[Soil].SoilTemperature.AverageSoilTemp[1] as SoilTemperature@1 [Soil].SoilTemperature.AverageSoilTemp[2] as SoilTemperature@25 [Soil].SoilTemperature.AverageSoilTemp[3] as SoilTemperature@75

The "@" character, however, is not parsed as part of the output alias. Would there be any objections if I allowed the"@" character to appear in alias names? Is this particular case, it seems to be a useful notation.

hol353 commented 4 years ago

No objection - go for it.

sno036 commented 4 years ago

Thanks for this and for the refactoring. I don't think I'll be back to the doco and validation for a while. The first (and perhaps primary) validation I was intending was a comparison against the analytical solution.

@zur003 - if you pull from my SoilTemperature branch you will see "soil temp.docx" which contains the original documentation.

See also the analytical solution from @rcichota in this zip file. SoilTemperature-AnalyticalSolution.zip

I started working on this using an old publication of his and ran into difficulties. Rogerio's reply was as below but I've not been able to get back to this since then (end of March):

Hi Val, I had a good look at the formulas, they all seem right, but as we suspected the devil was in the details. In a nutshell rounding errors. The actual explanation is way more complex (or there is a collection of small ones). One thing that I believe makes the problem much bigger is the format of the equations, which use frequency and phase constant instead of periods and lags. Once I switch the way to look at the functions I realise that I several small errors compounded to make those weird graphs. Because of the small numbers it doesn’t seem to make much of a difference if the frequency is 0.0000727 or 0.000072722… but it makes all the difference! While at it, I re-wrote the equations to use period and lag, they are in the word doc attached and both functions with the parameters better explained (I hope) in the excel spreadsheet. Hope this helps, and happy to run through this and show the effect of rounding in the calculations. Cheers

zur003 commented 4 years ago

@sno036 - I'm not sure where to find your "SoilTemperature" Branch. Is that that SoilTemperatureValidation branch of this repository? If so, I don't see a "soil temp.docx" file, although I do see an encrypted file "ObservedFACTS.7z" which might contain it.

sno036 commented 4 years ago

@zur003 - the branch is at https://github.com/sno036/ApsimX/tree/SoilTemperature/Prototypes/SoilTemperature

Here is the document - if GitHob lets be add it! soil temp.docx

Here is the zip file (which I had not added to the branch I see) [SoilTemperature-AnalyticalSolution.zip] (https://github.com/APSIMInitiative/ApsimX/files/4679904/SoilTemperature-AnalyticalSolution.zip)

rcichota commented 4 years ago

Great that this is going ahead, happy to help if needed with the analytical functions, etc. Only problem is time, as the end of financial year looms... Here at PFR we do have datasets for soil temperature under cropping, we can certainly put them together when we come to that stage.

zur003 commented 4 years ago

@rcichota, @sno036 - I'm just trying to clarify what has been done and what needs to be done. @sno036 indicated above that she would like to use the analytical solution as validation for the APSIM model, and @rcichota has implemented the analytical solution in Excel. Is that correct? Is there more to be done with the analytical approach, or is it just a matter of taking values from the spreadsheet (perhaps extending it to use more layers/nodes) to use as validation?

sno036 commented 4 years ago

Thanks @zur003! My intention for the analytical solution was to test the numerics and also if the soil layering that most people use (many centimetres even at the surface) with SoilWater are causing any issues with the quality of the solution. This would be under highly idealised conditions but is a necessary step - the advantage is that it is only testing the soil temperature solution (not the water, crop, ...). I am happy to take this up later on (after the end of this financial year) but if you have the capacity then:

I would start with soil layers of about 1 cm each (where are pretty confident that the solution will work) but be set up to expand these out to more normal layering (can probably use the layering component for these).

With the above then are about (I think - I may have forgotten something) to use the spreadsheet outputs are validation/testing data for the idealised setting for various depths in the soil. I would also like to understand how well the model copes with initial soil temperatures.

After that I would test things that should affect damping depth (bulk density, carbon, water content, texture ...).

In your email you mentioned the "NGF" data? I know you have limited time. It would be great to get these into the test set. As you already (I assume) have a good understanding of that data it would be great to get them into a simulation as we might lose that information while you go on long service leave.

Make sense at all?

zur003 commented 4 years ago

Hi Val,

That all makes sense, but I do have a question for you. What is the relationship of SoilTemp and SoilTemp2 in ASPIMClassic? At first glance, SoilTemp2 appears to be a VB port of SoilTemp’s Fortran, but the two models give rather different results. Is this by design?

Cheers,

Eric Zurcher

sno036 commented 4 years ago

Hi Val, That all makes sense, but I do have a question for you. What is the relationship of SoilTemp and SoilTemp2 in ASPIMClassic? At first glance, SoilTemp2 appears to be a VB port of SoilTemp’s Fortran, but the two models give rather different results. Is this by design? Cheers, Eric Zurcher

HI Eric, The impetus for porting from the Fortran was that we wanted to add more detail to the boundary conditions (a atmospheric stability correction). John Hargreaves came to NZ for a stint and did the port. He first did a straight translation and the VB and Fortran versions have the same results. Then the stability corrections were added and they did change the results.. Hope this helps.

zur003 commented 4 years ago

Thanks, Val. That too makes sense. I just wanted to be sure I understood how we got to the current point.

Eric Zurcher Software Developer | Grazplan Project | CSIRO Eric.Zurcher@csiro.aumailto:Eric.Zurcher@csiro.au | 02 6246 5143 | 0415 631 421

From: Val Snow notifications@github.com Sent: Friday, 5 June 2020 1:29 PM To: APSIMInitiative/ApsimX ApsimX@noreply.github.com Cc: Zurcher, Eric (A&F, Black Mountain) Eric.Zurcher@csiro.au; Mention mention@noreply.github.com Subject: Re: [APSIMInitiative/ApsimX] Port the old apsim SoilTemp2 model (#4080)

Hi Val, That all makes sense, but I do have a question for you. What is the relationship of SoilTemp and SoilTemp2 in ASPIMClassic? At first glance, SoilTemp2 appears to be a VB port of SoilTemp’s Fortran, but the two models give rather different results. Is this by design? Cheers, Eric Zurcher

HI Eric, The impetus for porting from the Fortran was that we wanted to add more detail to the boundary conditions (a atmospheric stability correction). John Hargreaves came to NZ for a stint and did the port. He first did a straight translation and the VB and Fortran versions have the same results. Then the stability corrections were added and they did change the results.. Hope this helps.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/APSIMInitiative/ApsimX/issues/4080#issuecomment-639236747, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AC2UHLFOHAU5LZIP5PR4MDDRVBRBRANCNFSM4IK57UPQ.

sno036 commented 4 years ago

@sarahcleary - to add as the top comment. SoilTemp2 is a numerical solution of soil temperature changes. It is a port of the original model in Classic and is based on the solution in Campbell's "Soil Physics with Basic". There have been various reports that this model performed better than the older CERES soil temperature model. The proposal is that once SoilTemp2 is ported, the CERES model would be removed.

hol353 commented 4 years ago

I have created an 'Experiment' in SoilTemperature.apsimx that is the beginning of a test of the model against the analytical solution in the spreadsheet (which I've committed) as specified by @sno036 above.

sno036 commented 2 years ago

@peter-devoil - this models needs SWIM first. Getting closer on that - Sep/Oct probably and then can think about this.

sno036 commented 1 year ago

Note #7417 about initialisation needs

Chloe0317 commented 1 month ago

@sno036 @hol353 @HamishBrownPFR When reviewing the soil temperature model, I suggested that we could extract soil texture information from the currently available digital soil mapping products, rather than using a default texture for all simulations. Attached is an R script I have developed to extract soil sand, silt, and clay content within Australia from the latest Soil and Landscape Grid of Australia DSM product, and from ISRIC SoilGrids for other parts of the world. The script reports these values in the native mapped depth intervals of the GlobalSoilMap project (0-5, 5-15, 15-30, 30-60, 60-100, and 100-200 cm).

Although it may be too labour intensive to incorporate this into APSIM Next Gen at the moment, I believe this code could be useful for obtaining a more realistic initialisation of soil texture, and consequently, the soil temperature model. I am sharing it here for anyone who might find it helpful, though not sure how widely this can reach.

I'm also wondering if you will see value e.g., to link this in the documentation for the time being so that if users would like to use the code to initialise their soil texture they can easily find it.

https://github.com/Chloe0317/dsm/blob/main/SoilTextureExtraction.R

hol353 commented 1 month ago

Thanks for this @Chloe0317. Yes, there is merit in your suggestion. There are quite a few data sources that could be used to in-fill a lot of soil parameters, not just particle size information. Can you create a separate GitHub Issue for this because your suggestion probably shouldn't hold up the release of the Soil Temperature model and we don't want to lose your idea?

Chloe0317 commented 1 month ago

Thanks @hol353, I'll open a new issue as I agree that this should not affect the release of the soil temperature model and rather, serve as a complementary tool and can be applied more widely for other parameters.

yashvirchauhan commented 1 month ago

Interesting and useful, but there may be a need to ground truth this information. Knowing soil texture is not much difficult and should be done if reliability of soil temperature being predicted is important. Else, this may be another source of error. Estimating soil texture by hand | Agriculture and Foodhttps://www.agric.wa.gov.au/soil-constraints/soil-texture-estimating-hand#:~:text=Add%20enough%20water%20to%20make,the%20soil%20is%20very%20sandy.

From: Chloe0317 @.> Sent: Monday, 5 August 2024 11:04 AM To: APSIMInitiative/ApsimX @.> Cc: Yash Chauhan @.>; Mention @.> Subject: Re: [APSIMInitiative/ApsimX] Port the old apsim SoilTemp2 model (#4080)

Thanks @hol353, I'll open a new issue as I agree that this should not affect the release of the soil temperature model and rather, serve as a complementary tool and can be applied more widely for other parameters. — Reply to this email

Thanks @hol353https://urldefense.com/v3/__https://github.com/hol353__;!!PUY2jUP3Fp7oEg!AyjhrJ4HEyV_csmCQmUJIzxnbYGtPO91Z9y3Hnk5titfm2BmvG2esN22Ibe1uT_Zivk8bUH69B47PEFC-xxpRPx0SU0JUQgJirc$, I'll open a new issue as I agree that this should not affect the release of the soil temperature model and rather, serve as a complementary tool and can be applied more widely for other parameters.

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/APSIMInitiative/ApsimX/issues/4080*issuecomment-2267990448__;Iw!!PUY2jUP3Fp7oEg!AyjhrJ4HEyV_csmCQmUJIzxnbYGtPO91Z9y3Hnk5titfm2BmvG2esN22Ibe1uT_Zivk8bUH69B47PEFC-xxpRPx0SU0JdEQV2zo$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AKG7IT7XGLSND3DZN53WHT3ZP3FQ5AVCNFSM6AAAAABL3Y2J3CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRXHE4TANBUHA__;!!PUY2jUP3Fp7oEg!AyjhrJ4HEyV_csmCQmUJIzxnbYGtPO91Z9y3Hnk5titfm2BmvG2esN22Ibe1uT_Zivk8bUH69B47PEFC-xxpRPx0SU0JwGpiSVA$. You are receiving this because you were mentioned.Message ID: @.***>

The information in this email together with any attachments is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. There is no waiver of any confidentiality/privilege by your inadvertent receipt of this material. Any form of review, disclosure, modification, distribution and/or publication of this email message is prohibited, unless as a necessary part of Departmental business. If you have received this message in error, you are asked to inform the sender as quickly as possible and delete this message and any copies of this message from your computer and/or your computer system network.

Chloe0317 commented 1 month ago

Hi Yash,

I have started a new issue as per Dean's suggestion as there are potentially more applications for it. I agree using mapped soil parameters can be another source of error, though they will probably still provide more value when certain attributes are un-measured and present a quantitative way of model initialisation. The mapped products have respective uncertainty assessments such as prediction confidence intervals that can be accessed and for SoilGrids the actual soil profile datasets used to produce the maps can also be accessed from the ISRIC portal. One can thus account for such uncertainty and their propagation in the modelling process through statistical means. Other mapped soil parameters available include e.g., DUL and CLL, newly mapped in 2021. I think their prediction accuracies were quite reasonable and will be useful for regional/national scale modelling.