comses / ming

Example model coupling between a LandLab hydrology model and the DSSAT crop model

GNU General Public License v3.0

0 stars 1 forks source link

script dssat runs on hydrology model outputs #9

Open alee opened 7 years ago

alee commented 7 years ago

Script + automate runs of dssat on the hydrology model output weather station files

We're almost at the home stretch! After this we'll want to script the stitching together of the output files to generate some pretty pictures. Thanks to @cmbarton @chporter @mcflugen @GeraldCNelson and Jordan for all your help & contributions!

We should probably use ARAM8001.MZX as the base template based on Cheryl's description below. The Python script driving the hydrology model generates a fresh .WTH file and could also generate an accompanying .MZX file referencing that new .WTH file, then run dssat A <new-file.MZX>.

From @chporter

ARAM8001.MZX – I think this is equivalent to the file you had before, but all daily output is switched off and the summary file is in csv format. Use this one if DSSAT is to calculate infiltration from rainfall input.
ARAM8091.MZX – same, except that a different soil is used (AMES000910 instead of the original soil, AMES000010). The new soil reduces the SCS runoff curve number to 2.0 (instead of 80.0), which forces all the “rainfall” to infiltrate the soil, with zero runoff. Use this one if the hydrology model passes infiltration to DSSAT.

cmbarton commented 7 years ago

Whoo hoo!!!

C. Michael Barton Director, Center for Social Dynamics & Complexity Professor of Anthropology, School of Human Evolution & Social Change Head, Graduate Faculty in Complex Adaptive Systems Science Arizona State University

voice: 480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC) fax: 480-965-7671 (SHESC), 480-727-0709 (CSDC) www: http://www.public.asu.edu/~cmbarton, http://csdc.asu.edu

On Nov 17, 2017, at 4:46 PM, A Lee notifications@github.com<mailto:notifications@github.com> wrote:

Script + automate runs of dssat on the hydrology model output weather station files

We're almost at the home stretch! After this we'll want to script the stitching together of the output files to generate some pretty pictures. Thanks to @cmbartonhttps://github.com/cmbarton @chporterhttps://github.com/chporter @mcflugenhttps://github.com/mcflugen @GeraldCNelsonhttps://github.com/geraldcnelson for pushing this forward thus far.

We should probably use ARAM8001.MZX as the base template based on Cheryl's description below. The Python script driving the hydrology model generates a fresh .WTH file and could also generate an accompanying .MZX file referencing that new .WTH file, then run dssat A .

From @chporterhttps://github.com/chporter

ARAM8001.MZX – I think this is equivalent to the file you had before, but all daily output is switched off and the summary file is in csv format. Use this one if DSSAT is to calculate infiltration from rainfall input.
ARAM8091.MZX – same, except that a different soil is used (AMES000910 instead of the original soil, AMES000010). The new soil reduces the SCS runoff curve number to 2.0 (instead of 80.0), which forces all the “rainfall” to infiltrate the soil, with zero runoff. Use this one if the hydrology model passes infiltration to DSSAT.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/comses/ming/issues/9, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AIjVr0HV3LFYJPH24ctsmc0K3HFq9SiHks5s3hrngaJpZM4QixHA.

chporter commented 7 years ago

Sounds like great progress! How best can we reality check inputs/outputs? Cheryl

From: Michael Barton [mailto:notifications@github.com] Sent: Friday, November 17, 2017 6:55 PM To: comses/ming ming@noreply.github.com Cc: Porter, Cheryl H cporter@ufl.edu; Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

Whoo hoo!!!

voice: 480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC) fax: 480-965-7671 (SHESC), 480-727-0709 (CSDC) www: http://www.public.asu.edu/~cmbarton, http://csdc.asu.edu

On Nov 17, 2017, at 4:46 PM, A Lee notifications@github.com<mailto:notifications@github.com> wrote:

Script + automate runs of dssat on the hydrology model output weather station files

From @chporterhttps://github.com/chporter

ARAM8001.MZX – I think this is equivalent to the file you had before, but all daily output is switched off and the summary file is in csv format. Use this one if DSSAT is to calculate infiltration from rainfall input.
ARAM8091.MZX – same, except that a different soil is used (AMES000910 instead of the original soil, AMES000010). The new soil reduces the SCS runoff curve number to 2.0 (instead of 80.0), which forces all the “rainfall” to infiltrate the soil, with zero runoff. Use this one if the hydrology model passes infiltration to DSSAT.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_comses_ming_issues_9-23issuecomment-2D345397039&d=DwMFaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=mD5wqN6S7Prx_hYQhHAoOw&m=FJgcAwUUBSruI-TVwqcbTOkvS1e7GFHMsbD47-aAIhA&s=FdGZRRrdLU76Vbt2IH9ooaxDpAqj40-dvzPmfXkkccc&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AATima6iZeTRGA1soLXEBma9ZtusqIO5ks5s3hy2gaJpZM4QixHA&d=DwMFaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=mD5wqN6S7Prx_hYQhHAoOw&m=FJgcAwUUBSruI-TVwqcbTOkvS1e7GFHMsbD47-aAIhA&s=CJbujFftYGb2wvck2_egj7CmUSUYT1uP3uXMaWSLWeM&e=.

mcflugen commented 7 years ago

@alee, I've generated the 65,536 DSSAT WTH files - one for each cell of our grid. Do you want to try to run DSSAT on each of these? They're pretty big - together they're about 77GB (or 12GB zipped up). The zipped tarball is on our ftp site for you to grab if you're interested.

alee commented 7 years ago

Great, thanks Eric! Sure, I can grab them from FTP, want to point me at it via email?

cmbarton commented 7 years ago

This is all wonderful news. If we end up running DSSAT on a grid, we'll need to run it twice: once with the default values, and again with the hydrology model generated values.

I'll want to have the raster grid (what is the format???) of the flow accumulation from the hydrology model, and the grids for DSSAT. Definitely yield and maybe biomass too if that is generated by default. Any other values I might want to show??

BTW, does the hydrology model have a name?

Michael

On Nov 21, 2017, at 12:14 AM, A Lee notifications@github.com wrote:

Great, thanks Eric! Sure, I can grab them from FTP, want to point me at it via email?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-345937469, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVrxyIQ4IMXFdBlDz3H7zVKu4XADv0ks5s4nhRgaJpZM4QixHA.

GeraldCNelson commented 7 years ago

I’d say a map of water availability per pixel with no flow and one with. Or a delta between the two.

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 11:51 AM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

This is all wonderful news. If we end up running DSSAT on a grid, we'll need to run it twice: once with the default values, and again with the hydrology model generated values.

BTW, does the hydrology model have a name?

Michael

On Nov 21, 2017, at 12:14 AM, A Lee notifications@github.com wrote:

Great, thanks Eric! Sure, I can grab them from FTP, want to point me at it via email?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-345937469, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVrxyIQ4IMXFdBlDz3H7zVKu4XADv0ks5s4nhRgaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

cmbarton commented 7 years ago

Good idea. Thanks

Michael

On Nov 21, 2017, at 11:53 AM, Gerald Charles Nelson notifications@github.com wrote:

I’d say a map of water availability per pixel with no flow and one with. Or a delta between the two.

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 11:51 AM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

This is all wonderful news. If we end up running DSSAT on a grid, we'll need to run it twice: once with the default values, and again with the hydrology model generated values.

I'll want to have the raster grid (what is the format???) of the flow accumulation from the hydrology model, and the grids for DSSAT. Definitely yield and maybe biomass too if that is generated by default. Any other values I might want to show??

BTW, does the hydrology model have a name?

Michael

On Nov 21, 2017, at 12:14 AM, A Lee notifications@github.com wrote:

Great, thanks Eric! Sure, I can grab them from FTP, want to point me at it via email?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-345937469, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVrxyIQ4IMXFdBlDz3H7zVKu4XADv0ks5s4nhRgaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346125165, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVr_MQP8WScnA48Jdv2J3L6R5vvrMrks5s4xw2gaJpZM4QixHA.

cmbarton commented 7 years ago

Seems like a good idea to do a flow diagram of the steps needed for model coupling in general and specifically with this pilot project.

Michael

On Nov 21, 2017, at 11:53 AM, Gerald Charles Nelson notifications@github.com wrote:

I’d say a map of water availability per pixel with no flow and one with. Or a delta between the two.

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 11:51 AM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

This is all wonderful news. If we end up running DSSAT on a grid, we'll need to run it twice: once with the default values, and again with the hydrology model generated values.

I'll want to have the raster grid (what is the format???) of the flow accumulation from the hydrology model, and the grids for DSSAT. Definitely yield and maybe biomass too if that is generated by default. Any other values I might want to show??

BTW, does the hydrology model have a name?

Michael

On Nov 21, 2017, at 12:14 AM, A Lee notifications@github.com wrote:

Great, thanks Eric! Sure, I can grab them from FTP, want to point me at it via email?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-345937469, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVrxyIQ4IMXFdBlDz3H7zVKu4XADv0ks5s4nhRgaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346125165, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVr_MQP8WScnA48Jdv2J3L6R5vvrMrks5s4xw2gaJpZM4QixHA.

GeraldCNelson commented 7 years ago

I’d also suggest pointing out that this form of coupling is just a sophisticated version of sneakernet. Run model A and output relevant results to a text file. Model B uses the text file as one of its inputs. In the current setup, model B is running inside a docker container so at least in principle it is system-agnostic. But a compiled version of Model B that is system specific could also make use of the text file.

Now that Allen and Eric (and Jordan?) have the models running separately and sharing data in one direction, do they have any thoughts about how to set up two way interactions via BMI?

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 11:59 AM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

Seems like a good idea to do a flow diagram of the steps needed for model coupling in general and specifically with this pilot project.

Michael

On Nov 21, 2017, at 11:53 AM, Gerald Charles Nelson notifications@github.com wrote:

I’d say a map of water availability per pixel with no flow and one with. Or a delta between the two.

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 11:51 AM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

This is all wonderful news. If we end up running DSSAT on a grid, we'll need to run it twice: once with the default values, and again with the hydrology model generated values.

I'll want to have the raster grid (what is the format???) of the flow accumulation from the hydrology model, and the grids for DSSAT. Definitely yield and maybe biomass too if that is generated by default. Any other values I might want to show??

BTW, does the hydrology model have a name?

Michael

On Nov 21, 2017, at 12:14 AM, A Lee notifications@github.com wrote:

Great, thanks Eric! Sure, I can grab them from FTP, want to point me at it via email?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-345937469, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVrxyIQ4IMXFdBlDz3H7zVKu4XADv0ks5s4nhRgaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346125165, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVr_MQP8WScnA48Jdv2J3L6R5vvrMrks5s4xw2gaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

cmbarton commented 7 years ago

Since Eric is autogenerating the WTH files, it is simply a matter of updating the script that runs the hydrology model in Docker and BMI, and parses output into DSSAT format to also initiate DSSAT via BMI and Docker.

Michael

On Nov 21, 2017, at 1:59 PM, Gerald Charles Nelson notifications@github.com wrote:

I’d also suggest pointing out that this form of coupling is just a sophisticated version of sneakernet. Run model A and output relevant results to a text file. Model B uses the text file as one of its inputs. In the current setup, model B is running inside a docker container so at least in principle it is system-agnostic. But a compiled version of Model B that is system specific could also make use of the text file.

Now that Allen and Eric (and Jordan?) have the models running separately and sharing data in one direction, do they have any thoughts about how to set up two way interactions via BMI?

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 11:59 AM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

Seems like a good idea to do a flow diagram of the steps needed for model coupling in general and specifically with this pilot project.

Michael

On Nov 21, 2017, at 11:53 AM, Gerald Charles Nelson notifications@github.com wrote:

I’d say a map of water availability per pixel with no flow and one with. Or a delta between the two.

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 11:51 AM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

This is all wonderful news. If we end up running DSSAT on a grid, we'll need to run it twice: once with the default values, and again with the hydrology model generated values.

I'll want to have the raster grid (what is the format???) of the flow accumulation from the hydrology model, and the grids for DSSAT. Definitely yield and maybe biomass too if that is generated by default. Any other values I might want to show??

BTW, does the hydrology model have a name?

Michael

On Nov 21, 2017, at 12:14 AM, A Lee notifications@github.com wrote:

Great, thanks Eric! Sure, I can grab them from FTP, want to point me at it via email?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-345937469, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVrxyIQ4IMXFdBlDz3H7zVKu4XADv0ks5s4nhRgaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346125165, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVr_MQP8WScnA48Jdv2J3L6R5vvrMrks5s4xw2gaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346159184, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVr3GGp71Uj15C3d_oBrsivChB7a-Gks5s4znJgaJpZM4QixHA.

GeraldCNelson commented 7 years ago

I don’t think that gets at the ‘real’ two-way interaction we want. Infiltration at pixel x,y in DSSAT needs to go back to the hydrology model and reduce the flow to the next pixels.

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 2:55 PM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

Michael

On Nov 21, 2017, at 1:59 PM, Gerald Charles Nelson notifications@github.com wrote:

I’d also suggest pointing out that this form of coupling is just a sophisticated version of sneakernet. Run model A and output relevant results to a text file. Model B uses the text file as one of its inputs. In the current setup, model B is running inside a docker container so at least in principle it is system-agnostic. But a compiled version of Model B that is system specific could also make use of the text file.

Now that Allen and Eric (and Jordan?) have the models running separately and sharing data in one direction, do they have any thoughts about how to set up two way interactions via BMI?

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 11:59 AM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

Seems like a good idea to do a flow diagram of the steps needed for model coupling in general and specifically with this pilot project.

Michael

On Nov 21, 2017, at 11:53 AM, Gerald Charles Nelson notifications@github.com wrote:

I’d say a map of water availability per pixel with no flow and one with. Or a delta between the two.

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 11:51 AM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

This is all wonderful news. If we end up running DSSAT on a grid, we'll need to run it twice: once with the default values, and again with the hydrology model generated values.

I'll want to have the raster grid (what is the format???) of the flow accumulation from the hydrology model, and the grids for DSSAT. Definitely yield and maybe biomass too if that is generated by default. Any other values I might want to show??

BTW, does the hydrology model have a name?

Michael

On Nov 21, 2017, at 12:14 AM, A Lee notifications@github.com wrote:

Great, thanks Eric! Sure, I can grab them from FTP, want to point me at it via email?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-345937469, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVrxyIQ4IMXFdBlDz3H7zVKu4XADv0ks5s4nhRgaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346125165, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVr_MQP8WScnA48Jdv2J3L6R5vvrMrks5s4xw2gaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346159184, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVr3GGp71Uj15C3d_oBrsivChB7a-Gks5s4znJgaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

cmbarton commented 7 years ago

No. This is one-way interaction. But it opens the potential for on the fly, dynamic model coupling. Via more complete BMI, such 2-way coupling could be created. But this is the first real step in implementing this idea and I'm very happy that it actually works by mid-November.

Michael

On Nov 21, 2017, at 3:02 PM, Gerald Charles Nelson notifications@github.com wrote:

I don’t think that gets at the ‘real’ two-way interaction we want. Infiltration at pixel x,y in DSSAT needs to go back to the hydrology model and reduce the flow to the next pixels.

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 2:55 PM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

Since Eric is autogenerating the WTH files, it is simply a matter of updating the script that runs the hydrology model in Docker and BMI, and parses output into DSSAT format to also initiate DSSAT via BMI and Docker.

Michael

On Nov 21, 2017, at 1:59 PM, Gerald Charles Nelson notifications@github.com wrote:

I’d also suggest pointing out that this form of coupling is just a sophisticated version of sneakernet. Run model A and output relevant results to a text file. Model B uses the text file as one of its inputs. In the current setup, model B is running inside a docker container so at least in principle it is system-agnostic. But a compiled version of Model B that is system specific could also make use of the text file.

Now that Allen and Eric (and Jordan?) have the models running separately and sharing data in one direction, do they have any thoughts about how to set up two way interactions via BMI?

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 11:59 AM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

Seems like a good idea to do a flow diagram of the steps needed for model coupling in general and specifically with this pilot project.

Michael

On Nov 21, 2017, at 11:53 AM, Gerald Charles Nelson notifications@github.com wrote:

I’d say a map of water availability per pixel with no flow and one with. Or a delta between the two.

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 11:51 AM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

This is all wonderful news. If we end up running DSSAT on a grid, we'll need to run it twice: once with the default values, and again with the hydrology model generated values.

I'll want to have the raster grid (what is the format???) of the flow accumulation from the hydrology model, and the grids for DSSAT. Definitely yield and maybe biomass too if that is generated by default. Any other values I might want to show??

BTW, does the hydrology model have a name?

Michael

On Nov 21, 2017, at 12:14 AM, A Lee notifications@github.com wrote:

Great, thanks Eric! Sure, I can grab them from FTP, want to point me at it via email?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-345937469, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVrxyIQ4IMXFdBlDz3H7zVKu4XADv0ks5s4nhRgaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346125165, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVr_MQP8WScnA48Jdv2J3L6R5vvrMrks5s4xw2gaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346159184, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVr3GGp71Uj15C3d_oBrsivChB7a-Gks5s4znJgaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346175860, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVrzfXWawhl4dJWLzpzate6VhIpqdDks5s40iQgaJpZM4QixHA.

GeraldCNelson commented 7 years ago

You just don’t have enough faith in people ☺

My comment about 2 way coupling referred to what goes in the flow chart.

But I still would like to hear from Eric and Allen about what they see as challenges to two way coupling for this particular combination.

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 3:12 PM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

Michael

On Nov 21, 2017, at 3:02 PM, Gerald Charles Nelson notifications@github.com wrote:

I don’t think that gets at the ‘real’ two-way interaction we want. Infiltration at pixel x,y in DSSAT needs to go back to the hydrology model and reduce the flow to the next pixels.

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 2:55 PM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

Since Eric is autogenerating the WTH files, it is simply a matter of updating the script that runs the hydrology model in Docker and BMI, and parses output into DSSAT format to also initiate DSSAT via BMI and Docker.

Michael

On Nov 21, 2017, at 1:59 PM, Gerald Charles Nelson notifications@github.com wrote:

I’d also suggest pointing out that this form of coupling is just a sophisticated version of sneakernet. Run model A and output relevant results to a text file. Model B uses the text file as one of its inputs. In the current setup, model B is running inside a docker container so at least in principle it is system-agnostic. But a compiled version of Model B that is system specific could also make use of the text file.

Now that Allen and Eric (and Jordan?) have the models running separately and sharing data in one direction, do they have any thoughts about how to set up two way interactions via BMI?

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 11:59 AM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

Seems like a good idea to do a flow diagram of the steps needed for model coupling in general and specifically with this pilot project.

Michael

On Nov 21, 2017, at 11:53 AM, Gerald Charles Nelson notifications@github.com wrote:

I’d say a map of water availability per pixel with no flow and one with. Or a delta between the two.

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 11:51 AM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

This is all wonderful news. If we end up running DSSAT on a grid, we'll need to run it twice: once with the default values, and again with the hydrology model generated values.

I'll want to have the raster grid (what is the format???) of the flow accumulation from the hydrology model, and the grids for DSSAT. Definitely yield and maybe biomass too if that is generated by default. Any other values I might want to show??

BTW, does the hydrology model have a name?

Michael

On Nov 21, 2017, at 12:14 AM, A Lee notifications@github.com wrote:

Great, thanks Eric! Sure, I can grab them from FTP, want to point me at it via email?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-345937469, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVrxyIQ4IMXFdBlDz3H7zVKu4XADv0ks5s4nhRgaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346125165, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVr_MQP8WScnA48Jdv2J3L6R5vvrMrks5s4xw2gaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346159184, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVr3GGp71Uj15C3d_oBrsivChB7a-Gks5s4znJgaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346175860, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVrzfXWawhl4dJWLzpzate6VhIpqdDks5s40iQgaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

cmbarton commented 7 years ago

We DO want to get to this 2-way coupling but this is a start. If I include it in the flow chart, I'll need to use a dashed line and some question marks.

Michael

On Nov 21, 2017, at 3:15 PM, Gerald Charles Nelson notifications@github.com wrote:

You just don’t have enough faith in people ☺

My comment about 2 way coupling referred to what goes in the flow chart.

But I still would like to hear from Eric and Allen about what they see as challenges to two way coupling for this particular combination.

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 3:12 PM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

No. This is one-way interaction. But it opens the potential for on the fly, dynamic model coupling. Via more complete BMI, such 2-way coupling could be created. But this is the first real step in implementing this idea and I'm very happy that it actually works by mid-November.

Michael

On Nov 21, 2017, at 3:02 PM, Gerald Charles Nelson notifications@github.com wrote:

I don’t think that gets at the ‘real’ two-way interaction we want. Infiltration at pixel x,y in DSSAT needs to go back to the hydrology model and reduce the flow to the next pixels.

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 2:55 PM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

Since Eric is autogenerating the WTH files, it is simply a matter of updating the script that runs the hydrology model in Docker and BMI, and parses output into DSSAT format to also initiate DSSAT via BMI and Docker.

Michael

On Nov 21, 2017, at 1:59 PM, Gerald Charles Nelson notifications@github.com wrote:

I’d also suggest pointing out that this form of coupling is just a sophisticated version of sneakernet. Run model A and output relevant results to a text file. Model B uses the text file as one of its inputs. In the current setup, model B is running inside a docker container so at least in principle it is system-agnostic. But a compiled version of Model B that is system specific could also make use of the text file.

Now that Allen and Eric (and Jordan?) have the models running separately and sharing data in one direction, do they have any thoughts about how to set up two way interactions via BMI?

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 11:59 AM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

Seems like a good idea to do a flow diagram of the steps needed for model coupling in general and specifically with this pilot project.

Michael

On Nov 21, 2017, at 11:53 AM, Gerald Charles Nelson notifications@github.com wrote:

I’d say a map of water availability per pixel with no flow and one with. Or a delta between the two.

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: Michael Barton notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 11:51 AM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

This is all wonderful news. If we end up running DSSAT on a grid, we'll need to run it twice: once with the default values, and again with the hydrology model generated values.

I'll want to have the raster grid (what is the format???) of the flow accumulation from the hydrology model, and the grids for DSSAT. Definitely yield and maybe biomass too if that is generated by default. Any other values I might want to show??

BTW, does the hydrology model have a name?

Michael

On Nov 21, 2017, at 12:14 AM, A Lee notifications@github.com wrote:

Great, thanks Eric! Sure, I can grab them from FTP, want to point me at it via email?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-345937469, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVrxyIQ4IMXFdBlDz3H7zVKu4XADv0ks5s4nhRgaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346125165, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVr_MQP8WScnA48Jdv2J3L6R5vvrMrks5s4xw2gaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346159184, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVr3GGp71Uj15C3d_oBrsivChB7a-Gks5s4znJgaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346175860, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVrzfXWawhl4dJWLzpzate6VhIpqdDks5s40iQgaJpZM4QixHA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346178880, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVr32gVSHs1gdkgUeO1PQsqbUksDM4ks5s40t8gaJpZM4QixHA.

alee commented 7 years ago

@cmbarton Can you clarify what you mean by running DSSAT twice on each grid cell? We only have the baseline original input files that had weather station data for somewhere in Ames and were used as initial input to the hydrology model to route precipitation across the landscape. So we only really need to run it once (and we already have) to generate the original baseline values. The generated sample data currently live at https://github.com/comses/ming/tree/master/results/sample

cmbarton commented 7 years ago

You are right. I forgot that the water is the same throughout the entire grid without hydrological routing. So a difference map is simply the routed crop model output minus the single value from the default run (we are assuming the same soil throughout).

Michael Barton

On Nov 21, 2017, at 4:09 PM, A Lee notifications@github.com wrote:

@cmbarton https://github.com/cmbarton Can you clarify what you mean by running DSSAT twice on each grid cell? We only have the baseline original input files that had weather station data for somewhere in Ames and were used as initial input to the hydrology model to route precipitation across the landscape. So we only really need to run it once (and we already have) to generate the original baseline values. This is all at https://github.com/comses/ming/tree/master/results/sample https://github.com/comses/ming/tree/master/results/sample — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/comses/ming/issues/9#issuecomment-346191191, or mute the thread https://github.com/notifications/unsubscribe-auth/AIjVr1Vr7kr_TdeS19NfyxUtdMeh3_phks5s41gkgaJpZM4QixHA.

GeraldCNelson commented 7 years ago

I hope you all had a great Thanksgiving! Mine was!

I thought it might be helpful to get a report on where things stand on getting the AGU presentation ready. At some point it might be good to have a skype based tutorial on setting up and running all this stuff.

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: A Lee notifications@github.com Reply-To: comses/ming reply@reply.github.com Date: Tuesday, November 21, 2017 at 12:14 AM To: comses/ming ming@noreply.github.com Cc: Gerald Nelson nelson.gerald.c@gmail.com, Mention mention@noreply.github.com Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

Great, thanks Eric! Sure, I can grab them from FTP, want to point me at it via email?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

cmbarton commented 7 years ago

I started in on a presentation over Thanksgiving. With the CSDC move and last week of classes, I haven't been able to work on it Mon/Tues but will get back to it soon. I'll need some graphics like we discussed, but have an idea for how to structure the talk.

Michael

Michael Barton

On Nov 27, 2017, at 2:57 PM, Gerald Charles Nelson notifications@github.com<mailto:notifications@github.com> wrote:

I hope you all had a great Thanksgiving! Mine was!

Gerald C. Nelson

Professor Emeritus, UIUC

+1 217-390-7888 (cell)

+1 970-639-2079 (land line)

http://bit.ly/1arho7d

From: A Lee notifications@github.com<mailto:notifications@github.com> Reply-To: comses/ming reply@reply.github.com<mailto:reply@reply.github.com> Date: Tuesday, November 21, 2017 at 12:14 AM To: comses/ming ming@noreply.github.com<mailto:ming@noreply.github.com> Cc: Gerald Nelson nelson.gerald.c@gmail.com<mailto:nelson.gerald.c@gmail.com>, Mention mention@noreply.github.com<mailto:mention@noreply.github.com> Subject: Re: [comses/ming] script dssat runs on hydrology model outputs (#9)

Great, thanks Eric! Sure, I can grab them from FTP, want to point me at it via email?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/comses/ming/issues/9#issuecomment-347342110, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AIjVr4STKhSsg5oncD22rLIcDZBGs4USks5s6zA7gaJpZM4QixHA.