Add output plots

[dmey]

This PR adds two scripts to run (bash) all example cases and plot (matlab) case 201. It also adds the plotted outputs to the readme and examples.

[bss116]

I was thinking we would just give one example of an output plot with interesting data, rather than plots for each of the example cases. With the current runtimes it is clear that they look sad, and when we set up the examples we intentionally chose low resolution to make these feasible to run on a local machine. For proper results, you would need to run simulations in a HPC setting.

002 and 501 output vertical profiles (1D statistics) that can be plotted.

yes unfortunately there is no mask output for the blocks, this would be a good improvement to the output files. It is not so easy to implement because of the staggered grid where the variables are defined on different parts of the grid cell...

[dmey]

I was thinking we would just give one example of an output plot with interesting data, rather than plots for each of the example cases. With the current runtimes it is clear that they look sad, and when we set up the examples we intentionally chose low resolution to make these feasible to run on a local machine. For proper results, you would need to run simulations in a HPC setting.

002 and 501 output vertical profiles (1D statistics) that can be plotted.

yes unfortunately there is no mask output for the blocks, this would be a good improvement to the output files. It is not so easy to implement because of the staggered grid where the variables are defined on different parts of the grid cell...

OK shall we maybe show 502 then instead od 201. I thought that 201 would be more relevant given the energy balance but 502 has already well developed inputs. We can show that in the summary paper as well under example application. Any thoughts?

[samoliverowens]

@dmey you mentioned also including buildings in the output plots - do you mean something like this?

[dmey]

@dmey you mentioned also including buildings in the output plots - do you mean something like this?

That looks great!

[samoliverowens]

We could even do a 3D plot like this.

[dmey]

Even better 2 for 1 -- we show both domain and slice of the flow. That's great!

[bss116]

I would show both. The second one is nice to understand the location of the slice, but in the first one you see what's going on in between the blocks so there is more information about the field. What about if you pack these two plots plus the plain geometry plot into an extra Matlab script and we reference it together with showing these two plots at the end of the examples-docs?

[samoliverowens]

I would show both. The second one is nice to understand the location of the slice, but in the first one you see what's going on in between the blocks so there is more information about the field. What about if you pack these two plots plus the plain geometry plot into an extra Matlab script and we reference it together with showing these two plots at the end of the examples-docs?

Ok sounds good, I'll make a script with some functions. I'm also sporadically working on a Matlab post-processing setup, so this will be useful for that too.

[bss116]

I would show both. The second one is nice to understand the location of the slice, but in the first one you see what's going on in between the blocks so there is more information about the field. What about if you pack these two plots plus the plain geometry plot into an extra Matlab script and we reference it together with showing these two plots at the end of the examples-docs?

Ok sounds good, I'll make a script with some functions. I'm also sporadically working on a Matlab post-processing setup, so this will be useful for that too.

Great!

[dmey]

Ok sounds good, I'll make a script with some functions. I'm also sporadically working on a Matlab post-processing setup, so this will be useful for that too.

OK great thanks. So I will keep the script to run the examples in here but will remove the python plotting one.

[bss116]

Another thing @samoliverowens: I think it would be good to change the example geometry plots to the same view angle as you have in these plots here. I start to see some optical illusions when staring for too long at 102 with the grid origin completely centred 😄

OK great thanks. So I will keep the script to run the examples in here but will remove the python plotting one.

@dmey If there is anything in the plotting scripts you think might be useful to keep, you can also just move them to the python-processing branch. We still need to discuss this, but I think it might be useful to keep the branch in, as a basis to work on for someone who doesn't want to use/has a Matlab license

[dmey]

@dmey If there is anything in the plotting scripts you think might be useful to keep, you can also just move them to the python-processing branch. We still need to discuss this, but I think it might be useful to keep the branch in, as a basis to work on for someone who doesn't want to use/has a Matlab license

Plotting and slicing is really just 2 lines of code so I don;t think there is much of a point keeping this code in like this. We can defiantly discuss the Python stuff at a later point but given that the Matlab stuff is already in here and works, I would just target Matlab for now. I have not tried this out yet but if the functions are standard enough, they should also work in https://www.gnu.org/software/octave/index. @samoliverowens when you add the plotting script, if you modify DEVLOP.md with instructions on how to create the plots in MATLAB then I can try in https://www.gnu.org/software/octave/index.

[samoliverowens]

@dmey If there is anything in the plotting scripts you think might be useful to keep, you can also just move them to the python-processing branch. We still need to discuss this, but I think it might be useful to keep the branch in, as a basis to work on for someone who doesn't want to use/has a Matlab license

Plotting and slicing is really just 2 lines of code so I don;t think there is much of a point keeping this code in like this. We can defiantly discuss the Python stuff at a later point but given that the Matlab stuff is already in here and works, I would just target Matlab for now. I have not tried this out yet but if the functions are standard enough, they should also work in https://www.gnu.org/software/octave/index. @samoliverowens when you add the plotting script, if you modify DEVLOP.md with instructions on how to create the plots in MATLAB then I can try in https://www.gnu.org/software/octave/index.

Is it best if I put this into one script that plots the slice (in 2D and 3D)? Rather than in functions I mean.

[samoliverowens]

I have also experimented with the building appearance, let me know what you prefer...

[dmey]

Is it best if I put this into one script that plots the slice (in 2D and 3D)? Rather than in functions I mean.

In general something that we can run from the command line makes things easier when we need to re-run things in the future however if this is tricky in Matlab, feel free to just add a function and detail what steps are needed to generate the plots.

[samoliverowens]

Is it best if I put this into one script that plots the slice (in 2D and 3D)? Rather than in functions I mean.

In general something that we can run from the command line makes things easier when we need to re-run things in the future however if this is tricky in Matlab, feel free to just add a function and detail what steps are needed to generate the plots.

Ok I'll put it all in one script that can be run from the command line. This will involve hard-coding things like the experiment number (102), the field we're plotting (u), the slice plane (xz) and location (y=31.5), and the time (t ~ 1010s) - is that ok?

[dmey]

I have also experimented with the building appearance, let me know what you prefer...

@samoliverowens from my side I like this layout. One thing though I would probably switch to a diverging colourmap (e.g. like a red blue https://blogs.mathworks.com/pick/2011/12/09/a-vivid-colormap/)

[dmey]

Ok I'll put it all in one script that can be run from the command line. This will involve hard-coding things like the experiment number (102), the field we're plotting (u), the slice plane (xz) and location (y=31.5), and the time (t ~ 1010s) - is that ok?

Sure this is perfect! We will only need this for 102 so that's fine 👍

[samoliverowens]

I have also experimented with the building appearance, let me know what you prefer...

@samoliverowens from my side I like this layout. One thing though I would probably switch to a diverging colourmap (e.g. like a red blue https://blogs.mathworks.com/pick/2011/12/09/a-vivid-colormap/)

Ok will do - the red/blue I had previously is not one of Matlab's, so I took it out, but I'll add it in again.

https://www.mathworks.com/matlabcentral/fileexchange/25536-red-blue-colormap

[dmey]

Ok will do - the red/blue I had previously is not one of Matlab's, so I took it out, but I'll add it in again. https://www.mathworks.com/matlabcentral/fileexchange/25536-red-blue-colormap

I see -- OK then maybe it's just best to leave it as is as it may not work with Octave.

[samoliverowens]

Ok will do - the red/blue I had previously is not one of Matlab's, so I took it out, but I'll add it in again. https://www.mathworks.com/matlabcentral/fileexchange/25536-red-blue-colormap

I see -- OK then maybe it's just best to leave it as is as it may not work with Octave.

It's just a function that I can add to the script, so it should work I think. If not it can be defined in the main body of the script.

[dmey]

It's just a function that I can add to the script, so it should work I think. If not it can be defined in the main body of the script.

Great and sounds good

[samoliverowens]

@dmey shall I put this script somewhere in this branch? What should I call it?

[dmey]

@dmey shall I put this script somewhere in this branch? What should I call it?

@samoliverowens sure and if this is only for this example (201), how about adding it under tools/examples and name something like plot_example_201?

[samoliverowens]

Alright I have put it in tools/examples and called it plot_fielddump_slice to be a bit more specific, and I think don't have the experiment number suffixed.

[dmey]

@samoliverowens I have just checked the plots -- instead of showing them, can you just save plots to the current folder without pop ups?

[dmey]

@samoliverowens thanks. I have not merged branch add-figures into this and we should just merge this into master. Can you please give it a quick look and approve it so that I can merge it in. After this I will only need to create one for cleaning up the scripts and I think we are done. I will add these output plots and a couple of paragraphs to the example section of the paper at the weekend, have you had any luck with the rest?

[samoliverowens]

@samoliverowens thanks. I have not merged branch add-figures into this and we should just merge this into master. Can you please give it a quick look and approve it so that I can merge it in. After this I will only need to create one for cleaning up the scripts and I think we are done. I will add these output plots and a couple of paragraphs to the example section of the paper at the weekend, have you had any luck with the rest?

Yes I'll have a look at this PR today. Once it's approved I'll do a PR for my so/misc branch in order to close #118.

I've been busier than anticipated over the last week so I haven't gotten round to the paper, but from next week I should be able to get started.

[samoliverowens]

@dmey shall I make a script that plots the blocks in the style of the output plots?

[dmey]

@dmey shall I make a script that plots the blocks in the style of the output plots?

That'd be good but if that's not trivial and will take some time to write, maybe it's best to just to explain how to generate them with the current script in DEVELOP -- I left a TODO at https://github.com/uDALES/u-dales/pull/125/files#diff-234666c668a956cce482e25e1a9399c1d01026f259c36e0d721d5d30456ab6b5R53

[samoliverowens]

@dmey shall I make a script that plots the blocks in the style of the output plots?

That'd be good but if that's not trivial and will take some time to write, maybe it's best to just to explain how to generate them with the current script in DEVELOP -- I left a TODO at https://github.com/uDALES/u-dales/pull/125/files#diff-234666c668a956cce482e25e1a9399c1d01026f259c36e0d721d5d30456ab6b5R53

It's a bit ugly to get the ygrid, but it's fine. The script writes the images to the experiment directory, feel free to change it to something else.

[dmey]

It's a bit ugly to get the ygrid, but it's fine. The script writes the images to the experiment directory, feel free to change it to something else.

I think this is good -- it does what's supposed to! :) I have just added some info on how to run these in 19b2f1f and added the outputs to the gitignore... For me this is good to go in

[dmey]

BTW, re Octave, I did have a go at it later but it'd require some changes in the script to make things work so I would just leave it as is and just keep MATLAB as a dependency for plotting.

[bss116]

In the example docs, I would add an extra section (+ add to table of contents) where you write how to plot the outputs, or show the plots directly next to the input geometry and reference where the description on how to plot things is right in the beginning

[samoliverowens]

Shall I also turn plot_fielddump_slice into a function so that it can be called for other cases?

[dmey]

Shall I also turn plot_fielddump_slice into a function so that it can be called for other cases?

Sounds good!

[samoliverowens]

This will involve hard-coding things like the experiment number (102), the field we're plotting (u), the slice plane (xz) and location (y=31.5), and the time (t ~ 1010s) - is that ok?

We could also make these things arguments to the function too (I'd probably do the index rather than location - j not y and n not t - because there normally won't be an exact match)

[dmey]

In the example docs, I would add an extra section (+ add to table of contents) where you write how to plot the outputs, or show the plots directly next to the input geometry and reference where the description on how to plot things is right in the beginning

I have added a couple of sentences about this at the beginning see 9bc5c5b

[dmey]

We could also make these things arguments to the function too (I'd probably do the index rather than location - j not y and n not t - because there normally won't be an exact match)

I'm OK with either -- at the moment the scripts work fine, things can always be generalized at a later stage.

[bss116]

In the example docs, I would add an extra section (+ add to table of contents) where you write how to plot the outputs, or show the plots directly next to the input geometry and reference where the description on how to plot things is right in the beginning

I have added a couple of sentences about this at the beginning see 9bc5c5b

ok, with that I would add the plots to the beginning of 102 --- now they just come under the "coldstart" heading, which does not make much sense.

[dmey]

ok, with that I would add the plots to the beginning of 102 --- now they just come under the "coldstart" heading, which does not make much sense.

great -- done in https://github.com/uDALES/u-dales/pull/125/commits/2b8fcba32b34aaa42168b36cdefa0c1d66693de3

[samoliverowens]

plot_fielddump_slice is now a function (c2a171a3d3794591e650c142a52f7fde3588c45c)- the current plots for 102 can be reproduced using matlab -nosplash -nodesktop -r "cd('tools/examples'); plot_fielddump_slice('102','u','y',32,1); quit"

[dmey]

plot_fielddump_slice is now a function (c2a171a)- the current plots for 102 can be reproduced using matlab -nosplash -nodesktop -r "cd('tools/examples'); plot_fielddump_slice('102','u','y',32,1); quit"

Great -- docs updated for this in https://github.com/uDALES/u-dales/pull/125/commits/cd465ad42f526dde355586d5f4310e62e89eb198

[samoliverowens]

@dmey I'll have a final look over this PR tomorrow!

[dmey]

@dmey I'll have a final look over this PR tomorrow!

Thanks and sorry I've just merged it -- it should be OK though and if there are minor things we can patch it next time. I will work on simplify the scripts now and will send you a PR for that 😋