Top constant flux layer

[cbegeman]

Adds the option to create a constant flux layer at the top of the model domain. Does not allow a simultaneous top and bottom constant flux layer.

Generalizes certain routines to take a surface structure array as an input so that these routines can be called with the top surface structure array in relevant cases.

[cbegeman]

@xylar Preparing some comparison figures. There's no rush to review.

[cbegeman]

@xylar Now ready for your review. I had to track down a few more inconsistencies in the way the bottom and top boundary were treated before getting satisfactory results. You should have received an email notification that a few files were ready for you to "take", these are namelist files for top and bottom constant flux layer cases using this branch (the master branch takes a logical value for constant_flux_layer, indicating the bottom boundary). You'll have to assign the namelist filename, excluding '_p3d' suffix to the test_case variable in the runfile run_script_lanl.

Here are the results after 10 h for those cases, which have a pressure gradient in the +u direction (N. Hemisphere):

[cbegeman]

I believe the mixing length is affected by https://github.com/xylar/palm_les_lanl/issues/35 (spike at the bottom boundary in top constant flux layer case).

[xylar]

Thanks! I’ll be more than happy to work on all of these after next week (EGU). I may need you to “give” the files again if the expire. BTW, I don’t get a notification when you use give so you’ll always have to email me, too, to let me know.

[cbegeman]

@xylar Sounds good. It's too bad the -m option for give (which is supposed to send an email) doesn't work.

[xylar]

Yeah, the IC machines can't reach the outside world to send email, as far as I know.

[xylar]

I did some testing of this branch today. The test_bottom... namelist file you sent me is for a restart run:

&initialization_parameters
        nx = 127, ny = 127, nz = 128,
        dx = 0.5, dy = 0.5, dz = 0.5,

        fft_method = 'temperton-algorithm',

        ocean = .T.,
        idealized_diurnal = .F.,

        initializing_actions = 'read_restart_data'
...

and I don't have any restart files so it crashed:

...
forrtl: severe (29): file not found, unit 13, file /lustre/scratch4/turquoise/.mdt1/xylar/palm/test_bottomsl_oceanml/RUN_ifort.grizzly_hdf5_mpirun_test_bottomsl_oceanml/BININ/_000000
...

The test_top... run seems to work. Could you tell me how many nodes you would suggest I run on? I'm using 1 Grizzly node (32 cores) at the moment.

Could you also post some instructions for making your plots above? I'd like to be able to reproduce them myself.

I'll do some code review here in a moment.

[xylar]

@lconlon, @vanroekel or @qingli411, it might be good if one of you runs a test with this branch before we merge just to make sure no harm is done to your existing work.

[xylar]

Nice! Thanks for the clean-up and the clarification on initializing (rather than restarting) the bottom-flux test.

I just created a repo for PALM LES viz, which you should get an invitation to join. I created an ice_shelf folder. Feel free to rename, edit, reorganize as you see fit.

[xylar]

I am, indeed, able to run the bottom-flux test with the change to "set_constant_profiles". I'm just waiting on some viz instructions to look at the results but as far as I'm concerned, this is ready to merge.

[cbegeman]

@lconlon, @vanroekel or @qingli411, it might be good if one of you runs a test with this branch before we merge just to make sure no harm is done to your existing work.

@lconlon, @vanroekel or @qingli411 did any of you want to run a test of this branch before I merge?

[vanroekel]

@cbegeman I'm on travel this week and it's unlikely I'll be able to run a test in a reasonable amount of time. @qingli411 or @lconlon could one of you rerun one of your tests with this branch to check for any changes.

[qingli411]

@cbegeman Thanks for checking! I will try to run a test on this branch today.

[cbegeman]

@qingli411 Thank you!

[qingli411]

@cbegeman Did you notice any slow-down of the run with this branch? I was comparing this branch with commit 4a02919 using a Langmuir turbulence case and found that it is more than 4 times slower... It didn't finish within the 2-hour wall time (the other run finished within half an hour) so I haven't checked the detailed timing and the results. Just submitted a new run and will compare the results later.

[cbegeman]

@qingli411 I didn't see a significant slowdown in the test cases I ran, but I was comparing the latest merged commit. I'll take a look at the timings also.

[cbegeman]

@qingli411 You need to run after commit 9e994cc, which removes conditionals in the additional timers. This is responsible for the slowdown in my runs.

[qingli411]

@cbegeman So this PR is before the slowdown issue being fixed and the fix has been merged into master. OK, I see. Thanks!

[qingli411]

Here is a comparison of the mean profiles (averaged over the last inertial period) in a Langmuir turbulence case between commit 4a02919 (dashed line) and this branch (solid line). https://portal.nersc.gov/project/acme/qingli/plot_palm_mean_profile_test_top_constant_flux_layer-ddd0281.html The mean profiles generally agree with each other between the two runs. Given that these runs are relatively short (1 day), I think I'm fine with the small differences between these two. Note that I didn't activate constant_flux_layer for both runs.

@cbegeman Thanks for adding these new features!

[cbegeman]

@qingli411 Thanks for testing. For your information, these differences are likely due to the changes to rho_ocean terms merged with commit 31a24f8 and not the changes made in this branch. There are essentially no changes made to the code when constant_flux_layer is disabled.