search

ufs-community / UFS_UTILS

Utilities for the NCEP models.
Other
21 stars 104 forks source link

Make chgres_cube.fd read operational GFS FV3 IC to generate FV3 WAM IC #165

Closed HenryJuang-NOAA closed 3 years ago

HenryJuang-NOAA commented 3 years ago

We have done the checking on reading GFS GSM IC with tracers of spfh, clwmr, o3mr to WAM IC with the same kinds of tracers. We have to make chgres_cube can read current GFS IC (FV3 IC) and possible to add more tracers, such as o and o2 for WAM IC.

HenryJuang-NOAA commented 3 years ago

With the help from Kate Friedman, I can get GFS initial condition (IC) of FV3GFS v15 or v16. With the help from George Gayno, I had all the required configure nml files for GSM, v15, and v16 ICs to run chgres_cube. With the works have been tested by me in issues #163 and #164, we can run chgres_cube with different ICs to generate WAM IC. The next work is to pull request and commit my working version merged to develop branch with regressing test.

edwardhartnett commented 3 years ago

We also need a test.

HenryJuang-NOAA commented 3 years ago

You mean IC data to test, right? I am still checking the code, I will provide later.

On Fri, Feb 5, 2021 at 10:16 AM Edward Hartnett notifications@github.com wrote:

We also need a test.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/UFS_UTILS/issues/165#issuecomment-774094414, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALRWWEKGNZTWI6IZMGHKJ3LS5QDS5ANCNFSM4SE7YDXA .

HenryJuang-NOAA commented 3 years ago

The issues 0f #163 and #164 and this issue #165 have been resolved with newly created WAM IC tested successfully in FV3 WAM. The WAM IC is from my branch called udhmhj.02 under https://github.com/HenryJuang-NOAA/UFS_UTILS.git And the FV3WAM is in my branch called udhmhj.06 under
https://github.com/HenryJuang-NOAA/ufs-weather-model https://github.com/HenryJuang-NOAA/fv3atm https://github.com/HenryJuang-NOAA/GFDL_atmos_cubed_sphere https://github.com/HenryJuang-NOAA/ccpp-physics

To create WAM IC, use my branch udhmhj.02, the test location is ud/chgres_wam. There are three nml files point to gsm, v15, and v16 IC datasets and the run.sh can be submitted to get WAM IC from these nml files, one by one as needed.

HenryJuang-NOAA commented 3 years ago

This issue has been finished and ready for pull request. What is the unit test? something like regression test?

edwardhartnett commented 3 years ago

OK I don't think we can expect unit testing from this code. We need to provide better examples of what unit testing is first.

So if we have adequate regression testing, we can go with that.

HenryJuang-NOAA commented 3 years ago

I didn't build a regression test for it, but I have GSM, V15, and V16 IC and workflow to run chgres)cube with WAM option to create WAM IC, with G. Gayno's help. Ed, do you want WAM regression build?

edwardhartnett commented 3 years ago

I don't understand much of what you are asking.

You don't have a regression test or unit testing? We certainly need one or the other.

Can you explain to me how this code should be tested, if it has to change?

HenryJuang-NOAA commented 3 years ago

This issue is to create a WAM IC(initial condition) from operational GS or FV3 GFS V15, or V16 IC. WAM is a whole atmospheric model based on FV3 GFS with model top up into thermosphere (model top pressure is about 10^-7 pascal). To have the WAM IC, we need temperature profile and O, O2, O3, and remain N2 profiles from current GFS model top to WAM model top. There is a new file contains all necessary data and routines to create the vertical temperature and compositions profiles in function of latitudes and height. I created a subdirectory called ud/ to have workflow to test WAM IC, I have some GFS IC in my own directory. Thus, if we want to create a regression test for WAM in chgres_cube then we need to move the workflow of WAM IC and save GFS IC as an example under UFS_UTILS regression test system. It is what I was asking.

edwardhartnett commented 3 years ago

Then yes, we need the regression test.

HenryJuang-NOAA commented 3 years ago

Do we have any document which describes how to create baseline and do reg test? The subdirectory reg_tests/chgres_cube should be the place I can add reg_test into driver.xxx.sh with c96.fv3.wam.sh which I can follow c96.fv3.restart.sh etc to create. But I can access only some machines, for example, I have no account on jet. I can establish in hera first, is that okay?

GeorgeGayno-NOAA commented 3 years ago

Do we have any document which describes how to create baseline and do reg test? The subdirectory reg_tests/chgres_cube should be the place I can add reg_test into driver.xxx.sh with c96.fv3.wam.sh which I can follow c96.fv3.restart.sh etc to create. But I can access only some machines, for example, I have no account on jet. I can establish in hera first, is that okay?

Yes, go ahead and get it working on Hera for now.

HenryJuang-NOAA commented 3 years ago

I will create baseline in my directory and ask to save into HOMEreg later.

HenryJuang-NOAA commented 3 years ago

I believe I have done all places to document. Really don't know how to debug it to finish.... help!!

GeorgeGayno-NOAA commented 3 years ago

I believe I have done all places to document. Really don't know how to debug it to finish.... help!!

@HenryJuang-NOAA Please merge the latest updates from 'develop'. Then I can look for doxygen warnings.

HenryJuang-NOAA commented 3 years ago

updated develop with upstream, and merged to udhmhj.02, pushed.

On Wed, Mar 24, 2021 at 4:20 PM GeorgeGayno-NOAA @.***> wrote:

I believe I have done all places to document. Really don't know how to debug it to finish.... help!!

@HenryJuang-NOAA https://github.com/HenryJuang-NOAA Please merge the latest updates from 'develop'. Then I can look for doxygen warnings.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/UFS_UTILS/issues/165#issuecomment-806160528, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALRWWEMJTS5NERBIMWNQ7BTTFJCQPANCNFSM4SE7YDXA .

HenryJuang-NOAA commented 3 years ago

George, I checked the last push for udhmhj.02, there is no fail jobs. Doxygen doc is passed, right? Since this a dry-run, and I had a ud/ which I don't need to merge back to develop. What should I do next? How about create a new branch without ud/, then do real pull request to merge back to master develop? The regress test for wam ic can be another issue for future pull request, what do you think?

GeorgeGayno-NOAA commented 3 years ago

George, I checked the last push for udhmhj.02, there is no fail jobs. Doxygen doc is passed, right? Since this a dry-run, and I had a ud/ which I don't need to merge back to develop. What should I do next? How about create a new branch without ud/, then do real pull request to merge back to master develop? The regress test for wam ic can be another issue for future pull request, what do you think?

The ./ud directory must be deleted before merging. You can either keep those files locally or add them to another branch in your fork that won't be merged.

A regression test should be included in this PR.

HenryJuang-NOAA commented 3 years ago

George, I am doing to create reg_test for wam ic. What input data do you think I should use? or I have to have all different ic for wam ic? I need cold start wam ic from any archived data, say GFS GSM or GFS FV3, maybe for future cases, we should use operational GFS FV3 restart files or analyzed data ic? need your suggestion...

GeorgeGayno-NOAA commented 3 years ago

George, I am doing to create reg_test for wam ic. What input data do you think I should use? or I have to have all different ic for wam ic? I need cold start wam ic from any archived data, say GFS GSM or GFS FV3, maybe for future cases, we should use operational GFS FV3 restart files or analyzed data ic? need your suggestion...

Initially, I would not create a test for every possible input data type. You can pick the input data you think most folks will use. Another consideration: the v16 data has a much higher model top than v15. Which is the better test of your algorithm?

HenryJuang-NOAA commented 3 years ago

Agree. Which input in your input_data directory is v16?

On Fri, Mar 26, 2021 at 1:52 PM GeorgeGayno-NOAA @.***> wrote:

George, I am doing to create reg_test for wam ic. What input data do you think I should use? or I have to have all different ic for wam ic? I need cold start wam ic from any archived data, say GFS GSM or GFS FV3, maybe for future cases, we should use operational GFS FV3 restart files or analyzed data ic? need your suggestion...

Initially, I would not create a test for every possible input data type. You can pick the input data you think most folks will use. Another consideration: the v16 data has a much higher model top than v15. Which is the better test of your algorithm?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/UFS_UTILS/issues/165#issuecomment-808409849, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALRWWEOK36RXMBLBLIQ3IG3TFTCW3ANCNFSM4SE7YDXA .

GeorgeGayno-NOAA commented 3 years ago

Agree. Which input in your input_data directory is v16?

/scratch1/NCEPDEV/da/George.Gayno/noscrub/reg_tests/chgres_cube/input_data/fv3.netcdf

HenryJuang-NOAA commented 3 years ago

Okay! So I can start from c96.fv3.netcdf.sh to create c96.fv3.netcdf2wam.sh. Is it named okay? Well, let me do it first, we can change the file name later if we want. Thanks.

On Fri, Mar 26, 2021 at 2:23 PM GeorgeGayno-NOAA @.***> wrote:

Agree. Which input in your input_data directory is v16?

/scratch1/NCEPDEV/da/George.Gayno/noscrub/reg_tests/chgres_cube/input_data/fv3.netcdf

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/UFS_UTILS/issues/165#issuecomment-808427820, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALRWWEMM26QX6EWSLJU2MZDTFTGJJANCNFSM4SE7YDXA .

HenryJuang-NOAA commented 3 years ago

George, I have done reg_tests for this issue by adding c96.fv3.netcdf2wam.sh into all drivers, but only tested in hera. The netcdf2wam baseline data I saved to compare is under /scratch1/NCEPDEV/global/Henry.Juang/baseline_data/c96_fv3_netcdf2wam Please copy to your baseline_data place, then I will modify sh file to point to your baseline location. I think this issue can be done with PR as into a final step.

HenryJuang-NOAA commented 3 years ago

George

I need your help to copy baseline of WAM IC regtest. Thanks,

Henry

On Fri, Mar 26, 2021 at 2:23 PM GeorgeGayno-NOAA @.***> wrote:

Agree. Which input in your input_data directory is v16?

/scratch1/NCEPDEV/da/George.Gayno/noscrub/reg_tests/chgres_cube/input_data/fv3.netcdf

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/UFS_UTILS/issues/165#issuecomment-808427820, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALRWWEMM26QX6EWSLJU2MZDTFTGJJANCNFSM4SE7YDXA .

HenryJuang-NOAA commented 3 years ago

Thanks.

On Mon, May 24, 2021 at 10:09 AM GeorgeGayno-NOAA @.***> wrote:

Closed #165 https://github.com/NOAA-EMC/UFS_UTILS/issues/165 via d049b28 https://github.com/NOAA-EMC/UFS_UTILS/commit/d049b2880178c9823c5ce28082eb1d0ab28ca21c .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/UFS_UTILS/issues/165#event-4786908506, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALRWWELG4C4OUE4U45PSJTDTPJMYTANCNFSM4SE7YDXA .