Identify which part to extract from the existing code to slice out

m2lines / convection-parameterization-in-CAM

Deployment of the convection parameterization scheme (previously from within SAM) deployed into the CAM model

MIT License

3 stars 2 forks source link

Identify which part to extract from the existing code to slice out #1

Closed dorchard closed 1 year ago

dorchard commented 1 year ago

Current understanding of our starting point is: https://github.com/yaniyuval/Neural_nework_parameterization/blob/v.1.0.3/sam_code_NN/sam_cases/run_files_x8_5_layers/nn_convection_flux.f90
Which reqiures the netCDF file of weights trained from PyTorch at: https://github.com/yaniyuval/Neural_nework_parameterization/tree/v.1.0.3/NNs/qobsTTFFFFFTF30FFTFTF30TTFTFTFFF80FFTFTTF2699FFFF_X01_no_qp_no_adv_surf_F_Tin_qin_disteq_O_Trad_rest_Tadv_qadv_qout_qsed_RESCALED_7epochs_no_drop_REAL_NN_layers5in61out148_BN_F_te70.nc
[x] Extract nn_connvection_flux and its dependencies to this module

Questions

[ ] Once extracted how can we be sure it peforms as expected? Do we need to worry about that? (see https://github.com/m2lines/convection-parameterization-in-CAM/issues/13)
[ ] Can we run this independently to get a good idea that we have all the dependencies we need (and not severed any dependencies)?

jatkinson1000 commented 1 year ago

I've made a start on this in #7 Extracted and builds on command line. Could consider creating a small testbed program to test the module.

jatkinson1000 commented 1 year ago

Extraction completed in #7 and #8

Checking performance I'd suggest a testbed fortran program and comparison to the original pytorch? Added bonus of checking the extracted code built as a module is working correctly. This would be something for #9 (or possibly a new issue). @jdenholm and I could work together, him from the pytorch end, me from the Fortran?

jdenholm commented 1 year ago

If we can get the weights from the PyTorch model (presumably there is a PyTorch-compatible version somewhere) and the python files containing the model definition we can get some numbers to use as a test case. We don't really need realistic inputs to test the numbers which come out.

jatkinson1000 commented 1 year ago

Somewhere in here perhaps? https://github.com/yaniyuval/Neural_nework_parameterization/tree/main/NN_training

Makes me realise that we are probably also missing the .nc file with weights from the code we removed in NN_module/

jatkinson1000 commented 1 year ago

I have opened #13 as discussed above regarding testing, so would suggest we close this. This breaks the work into more manageable chunks as discussed at ICCS M2LiNES meeting.