Running ICON at DKRZ

[VeeramanikandanR]

Hi,

I am trying to use PAMTRA+ICON for simulating radar reflectivity along a ship track (I am using readIcon2momMeteogram). I installed PAMTRA in the local machine as well as at DKRZ and ran an example case (pamtra_icon_groundbased.ipynb). The radar reflectivity plot at DKRZ is getting cropped (as shown below - rightside), however the plot from local machine is correct. The same issue exists for my actual case too.

I have also copied the installation process which I followed in the below table. May I know if someone has encountered a similar issue? Thanks and Regards, Veera

[DaveOri]

Hi Veera. Thanks for using PAMTRA and thanks for the very detailed issue you posted. From the plot you posted, it seems likely that the issue comes from the computing of the hydrometer attenuation (the reflectivity fields are cropped from one level upwards), if this is true the MDV plot should look OK whereas the DWR plot should be cropped like the Z plot you posted. Could you perhaps confirm that by modifying the plotting routines by taking out -Aa from the reflectivity plot? We are not that experienced in the intel compiler, I will try to reproduce your bug today, but I need some time to fetch the tools.

Davide

[VeeramanikandanR]

Hi Davide, Thank you for your suggestions. MDV plot is also cropped (shown below) just like Z. All the plots look qualitatively similar even while taking out Aa. When I checked the nc output files, Z, MDV and DWR at DKRZ are filled with -9999.f in the places where there are significant values at local machine. Thanks and Regards, Veera

[maahn]

The example uses the parallel version of Pamtra which runs every profile separately. Apparently, the process dies somewhere on the way which means that your profiles are cropped becuase they are not finished. Pamtra has plenty of debug output, but unfortunately Jupyter Notebook doesn't show the output of Fortran code which is a known limitation. So please run the example from the terminal with ipython:

  ipython pamtra_icon_groundbased.ipynb

Also, debugging output is much easier to understand when you use the non-parallel version, so change pam.runParallelPamtra(np.array([9.4]), pp_deltaX=1, pp_deltaY=1, pp_deltaF=1, pp_local_workers=cores) to pam.runPamtra(np.array([9.4])) and let us know what you see. You can also increase the verbosity level with

pam.set['verbose'] = 5
pam.set['pyVerbose'] = 5 

Max

[VeeramanikandanR]

Dear Max,

Thank you for your suggestions. As you said, the process dies as shown below at dkrz. I think the problem lies in compiling PAMTRA using intel compiler in DKRZ as it is working fine with local machine. ........................................................................ 2020/11/18 08:45:44 info Start of calc_moment 2020/11/18 08:45:44 fatal in module calc_moment Something wrong with total number concentration! 2020/11/18 08:45:44 successError in calc_moment run_drop_size 2020/11/18 08:45:44 fatal in module hydrometeor_extinction Error in run_drop_size_dist 2020/11/18 08:45:44 fatal in module run_rt error in run_drop_size_dist! 1 1 2020/11/18 08:45:44 fatal in module run_pamtra Error in runrt! ........................................................................ Thanks, Veera

[DaveOri]

I am having a really hard time setting up the environment for the tests. I managed to get the intel compiler, but now it requires the dependencies to be also compiled with ifort and I am already struggling with netcdf. I do not know when I will be able to provide more assistance on this, perhaps someone that has access to Mistral can step in and investigate it further.

I agree with you that the compiler and library setup is most probably the cause of the error. PAMTRA is developed for gcc. Apart from the compiler-specific implementations of the fortran intrinsic other things that might affect your results are the use of intel-mkl in place of BLAS and FFTW3.

I see from the mistral documentation that there are modules available for gcc, fftw and openblas. Perhaps you might consider trying also that option. Beware of issue #24 if you use openblas.

Davide

[mariomech]

I'm gonna test it this afternoon with my installation at DKRZ that I'm using since quite some time for active and passive simulations without any major problems. I just have to wait till my recent simulations are done before changing the code to be able to use the importer as in the example.

[mariomech]

Dear Veera,

we found out what is going wrong on DKRZ with the intel compiler. It is due to the handling of floating point numbers beyond machine precision and a consistency test that was not very well designed.

If you change in _src/calcmoment.f90 line 76 to 81 from

if (sum(n_ds) - m_0 > 1.d-12) then
    msg = 'Something wrong with total number concentration!'
    errorstatus = error
    call report(errorstatus, msg, nameOfRoutine)
    return
  endif

to

if (abs((sum(n_ds) - m_0)/m_0) > 1.d-12) then
      print*, sum(n_ds), m_0, (sum(n_ds) - m_0)/m_0
      msg = 'Total number concentration m_0 and sum(n_ds) do not match!'
      errorstatus = warning
      call report(errorstatus, msg, nameOfRoutine)
endif

and recompile with

make;make pyinstall

your script should successfully run.

We will include this modfication permanently in our code.

Mario

Btw: this happened to me before on mistral and I only changed my code and never checked it back. Sorry for that.

[VeeramanikandanR]

Hi Mario,

Thank you so much for your help. It is working now.

Thanks and Regards, Veera