Compiling/running PyCLES using Docker

[The-Fonz]

I'm trying to run PyCLES to just play around with it, a.k.a. "my first LES", and have come so far as to be able to compile RRTMG and PyCLES, but when running it something goes wrong in the linking I believe. I am not very experienced with compiling/linking, and am currently a bit stuck, but it is probably something very small.

I'm using a Docker image for this purpose, I'll make a pull request with a working version if this is solved, might be nice for others as a starting point.

FROM ubuntu:19.10

# I am using a local pycles version for now as I'm editing setup.py
#RUN apt-get update && apt-get install -y git
#RUN git clone https://github.com/pressel/pycles.git

RUN apt-get update && apt-get install -y python python-pip libhdf5-dev hdf5-tools netcdf-bin libnetcdf-dev libpng-dev libfreetype6-dev csh
RUN apt-get update && apt-get install -y openmpi-bin openmpi-common

# Install instructions use mpi4py==1.3.1 but I had errors with missing mpi4py/libmpi.pyx
RUN pip install numpy==1.9.1 scipy==0.16.0 cython==0.23.1 mpi4py==3.0.3
RUN pip install matplotlib==1.4.3 netcdf4==1.1.9

# The pycles repo is located in this subfolder
COPY ./pycles/ pycles/

RUN cd pycles && python generate_parameters.py

# This will first compile RRTMG and then all PyCLES modules
# Will give some warnings about deprecated numpy api but that seems harmless
RUN cd pycles && CC=mpicc python setup.py build_ext --inplace

WORKDIR /pyc

I am using these parameters in setup.py, as the if-else sequence there seems to be for specific computers, as discussed in this issue.

if True:
    library_dirs = os.environ['PATH'].split(':')
    libraries = []
    libraries.append('mpi')
    libraries.append('gfortran')
    extensions = []
    extra_compile_args = []
    extra_compile_args += ['-std=c99', '-O3', '-march=native', '-Wno-unused',
                           '-Wno-#warnings', '-Wno-maybe-uninitialized', '-Wno-cpp', '-Wno-array-bounds', '-fPIC']
    extra_objects = ['./RRTMG/rrtmg_build/rrtmg_combined.o']
    netcdf_include = get_netcdf_include()
    netcdf_lib = os.path.join(get_netcdf_prefix(), 'lib')
    f_compiler = 'gfortran'
elif sys.platform == 'darwin':

I build this with docker build -t pycles . which takes a while due to RRTMG but finishes without errors, and run with docker run -it pycles. Then I do python generate_namelist.py StableBubble and python main.py StableBubble.in, but get this:

root@05d9e9b26401:/pycles# python main.py StableBubble.in 
Traceback (most recent call last):
  File "main.py", line 33, in <module>
    main()
  File "main.py", line 17, in main
    main3d(namelist)
  File "main.py", line 23, in main3d
    import Simulation3d
  File "Radiation.pxd", line 11, in init Simulation3d (Simulation3d.c:29814)
    cdef class RadiationBase:
ImportError: /pycles/Radiation.so: undefined symbol: _ZGVbN2v_exp

I had different errors before (with missing MPI functions) but apparently solved those, however now I'm a bit stuck. Would appreciate any help!

[lsterzinger]

Hi @The-Fonz, I'm having the same problem when trying to run it (not in docker though). I get ImportError: /pycles/Radiation.so: undefined symbol: _ZGVbN2v_exp Were you able to find any solution to this?

[The-Fonz]

@lsterzinger I haven't found a solution, still stuck in the same place! Was hoping one of the authors/maintainers would answer at some point!

[szy21]

@pressel any thought on this?

[huzizhan]

It might be related to the gfortran version when compiling pycles, I used the latest gfrotran compiler and found a similar problem, but not in Docker. Then I changed the gfortran version to 8.4 and the problem solved. @The-Fonz @lsterzinger

[adityakalaga]

Hi, @huzizhan I tried compiling with gfortran version 8.4, and python 3.9.12 on a Debian Linux system but still getting the error below: Traceback (most recent call last): File "/nlsasfs/home/fdmod/kaditya/pycles/main.py", line 33, in main() File "/nlsasfs/home/fdmod/kaditya/pycles/main.py", line 17, in main main3d(namelist) File "/nlsasfs/home/fdmod/kaditya/pycles/main.py", line 23, in main3d import Simulation3d File "Simulation3d.pyx", line 1, in init Simulation3d import time ImportError: /nlsasfs/home/fdmod/kaditya/pycles/Radiation.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZGVbN2v_exp

[huzizhan]

python version might also be the reason, you can try python 3.8.13(currently running in my arch linux) for testing @adityakalaga