undefined symbol: thetas_c

[Balinus]

Hi!

trying to run a simulation and I get the following error. Any hint would be appreciated!

python2 main.py SaturatedBubble.in 
Traceback (most recent call last):
  File "main.py", line 33, in <module>
    main()
  File "main.py", line 17, in main
    main3d(namelist)
  File "main.py", line 23, in main3d
    import Simulation3d
  File "Simulation3d.pyx", line 10, in init Simulation3d
    from AuxiliaryStatistics import AuxiliaryStatistics
ImportError: /home/proy/Programmes/pycles/AuxiliaryStatistics.so: undefined symbol: thetas_c

Tried other CASE_NAME and get the same error. Thanks!

[cwbctr1]

I have the same problem. Have you solved it?

[Balinus]

Nope!

[pressel]

My apologies for our slow reply to this issue, I'm happy to help resolve it. It would help me to know a little about the system on which you are attempting to run PyCLES as well as exactly the commands you are using to compile it.

[Balinus]

Humm, it's been a while. As far as I can remember, I used the provided command from the official doc : https://github.com/pressel/pycles/blob/master/docs/source/install.rst#building-on-linux

It's on Pop!_OS 17.10 (Ubuntu-based).

I'll ty to reinstall in the coming weeks and provide more details.

[PiyushX23]

Hi... I have the same issue with "thetas_c". It shows "undefined symbol:thetas_c." I followed the instruction given in "install.rst" file to install PyCLES. Can anyone help to resolve it?

[cmkaul]

I've just committed a change that should fix this problem. Please try again and let me know if the problem still occurs.

[Balinus]

Still get the error.

Cheers!

edit - system is based on Ubuntu 18.04 (Pop!_OS). Using OpenMPI. Compiled with standard commands provided in the installation.rst description. Python 3.7 (Anaconda)(to that effect, I needed to replace cPickle with pickle in Forcing.pyx).

python generate_namelist.py SaturatedBubble
False
{'conditional_stats': {},
 'damping': {'scheme': 'None'},
 'diffusion': {'qt_entropy_source': False},
 'fields_io': {'diagnostic_fields': ['ql',
                                     'temperature',
                                     'buoyancy_frequency',
                                     'viscosity'],
               'fields_dir': 'fields',
               'frequency': 100.0},
 'grid': {'dims': 3,
          'dx': 200.0,
          'dy': 200.0,
          'dz': 200.0,
          'gw': 3,
          'nx': 100,
          'ny': 5,
          'nz': 50},
 'meta': {'casename': 'SaturatedBubble',
          'simname': 'SaturatedBubble',
          'uuid': '039a4e8a-f5d2-4323-80e0-f37dea7cc68e'},
 'microphysics': {'phase_partitioning': 'liquid_only', 'scheme': 'None_SA'},
 'momentum_transport': {'order': 5},
 'mpi': {'nprocx': 1, 'nprocy': 1, 'nprocz': 1},
 'output': {'output_root': './'},
 'restart': {'frequency': 600.0,
             'init_from': False,
             'input_path': './',
             'output': True},
 'scalar_transport': {'order': 5},
 'sgs': {'UniformViscosity': {'diffusivity': 0.0, 'viscosity': 0.0},
         'scheme': 'UniformViscosity'},
 'stats_io': {'auxiliary': ['None'], 'frequency': 60.0, 'stats_dir': 'stats'},
 'thermodynamics': {'latentheat': 'constant'},
 'time_stepping': {'cfl_limit': 0.3,
                   'dt_initial': 10.0,
                   'dt_max': 10.0,
                   't_max': 1000.0,
                   'ts_type': 3}}
(base) proy@pop-os:~/GitRepos/pycles$ python main.py SaturatedBubble.in
Traceback (most recent call last):
  File "main.py", line 33, in <module>
    main()
  File "main.py", line 17, in main
    main3d(namelist)
  File "main.py", line 23, in main3d
    import Simulation3d
  File "Simulation3d.pyx", line 10, in init Simulation3d
    from AuxiliaryStatistics import AuxiliaryStatistics
ImportError: /home/proy/GitRepos/pycles/AuxiliaryStatistics.cpython-37m-x86_64-linux-gnu.so: undefined symbol: thetas_c

git log
commit df39365454195100f5e39281f16f7af933164351 (HEAD -> master, origin/master, origin/HEAD)
Author: Colleen Kaul <cmkaul@users.noreply.github.com>
Date:   Fri Mar 15 16:03:41 2019 -0700

    Update AuxiliaryStatistics.pyx

    fix to problem with importing thetas_c in AuxiliaryStatistics

[zhihong-tan]

Hi Philippe,

I did a clean install on my own system and ran the saturated bubble case, but did not reproduce the error (the run was successfuly completed). Could you please do the command 'head -n 80 AuxiliaryStatistics.c‘ and paste the output here? This would be helpful for determining if all libraries and include files are correctly linked.

Thanks, Zhihong

On Sat, Mar 16, 2019 at 12:23 PM Philippe Roy notifications@github.com wrote:

Still get the error.

python generate_namelist.py SaturatedBubble False {'conditional_stats': {}, 'damping': {'scheme': 'None'}, 'diffusion': {'qt_entropy_source': False}, 'fields_io': {'diagnostic_fields': ['ql', 'temperature', 'buoyancy_frequency', 'viscosity'], 'fields_dir': 'fields', 'frequency': 100.0}, 'grid': {'dims': 3, 'dx': 200.0, 'dy': 200.0, 'dz': 200.0, 'gw': 3, 'nx': 100, 'ny': 5, 'nz': 50}, 'meta': {'casename': 'SaturatedBubble', 'simname': 'SaturatedBubble', 'uuid': '039a4e8a-f5d2-4323-80e0-f37dea7cc68e'}, 'microphysics': {'phase_partitioning': 'liquid_only', 'scheme': 'None_SA'}, 'momentum_transport': {'order': 5}, 'mpi': {'nprocx': 1, 'nprocy': 1, 'nprocz': 1}, 'output': {'output_root': './'}, 'restart': {'frequency': 600.0, 'init_from': False, 'input_path': './', 'output': True}, 'scalar_transport': {'order': 5}, 'sgs': {'UniformViscosity': {'diffusivity': 0.0, 'viscosity': 0.0}, 'scheme': 'UniformViscosity'}, 'stats_io': {'auxiliary': ['None'], 'frequency': 60.0, 'stats_dir': 'stats'}, 'thermodynamics': {'latentheat': 'constant'}, 'time_stepping': {'cfl_limit': 0.3, 'dt_initial': 10.0, 'dt_max': 10.0, 't_max': 1000.0, 'ts_type': 3}} (base) proy@pop-os:~/GitRepos/pycles$ python main.py SaturatedBubble.in Traceback (most recent call last): File "main.py", line 33, in main() File "main.py", line 17, in main main3d(namelist) File "main.py", line 23, in main3d import Simulation3d File "Simulation3d.pyx", line 10, in init Simulation3d from AuxiliaryStatistics import AuxiliaryStatistics ImportError: /home/proy/GitRepos/pycles/AuxiliaryStatistics.cpython-37m-x86_64-linux-gnu.so: undefined symbol: thetas_c

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pressel/pycles/issues/34#issuecomment-473567239, or mute the thread https://github.com/notifications/unsubscribe-auth/ALYMnyZuPJL6lzBT9lm25-nEeNCoBpNEks5vXSh0gaJpZM4RlIxs .

[Balinus]

Here it is, Cheers!

(base) proy@pop-os:~/GitRepos/pycles$ head -n 80 AuxiliaryStatistics.c
/* Generated by Cython 0.29.2 */

/* BEGIN: Cython Metadata
{
    "distutils": {
        "depends": [
            "/home/proy/anaconda3/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h",
            "/home/proy/anaconda3/lib/python3.7/site-packages/numpy/core/include/numpy/ufuncobject.h",
            "Csrc/grid.h",
            "Csrc/prognostic_variables.h",
            "Csrc/thermodynamic_functions.h"
        ],
        "extra_compile_args": [
            "-std=c99",
            "-O3",
            "-march=native",
            "-Wno-unused",
            "-Wno-#warnings",
            "-Wno-maybe-uninitialized",
            "-Wno-cpp",
            "-Wno-array-bounds",
            "-fPIC"
        ],
        "include_dirs": [
            "/home/proy/anaconda3/lib/python3.7/site-packages/mpi4py/include",
            "/home/proy/anaconda3/lib/python3.7/site-packages/numpy/core/include",
            "./Csrc"
        ],
        "libraries": [
            "mpi",
            "gfortran"
        ],
        "library_dirs": [
            "",
            "/home/proy/.openmpi/lib/",
            "/home/proy/.openmpi/lib/",
            "/home/proy/.openmpi/lib/"
        ],
        "name": "AuxiliaryStatistics",
        "runtime_library_dirs": [
            "",
            "/home/proy/.openmpi/lib/",
            "/home/proy/.openmpi/lib/",
            "/home/proy/.openmpi/lib/"
        ],
        "sources": [
            "AuxiliaryStatistics.pyx"
        ]
    },
    "module_name": "AuxiliaryStatistics"
}
END: Cython Metadata */

#define PY_SSIZE_T_CLEAN
#include "Python.h"
#ifndef Py_PYTHON_H
    #error Python headers needed to compile C extensions, please install development version of Python.
#elif PY_VERSION_HEX < 0x02060000 || (0x03000000 <= PY_VERSION_HEX && PY_VERSION_HEX < 0x03030000)
    #error Cython requires Python 2.6+ or Python 3.3+.
#else
#define CYTHON_ABI "0_29_2"
#define CYTHON_HEX_VERSION 0x001D02F0
#define CYTHON_FUTURE_DIVISION 0
#include <stddef.h>
#ifndef offsetof
  #define offsetof(type, member) ( (size_t) & ((type*)0) -> member )
#endif
#if !defined(WIN32) && !defined(MS_WINDOWS)
  #ifndef __stdcall
    #define __stdcall
  #endif
  #ifndef __cdecl
    #define __cdecl
  #endif
  #ifndef __fastcall
    #define __fastcall
  #endif
#endif
#ifndef DL_IMPORT
  #define DL_IMPORT(t) t

[zhihong-tan]

Thanks for the information! Now I am finally able to reproduce your error message of 'undefined symbol: thetas_c'. Somehow this error only occurs in the Anaconda environment, not in the native environment on my university cluster.

I am going to look further into this issue and will let you know if I find a solution.

Thanks, Zhihong

On Sun, Mar 17, 2019 at 12:29 PM Philippe Roy notifications@github.com wrote:

Here it is, Cheers!

(base) proy@pop-os:~/GitRepos/pycles$ head -n 80 AuxiliaryStatistics.c / Generated by Cython 0.29.2 /

/ BEGIN: Cython Metadata { "distutils": { "depends": [ "/home/proy/anaconda3/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h", "/home/proy/anaconda3/lib/python3.7/site-packages/numpy/core/include/numpy/ufuncobject.h", "Csrc/grid.h", "Csrc/prognostic_variables.h", "Csrc/thermodynamic_functions.h" ], "extra_compile_args": [ "-std=c99", "-O3", "-march=native", "-Wno-unused", "-Wno-#warnings", "-Wno-maybe-uninitialized", "-Wno-cpp", "-Wno-array-bounds", "-fPIC" ], "include_dirs": [ "/home/proy/anaconda3/lib/python3.7/site-packages/mpi4py/include", "/home/proy/anaconda3/lib/python3.7/site-packages/numpy/core/include", "./Csrc" ], "libraries": [ "mpi", "gfortran" ], "library_dirs": [ "", "/home/proy/.openmpi/lib/", "/home/proy/.openmpi/lib/", "/home/proy/.openmpi/lib/" ], "name": "AuxiliaryStatistics", "runtime_library_dirs": [ "", "/home/proy/.openmpi/lib/", "/home/proy/.openmpi/lib/", "/home/proy/.openmpi/lib/" ], "sources": [ "AuxiliaryStatistics.pyx" ] }, "module_name": "AuxiliaryStatistics" } END: Cython Metadata /

define PY_SSIZE_T_CLEAN#include "Python.h"#ifndef Py_PYTHON_H

#error Python headers needed to compile C extensions, please install development version of Python.#elif PY_VERSION_HEX < 0x02060000 || (0x03000000 <= PY_VERSION_HEX && PY_VERSION_HEX < 0x03030000)
#error Cython requires Python 2.6+ or Python 3.3+.#else#define CYTHON_ABI "0_29_2"#define CYTHON_HEX_VERSION 0x001D02F0#define CYTHON_FUTURE_DIVISION 0#include <stddef.h>#ifndef offsetof

define offsetof(type, member) ( (size_t) & ((type*)0) -> member )#endif#if !defined(WIN32) && !defined(MS_WINDOWS)

ifndef __stdcall

#define __stdcall

endif

ifndef __cdecl

#define __cdecl

endif

ifndef __fastcall

#define __fastcall

endif#endif#ifndef DL_IMPORT

define DL_IMPORT(t) t

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pressel/pycles/issues/34#issuecomment-473687030, or mute the thread https://github.com/notifications/unsubscribe-auth/ALYMn_gEHkTJGjCwOrAfqdWwwJckKZYnks5vXnuAgaJpZM4RlIxs .

[zhihong-tan]

Hi Philippe,

I have experimented with various versions of GCC while keeping all other packages (python, cython, openmpi, etc.) unchanged in the native environment of my university cluster.

It seems the reported error does occur with GCC versions above 7.0 (I have tried 7.2.0 and 8.2.0 and both produce this error), so it is not an issue with Anaconda. However, this error does not occur with earlier versions of GCC (I have tried 4.8.5, 6.1.0, 6.2.0, and 6.3.0).

I have not been able to identify and fix the issue, but it may be worth a try to run the model with lower versions of GCC, if possible.

Thanks, Zhihong

On Sun, Mar 17, 2019 at 4:35 PM Zhihong Tan tzhihong@gmail.com wrote:

Thanks for the information! Now I am finally able to reproduce your error message of 'undefined symbol: thetas_c'. Somehow this error only occurs in the Anaconda environment, not in the native environment on my university cluster.

I am going to look further into this issue and will let you know if I find a solution.

Thanks, Zhihong

On Sun, Mar 17, 2019 at 12:29 PM Philippe Roy notifications@github.com wrote:

Here it is, Cheers!

(base) proy@pop-os:~/GitRepos/pycles$ head -n 80 AuxiliaryStatistics.c / Generated by Cython 0.29.2 /

/ BEGIN: Cython Metadata { "distutils": { "depends": [ "/home/proy/anaconda3/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h", "/home/proy/anaconda3/lib/python3.7/site-packages/numpy/core/include/numpy/ufuncobject.h", "Csrc/grid.h", "Csrc/prognostic_variables.h", "Csrc/thermodynamic_functions.h" ], "extra_compile_args": [ "-std=c99", "-O3", "-march=native", "-Wno-unused", "-Wno-#warnings", "-Wno-maybe-uninitialized", "-Wno-cpp", "-Wno-array-bounds", "-fPIC" ], "include_dirs": [ "/home/proy/anaconda3/lib/python3.7/site-packages/mpi4py/include", "/home/proy/anaconda3/lib/python3.7/site-packages/numpy/core/include", "./Csrc" ], "libraries": [ "mpi", "gfortran" ], "library_dirs": [ "", "/home/proy/.openmpi/lib/", "/home/proy/.openmpi/lib/", "/home/proy/.openmpi/lib/" ], "name": "AuxiliaryStatistics", "runtime_library_dirs": [ "", "/home/proy/.openmpi/lib/", "/home/proy/.openmpi/lib/", "/home/proy/.openmpi/lib/" ], "sources": [ "AuxiliaryStatistics.pyx" ] }, "module_name": "AuxiliaryStatistics" } END: Cython Metadata /

define PY_SSIZE_T_CLEAN#include "Python.h"#ifndef Py_PYTHON_H

#error Python headers needed to compile C extensions, please install development version of Python.#elif PY_VERSION_HEX < 0x02060000 || (0x03000000 <= PY_VERSION_HEX && PY_VERSION_HEX < 0x03030000)
#error Cython requires Python 2.6+ or Python 3.3+.#else#define CYTHON_ABI "0_29_2"#define CYTHON_HEX_VERSION 0x001D02F0#define CYTHON_FUTURE_DIVISION 0#include <stddef.h>#ifndef offsetof

define offsetof(type, member) ( (size_t) & ((type*)0) -> member )#endif#if !defined(WIN32) && !defined(MS_WINDOWS)

ifndef __stdcall

#define __stdcall

endif

ifndef __cdecl

#define __cdecl

endif

ifndef __fastcall

#define __fastcall

endif#endif#ifndef DL_IMPORT

define DL_IMPORT(t) t

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pressel/pycles/issues/34#issuecomment-473687030, or mute the thread https://github.com/notifications/unsubscribe-auth/ALYMn_gEHkTJGjCwOrAfqdWwwJckKZYnks5vXnuAgaJpZM4RlIxs .

[Balinus]

ok, thanks! Will try to use another gcc version. Cheers!

[zhihong-tan]

An easier fix is to disable the reference to the function thetas_c in AuxiliaryStatistics.pyx, i.e., replacing line #252: data[count] = thetas_c(PV.values[s_shift + ijk], PV.values[qt_shift + ijk]) With: data[count] = 0.0 This should allow the model to run normally with GCC 7, except without the output fields of cloud-average and cloud core-average thetas.

Please give this a try and let me know if it works. Thanks!

Zhihong

On Mon, Mar 18, 2019 at 9:12 AM Philippe Roy notifications@github.com wrote:

ok, thanks! Will try to use another gcc version. Cheers!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pressel/pycles/issues/34#issuecomment-473926023, or mute the thread https://github.com/notifications/unsubscribe-auth/ALYMn1ZE7jxkVNw-Od85-YHsfcOTT-arks5vX57AgaJpZM4RlIxs .