neurophysik / jitcdde

Just-in-time compilation for delay differential equations
Other
56 stars 14 forks source link

Cryptic error and traceback when compiling model with control_pars #39

Closed mwappner closed 2 years ago

mwappner commented 2 years ago

I'm getting a long and cryptic message when trying to compile a module to later save it to a file to make multiple short integrations in parallel. The error in question reads


CompileError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

SystemExit: error: command 'gcc' failed with exit status 1

/home/user/anaconda3/envs/dde/lib/python3.8/site-packages/IPython/core/interactiveshell.py:3426: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D.
  warn("To exit: use 'exit', 'quit', or Ctrl-D.", stacklevel=1)

with a really long traceback, which I can try to provide if needed. The system in question is 12-D composed of two 6-D coupled systems. Here's the code, which I fear I need to provide in full, since a more minimal example does work (see below). Assume get_maximum_delay is a function defined somewhere that retrieves the maximum value τ and τ*fc will have. You can set maxdelay=3 for testing purposes.

def make_model(parameters_lists):
    symbol_names = 'τ, βH, βC, βD, βN, η, α, σ, λm, λp, κC, κD, κE, fc'
    param_symbols = τ, βH, βC, βD, βN, η, α, σ, λm, λp, κC, κD, κE, fc = symengine.symbols(symbol_names)
    def model():
        h1, c1, d1, e1, n1, s1 = [y(i) for i in range(6)]
        h2, c2, d2, e2, n2, s2 = [y(i+6) for i in range(6)]
        h1d = y(0, t-τ)
        s1d = y(5, t-τ)
        h2d = y(6, t-τ)
        s2d = y(11, t-τ)
        h1c = y(0, t-τ*fc)
        h2c = y(6, t-τ*fc)    

        yield -h1 + βH * (1 + α * (σ * s1d)**η ) / ( (1 + (σ * s1d)**η) * (1 + h1d**η) )
        yield -c1 + βC / (1 + h1c**η) + λm * e1 - λp * c1 * d1 - κC * c1 * f2
        yield -d1 + βD + λm * e1 - λp * c1 * d1 - κD * d1 * f2
        yield -e1 - λm * e1 + λp * c1 * d1 - κE * e1 * f2
        yield -f1 + βN - f1 * (κC * c2 + κD * d2 + κE * e2)
        yield -s1 + f1 * (κC * c2 + κD * d2 + κE * e2)

        yield -h2 + βH * (1 + α * (σ * s2d)**η ) / ( (1 + (σ * s2d)**η) * (1 + h2d**η) )
        yield -c2 + βC / (1 + h2c**η) + λm * e2 - λp * c2 * d2 - κC * c2 * f1
        yield -d2 + βD + λm * e2 - λp * c2 * d2 - κD * d2 * f1
        yield -e2 - λm * e2 + λp * c2 * d2 - κE * e2 * f1
        yield -f2 + βN - f2 * (κC * c1 + κD * d1 + κE * e1)
        yield -s2 + f2 * (κC * c1 + κD * d1 + κE * e1)

    # initialize integrator
    maxdelay = get_maximum_delay(parameters_lists)
    DDE = jitcdde(model, verbose=True, max_delay=maxdelay, n=12, control_pars=param_symbols)
    model_location = DDE.save_compiled('model.so', overwrite=True)
    return model_location

After running this, which raises the error, the program is supposed to launch a bunch of processes to load the model and integrate it in parallel for a bunch of different parameter values.

Just as a sanity check, here's a very minimal example of a 2D system that does work when doing this. Ignore of course the fact that we are not multiprocessing here, since the error arises before calling the integration anyway.


import symengine
import numpy as np
from jitcdde import jitcdde, y, t

d = symengine.symbols('d')

def f():
    u = y(0)
    v = y(1)
    ud = y(0, t-d)
    vd = y(1, t-d)

    yield 0.5 * u * (1 - vd)
    yield -0.5 * v * (1 - ud)

DDE = jitcdde(f, control_pars=[d], max_delay=1, n=2)
file = DDE.save_compiled('testDDE', overwrite=True)

# this would be in a separate function in the actual case
for this_d in [0.1, 0.2, 0.3, 0.4]:
    I = jitcdde(module_location='testDDE.so', n=2, max_delay=1)
    I.constant_past([1,2])
    I.set_parameters(this_d)
    I.step_on_discontinuities()

    tt = I.t + np.arange(0, 30, 0.5)
    data = np.vstack( [I.integrate(this_t) for this_t in tt] )
Wrzlprmft commented 2 years ago

I tried to reproduce your problem by running the following code:

import symengine
import numpy as np
from jitcdde import jitcdde, y, t

symbol_names = 'τ, βH, βC, βD, βN, η, α, σ, λm, λp, κC, κD, κE, fc'
param_symbols = τ, βH, βC, βD, βN, η, α, σ, λm, λp, κC, κD, κE, fc = symengine.symbols(symbol_names)

f1 = 1
f2 = 1

def model():
    h1, c1, d1, e1, n1, s1 = [y(i) for i in range(6)]
    h2, c2, d2, e2, n2, s2 = [y(i+6) for i in range(6)]
    h1d = y(0, t-τ)
    s1d = y(5, t-τ)
    h2d = y(6, t-τ)
    s2d = y(11, t-τ)
    h1c = y(0, t-τ*fc)
    h2c = y(6, t-τ*fc)    

    yield -h1 + βH * (1 + α * (σ * s1d)**η ) / ( (1 + (σ * s1d)**η) * (1 + h1d**η) )
    yield -c1 + βC / (1 + h1c**η) + λm * e1 - λp * c1 * d1 - κC * c1 * f2
    yield -d1 + βD + λm * e1 - λp * c1 * d1 - κD * d1 * f2
    yield -e1 - λm * e1 + λp * c1 * d1 - κE * e1 * f2
    yield -f1 + βN - f1 * (κC * c2 + κD * d2 + κE * e2)
    yield -s1 + f1 * (κC * c2 + κD * d2 + κE * e2)

    yield -h2 + βH * (1 + α * (σ * s2d)**η ) / ( (1 + (σ * s2d)**η) * (1 + h2d**η) )
    yield -c2 + βC / (1 + h2c**η) + λm * e2 - λp * c2 * d2 - κC * c2 * f1
    yield -d2 + βD + λm * e2 - λp * c2 * d2 - κD * d2 * f1
    yield -e2 - λm * e2 + λp * c2 * d2 - κE * e2 * f1
    yield -f2 + βN - f2 * (κC * c1 + κD * d1 + κE * e1)
    yield -s2 + f2 * (κC * c1 + κD * d1 + κE * e1)

# initialize integrator
DDE = jitcdde(model, verbose=True, max_delay=3, n=12, control_pars=param_symbols)
model_location = DDE.save_compiled('model.so', overwrite=True)

… which works fine for me.

Note that I had to pick some values for f1 and f2 here, so my wild guess would be that those have a problematic type, structure, value, or similar in your case. How you wrap things in functions and similar should not affect the kind of error you are observing.

mwappner commented 2 years ago

Your code fails for me with the same error. Maybe there's some compiler arguments I can tweak to avoid this?

PS: in the code above I apparently forgot to replace n1, n2 with f1 f2 in the variable definition lines:

    h1, c1, d1, e1, n1, s1 = [y(i) for i in range(6)]
    h2, c2, d2, e2, n2, s2 = [y(i+6) for i in range(6)]

becomes

    h1, c1, d1, e1, f1, s1 = [y(i) for i in range(6)]
    h2, c2, d2, e2, f2, s2 = [y(i+6) for i in range(6)]

i was doing this while cleaning the code a bit to post it here, it seems I forgot to test that last change.

mwappner commented 2 years ago

Also note that running this with actual values for the parameters, instead of the symbols as control parameters does work. Issue here is that each parallel process needs to compile its own model and, since the integrations are quite short, about 80% of the time is spent compiling (which is still a short time, mind you).

Here's a set of example parameter values

from jitcdde import jitcdde, y, t

def model():
    τ, βH, βC, βD, βN, η, α, σ, λm, λp, κC, κD, κE, fc = [3, 1, 3, 3, 5, 0, 0.061, 0.0075, 0, 2.3, 5, 0.1, 1.0, 0]

    h1, c1, d1, e1, f1, s1 = [y(i) for i in range(6)]
    h2, c2, d2, e2, f2, s2 = [y(i+6) for i in range(6)]
    h1d = y(0, t-τ)
    s1d = y(5, t-τ)
    h2d = y(6, t-τ)
    s2d = y(11, t-τ)
    h1c = y(0, t-τ*fc)
    h2c = y(6, t-τ*fc)    

    yield -h1 + βH * (1 + α * (σ * s1d)**η ) / ( (1 + (σ * s1d)**η) * (1 + h1d**η) )
    yield -c1 + βC / (1 + h1c**η) + λm * e1 - λp * c1 * d1 - κC * c1 * f2
    yield -d1 + βD + λm * e1 - λp * c1 * d1 - κD * d1 * f2
    yield -e1 - λm * e1 + λp * c1 * d1 - κE * e1 * f2
    yield -f1 + βN - f1 * (κC * c2 + κD * d2 + κE * e2)
    yield -s1 + f1 * (κC * c2 + κD * d2 + κE * e2)

    yield -h2 + βH * (1 + α * (σ * s2d)**η ) / ( (1 + (σ * s2d)**η) * (1 + h2d**η) )
    yield -c2 + βC / (1 + h2c**η) + λm * e2 - λp * c2 * d2 - κC * c2 * f1
    yield -d2 + βD + λm * e2 - λp * c2 * d2 - κD * d2 * f1
    yield -e2 - λm * e2 + λp * c2 * d2 - κE * e2 * f1
    yield -f2 + βN - f2 * (κC * c1 + κD * d1 + κE * e1)
    yield -s2 + f2 * (κC * c1 + κD * d1 + κE * e1)

# initialize integrator
DDE = jitcdde(model)
DDE.compile_C()
Wrzlprmft commented 2 years ago

Your code fails for me with the same error. Maybe there's some compiler arguments I can tweak to avoid this?

Likely, but in that case I need the full error message and system and version information. Otherwise, I can only wildly guess. Also please check that you have the latest version of everything, if you haven’t already done so.

My best wild guess for a quick fix would be trying to change the compiler, but that really shouldn’t be necessary.

mwappner commented 2 years ago

Changing compiler to Clang worked! (I just installed clang in my system and added os.environ['CC']='clang' at the beginning of the file.) Thanks for the dedication, btw, I noticed I didn't say it earlier.

I'll leave the information on my error when using gcc below, in case you want to look into it, but feel free to close the issue if not.

jitcdde version: 1.8.0 OS: Ubuntu 20.04.3 Python: 3.8.5 gcc version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0

The following is the full exception. There's also a bunch of... things that I assume is the traceback of the compiler or something like that. I got it when running DDE.compile_C(verbose=True), you can find it here: https://pastebin.com/raw/pDdBi9VX

An exception has occurred, use %tb to see the full traceback.

Traceback (most recent call last):

  File "/home/user/anaconda3/envs/dde/lib/python3.8/distutils/unixccompiler.py", line 117, in _compile
    self.spawn(compiler_so + cc_args + [src, '-o', obj] +

  File "/home/user/anaconda3/envs/dde/lib/python3.8/distutils/ccompiler.py", line 910, in spawn
    spawn(cmd, dry_run=self.dry_run)

  File "/home/user/anaconda3/envs/dde/lib/python3.8/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)

  File "/home/user/anaconda3/envs/dde/lib/python3.8/distutils/spawn.py", line 157, in _spawn_posix
    raise DistutilsExecError(

DistutilsExecError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/home/user/anaconda3/envs/dde/lib/python3.8/distutils/core.py", line 148, in setup
    dist.run_commands()

  File "/home/user/anaconda3/envs/dde/lib/python3.8/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)

  File "/home/user/anaconda3/envs/dde/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()

  File "/home/user/anaconda3/envs/dde/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)

  File "/home/user/anaconda3/envs/dde/lib/python3.8/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()

  File "/home/user/anaconda3/envs/dde/lib/python3.8/site-packages/jitcxde_common/_jitcxde.py", line 304, in build_extensions
    build_ext.build_extensions(self)

  File "/home/user/anaconda3/envs/dde/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()

  File "/home/user/anaconda3/envs/dde/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)

  File "/home/user/anaconda3/envs/dde/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 196, in build_extension
    _build_ext.build_extension(self, ext)

  File "/home/user/anaconda3/envs/dde/lib/python3.8/distutils/command/build_ext.py", line 528, in build_extension
    objects = self.compiler.compile(sources,

  File "/home/user/anaconda3/envs/dde/lib/python3.8/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)

  File "/home/user/anaconda3/envs/dde/lib/python3.8/distutils/unixccompiler.py", line 120, in _compile
    raise CompileError(msg)

CompileError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

SystemExit: error: command 'gcc' failed with exit status 1
Wrzlprmft commented 2 years ago

The crucial lines of the error messages are the ones like these here:

/tmp/tmp0y2sbg63/jitced_1.c:50:19: error: stray ‘\317’ in program
   50 |  double parameter_τ;
      |                   ^

It appears that your version of the GCC cannot handle non-ASCII variable names. To fix this, either update your GCC as mine (Ubuntu 10.3.0-1ubuntu1) doesn’t exhibit this problem, or change the symbol_names as follows:

symbol_names = 'tau, beta_H, beta_C, beta_D, beta_N, eta, alpha, sigma, lambda_m, lambda_p, kappa_C, kappa_D, kappa_E, fc'

You only need to change that one. You can keep the non-ASCII variable names in your Python code.

mwappner commented 2 years ago

Oh, awesome, thanks for the tip!