search

MiniZinc / libminizinc

The MiniZinc compiler
http://www.minizinc.org
Other
497 stars 78 forks source link

Minizinc 2.8.5 + Gecode `release/6.3.0`: `tests/spec/unit/test-globals-float.mzn` causes OOM #835

Open LebedevRI opened 3 weeks ago

LebedevRI commented 3 weeks ago

After running into unsatisfactory state of debian minizinc packages one too many times, i've decided to build it locally.

Versions:

All built using clang-18 -O3 -DNDEBUG without LTO/PGO, same happens with -O2 -UNDEBUG.

Now that i've finally reached the stage where tests can be run, i'm running into an issue. I'm running pytest with -n 1, so i can pinpoint where the problem is.

The last output is:

spec/unit/regression/test_parout.mzn::default.1.cbc
[gw0] [ 64%] SKIPPED spec/unit/regression/test_parout.mzn::default.1.cbc
spec/unit/regression/var_self_assign_bug.mzn::default.0.gecode
[gw0] [ 64%] PASSED spec/unit/regression/var_self_assign_bug.mzn::default.0.gecode
spec/unit/regression/var_self_assign_bug.mzn::default.0.chuffed
[gw0] [ 64%] PASSED spec/unit/regression/var_self_assign_bug.mzn::default.0.chuffed
spec/unit/regression/var_self_assign_bug.mzn::default.0.cbc
[gw0] [ 64%] SKIPPED spec/unit/regression/var_self_assign_bug.mzn::default.0.cbc
<this is the last output. after a while, OOM happens>

While waiting for OOM killer to trigger (after fzn-gecode consumes 100+Gigs of RAM!), this is the relevant output of ps:

 145434 ?        S      0:00 /usr/bin/make -f debian/rules test_minizinc_late
 145451 ?        S      0:00 /bin/sh -c cd /build/minizinc-meta-27hHCQ/minizinc-meta-1.0.0/libminizinc/tests/; \ PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/build/minizinc-meta-27hHCQ/minizinc-meta-1.0.0-build/build/stage-2/minizinc-meta-install/usr/bin" PYTHONPATH=":/build/minizinc-meta-27hHCQ/minizinc-meta-1.0.0-build/build/stage-2/minizinc-meta-install/usr/lib/python3.12/dist-packages/" pytest --suite default --solvers gecode,chuffed -n 1
 145452 ?        Sl     0:00 /usr/bin/python3 /usr/bin/pytest --suite default --solvers gecode,chuffed -n 1
 145454 ?        Sl     0:02 /usr/bin/python3 -u -c import sys;exec(eval(sys.stdin.readline()))
 147367 pts/3    Ss     0:00 /bin/bash
 148088 ?        I      0:00 [kworker/26:0]
 148360 ?        S      0:00 /build/minizinc-meta-27hHCQ/minizinc-meta-1.0.0-build/build/stage-2/minizinc-meta-install/usr/bin/minizinc --solver org.gecode.gecode@6.3.0 --allow-multiple-assignments --output-mode json --output-time --output-objective --output-output-item --statistics --intermediate-solutions /build/minizinc-meta-27hHCQ/minizinc-meta-1.0.0/libminizinc/tests/spec/unit/test-globals-float.mzn --json-stream
 148362 ?        R      0:55 /build/minizinc-meta-27hHCQ/minizinc-meta-1.0.0-build/build/stage-2/minizinc-meta-install/usr/bin/fzn-gecode -s /tmp/mznfileyzWV5u.fzn
 148412 pts/3    R+     0:00 ps ax
 148413 pts/3    D+     0:00 /bin/bash

So it is indeed tests/spec/unit/test-globals-float.mzn. Thoughts?

This raises a bigger issue: looking at fzn-gecode, is there really no memory limit handling?

LebedevRI commented 3 weeks ago

Looks like that is the only really problematic test, but two other tests crash with assertion:

`lib/flatten.cpp:1895: MiniZinc::CallStackItem::CallStackItem(EnvI &, Expression *): Assertion `Expression::type(e).bt() != Type::BT_UNKNOWN' failed.` ``` [gw0] [ 94%] FAILED spec/unit/types/alias.mzn::default.0.gecode _ /<>/libminizinc/tests/spec/unit/types/alias.mzn::default.0.gecode _ [gw0] linux -- Python 3.12.5 /usr/bin/python3 /usr/lib/python3/dist-packages/_pytest/runner.py:341: in from_call result: TResult | None = func() /usr/lib/python3/dist-packages/_pytest/runner.py:242: in lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise /usr/lib/python3/dist-packages/pluggy/_hooks.py:513: in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) /usr/lib/python3/dist-packages/pluggy/_manager.py:120: in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) /usr/lib/python3/dist-packages/pluggy/_callers.py:139: in _multicall raise exception.with_traceback(exception.__traceback__) /usr/lib/python3/dist-packages/pluggy/_callers.py:122: in _multicall teardown.throw(exception) # type: ignore[union-attr] /usr/lib/python3/dist-packages/_pytest/threadexception.py:92: in pytest_runtest_call yield from thread_exception_runtest_hook() /usr/lib/python3/dist-packages/_pytest/threadexception.py:68: in thread_exception_runtest_hook yield /usr/lib/python3/dist-packages/pluggy/_callers.py:122: in _multicall teardown.throw(exception) # type: ignore[union-attr] /usr/lib/python3/dist-packages/_pytest/unraisableexception.py:95: in pytest_runtest_call yield from unraisable_exception_runtest_hook() /usr/lib/python3/dist-packages/_pytest/unraisableexception.py:70: in unraisable_exception_runtest_hook yield /usr/lib/python3/dist-packages/pluggy/_callers.py:122: in _multicall teardown.throw(exception) # type: ignore[union-attr] /usr/lib/python3/dist-packages/_pytest/logging.py:848: in pytest_runtest_call yield from self._runtest_for(item, "call") /usr/lib/python3/dist-packages/_pytest/logging.py:831: in _runtest_for yield /usr/lib/python3/dist-packages/pluggy/_callers.py:122: in _multicall teardown.throw(exception) # type: ignore[union-attr] /usr/lib/python3/dist-packages/_pytest/capture.py:879: in pytest_runtest_call return (yield) /usr/lib/python3/dist-packages/pluggy/_callers.py:122: in _multicall teardown.throw(exception) # type: ignore[union-attr] /usr/lib/python3/dist-packages/_pytest/skipping.py:257: in pytest_runtest_call return (yield) /usr/lib/python3/dist-packages/pluggy/_callers.py:103: in _multicall res = hook_impl.function(*args) /usr/lib/python3/dist-packages/_pytest/runner.py:174: in pytest_runtest_call item.runtest() conftest.py:256: in runtest assert False, message E AssertionError: expected one of E E !Result E solution: !Solution E tup: E - 2 E - true E tuptup: E - 2 E - true E - 2 E - true E x: 3 E y: E - null E - 1 E status: SATISFIED E E E but got E E !Error E message: "minizinc: /<>/libminizinc/lib/flatten.cpp:1895: E MiniZinc::CallStackItem::CallStackItem(EnvI &, Expression *): Assertion `Expression::type(e).bt() E != Type::BT_UNKNOWN' failed.\nFile fragment:\n1894: CallStackItem::CallStackItem(EnvI& E env0, Expression* e) : _env(env0), _csiType(CSI_NONE) {\n1895: assert(Expression::type(e).bt() E != Type::BT_UNKNOWN);\n1896: \n" E type: MiniZincError E E assert False ```

(How can a test, from the same version as the source code, fail.)

And this one looks like a gecode change:

`[ 80%] FAILED spec/unit/general/array_intersect_context.mzn::default.0.gecode ` ``` [gw0] [ 80%] FAILED spec/unit/general/array_intersect_context.mzn::default.0.gecode _ /<>/libminizinc/tests/spec/unit/general/array_intersect_context.mzn::default.0.gecode _ [gw0] linux -- Python 3.12.5 /usr/bin/python3 /usr/lib/python3/dist-packages/_pytest/runner.py:341: in from_call result: TResult | None = func() /usr/lib/python3/dist-packages/_pytest/runner.py:242: in lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise /usr/lib/python3/dist-packages/pluggy/_hooks.py:513: in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) /usr/lib/python3/dist-packages/pluggy/_manager.py:120: in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) /usr/lib/python3/dist-packages/pluggy/_callers.py:139: in _multicall raise exception.with_traceback(exception.__traceback__) /usr/lib/python3/dist-packages/pluggy/_callers.py:122: in _multicall teardown.throw(exception) # type: ignore[union-attr] /usr/lib/python3/dist-packages/_pytest/threadexception.py:92: in pytest_runtest_call yield from thread_exception_runtest_hook() /usr/lib/python3/dist-packages/_pytest/threadexception.py:68: in thread_exception_runtest_hook yield /usr/lib/python3/dist-packages/pluggy/_callers.py:122: in _multicall teardown.throw(exception) # type: ignore[union-attr] /usr/lib/python3/dist-packages/_pytest/unraisableexception.py:95: in pytest_runtest_call yield from unraisable_exception_runtest_hook() /usr/lib/python3/dist-packages/_pytest/unraisableexception.py:70: in unraisable_exception_runtest_hook yield /usr/lib/python3/dist-packages/pluggy/_callers.py:122: in _multicall teardown.throw(exception) # type: ignore[union-attr] /usr/lib/python3/dist-packages/_pytest/logging.py:848: in pytest_runtest_call yield from self._runtest_for(item, "call") /usr/lib/python3/dist-packages/_pytest/logging.py:831: in _runtest_for yield /usr/lib/python3/dist-packages/pluggy/_callers.py:122: in _multicall teardown.throw(exception) # type: ignore[union-attr] /usr/lib/python3/dist-packages/_pytest/capture.py:879: in pytest_runtest_call return (yield) /usr/lib/python3/dist-packages/pluggy/_callers.py:122: in _multicall teardown.throw(exception) # type: ignore[union-attr] /usr/lib/python3/dist-packages/_pytest/skipping.py:257: in pytest_runtest_call return (yield) /usr/lib/python3/dist-packages/pluggy/_callers.py:103: in _multicall res = hook_impl.function(*args) /usr/lib/python3/dist-packages/_pytest/runner.py:174: in pytest_runtest_call item.runtest() conftest.py:256: in runtest assert False, message E AssertionError: expected one of E E !Result E solution: !SolutionSet E - !Solution E b: true E x: E - !!set E 1: null E - !!set E 2: null E - !!set E 3: null E - !Solution E b: false E x: E - !!set E 1: null E 2: null E 3: null E - !!set E 1: null E 2: null E 3: null E - !!set E 1: null E 2: null E 3: null E status: ALL_SOLUTIONS E E E but got E E !Result E solution: E - !Solution E b: false E x: E - !Range 1..3 E - !Range 1..3 E - !Range 1..3 E - !Solution E b: true E x: E - !!set E 1: null E - !!set E 2: null E - !!set E 3: null E status: ALL_SOLUTIONS E E assert False ```