search

alnurali / cvxstoc

Disciplined convex stochastic programming. For the cvxstoc home page, please see:
http://alnurali.github.io/cvxstoc/
31 stars 11 forks source link

Examples lead to Segmentation fault (missing dependency constrains?) #8

Open ibayer opened 3 years ago

ibayer commented 3 years ago

I tried to reproduce the documentation examples but run repeatedly into segmentation faults that I don't know how to debug.

I assume this is due to some version incompatibilities but randomly pinning old versions of scipy, cvxstoc, pymc and cvxpy did lead me to a working combination.

Knowing a working set of dependencies versions would probably help me a lot to track this issue down. Any pointer in this direction would be very much appreciated.

Here are the steps to reproduce the segmentation fault:

cat portfolio.py

from cvxstoc import NormalRandomVariable, expectation, prob
from cvxpy import Maximize, Problem
from cvxpy.expressions.variable import Variable
import numpy

# Create problem data.
n = 10
mu = numpy.zeros(n)
Sigma = 0.1*numpy.eye(n)
p = NormalRandomVariable(mu, Sigma)
alpha = -1
beta = 0.05

# Create and solve stochastic optimization problem.
x = Variable(n)
p = Problem(Maximize(expectation(x.T*p, num_samples=100)),
            [x >= 0, x.T*numpy.ones(n) == 1,
             prob(x.T*p <= alpha, num_samples=100) <= beta])
p.solve(verbose=True)

output:

python portfolio.py 
OF9Tn2eO-py3.7/lib/python3.7/site-packages/cvxpy/expressions/expression.py:556: UserWarning: 
This use of ``*`` has resulted in matrix multiplication.
Using ``*`` for matrix multiplication has been deprecated since CVXPY 1.1.
    Use ``*`` for matrix-scalar and vector-scalar multiplication.
    Use ``@`` for matrix-matrix and matrix-vector multiplication.
    Use ``multiply`` for elementwise multiplication.
This code path has been hit 1 times so far.

  warnings.warn(msg, UserWarning)
OF9Tn2eO-py3.7/lib/python3.7/site-packages/pymc/MCMC.py:81: UserWarning: Instantiating a Model object directly is deprecated. We recommend passing variables directly to the Model subclass.
  warnings.warn(message)
/home/ibayer/.cache/pypoetry/virtualenvs/new-yorker-OF9Tn2eO-py3.7/lib/python3.7/site-packages/cvxpy/expressions/expression.py:556: UserWarning: 
This use of ``*`` has resulted in matrix multiplication.
Using ``*`` for matrix multiplication has been deprecated since CVXPY 1.1.
    Use ``*`` for matrix-scalar and vector-scalar multiplication.
    Use ``@`` for matrix-matrix and matrix-vector multiplication.
    Use ``multiply`` for elementwise multiplication.
This code path has been hit 2 times so far.

  warnings.warn(msg, UserWarning)
OF9Tn2eO-py3.7/lib/python3.7/site-packages/cvxpy/expressions/expression.py:556: UserWarning: 
This use of ``*`` has resulted in matrix multiplication.
Using ``*`` for matrix multiplication has been deprecated since CVXPY 1.1.
    Use ``*`` for matrix-scalar and vector-scalar multiplication.
    Use ``@`` for matrix-matrix and matrix-vector multiplication.
    Use ``multiply`` for elementwise multiplication.
This code path has been hit 3 times so far.

  warnings.warn(msg, UserWarning)
OF9Tn2eO-py3.7/lib/python3.7/site-packages/pymc/MCMC.py:81: UserWarning: Instantiating a Model object directly is deprecated. We recommend passing variables directly to the Model subclass.
  warnings.warn(message)
===============================================================================
                                     CVXPY                                     
                                    v1.1.11                                    
===============================================================================
(CVXPY) Mar 23 04:08:37 PM: Your problem has 11.0 variables, 3 constraints, and 10 parameters.
(CVXPY) Mar 23 04:08:37 PM: It is compliant with the following grammars: DCP, DQCP
(CVXPY) Mar 23 04:08:37 PM: CVXPY will first compile your problem; then, it will invoke a numerical solver to obtain a solution.
-------------------------------------------------------------------------------
                                  Compilation                                  
-------------------------------------------------------------------------------
(CVXPY) Mar 23 04:08:37 PM: Compiling problem (target solver=ECOS).
(CVXPY) Mar 23 04:08:37 PM: Reduction chain: FlipObjective -> Dcp2Cone -> CvxAttr2Constr -> ConeMatrixStuffing -> ECOS
(CVXPY) Mar 23 04:08:37 PM: Applying reduction FlipObjective
(CVXPY) Mar 23 04:08:37 PM: Applying reduction Dcp2Cone
(CVXPY) Mar 23 04:08:37 PM: Applying reduction CvxAttr2Constr
(CVXPY) Mar 23 04:08:37 PM: Applying reduction ConeMatrixStuffing
Segmentation fault (core dumped)

I'm running this is a fresh env on ubuntu 18.04.

cat pyproject.toml 
[tool.poetry]
name = "test"
version = "0.1.0"
description = ""
authors = [""]

[tool.poetry.dependencies]
python = "3.7.3"
cvxstoc = "^0.2.2"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
Here is also the full log of my pinned dependencies.

```bash cat poetry.lock [[package]] name = "cvxpy" version = "1.1.11" description = "A domain-specific language for modeling convex optimization problems in Python." category = "main" optional = false python-versions = ">=3.5" [package.dependencies] ecos = ">=2" numpy = ">=1.15" osqp = ">=0.4.1" scipy = ">=1.1.0" scs = ">=1.1.6" [[package]] name = "cvxstoc" version = "0.2.2" description = "A domain-specific language for modeling stochastic convex optimization problems in Python." category = "main" optional = false python-versions = "*" [package.dependencies] cvxpy = ">=0.3.5" numpy = ">=1.10" pymc = ">=2.3.4" scipy = ">=0.16" [[package]] name = "ecos" version = "2.0.7.post1" description = "This is the Python package for ECOS: Embedded Cone Solver. See Github page for more information." category = "main" optional = false python-versions = "*" [package.dependencies] numpy = ">=1.6" scipy = ">=0.9" [[package]] name = "numpy" version = "1.20.1" description = "NumPy is the fundamental package for array computing with Python." category = "main" optional = false python-versions = ">=3.7" [[package]] name = "osqp" version = "0.6.2.post0" description = "OSQP: The Operator Splitting QP Solver" category = "main" optional = false python-versions = "*" [package.dependencies] numpy = ">=1.7" qdldl = "*" scipy = ">=0.13.2" [[package]] name = "pymc" version = "2.3.8" description = "Markov Chain Monte Carlo sampling toolkit." category = "main" optional = false python-versions = "*" [[package]] name = "qdldl" version = "0.1.5.post0" description = "QDLDL, a free LDL factorization routine." category = "main" optional = false python-versions = "*" [package.dependencies] numpy = ">=1.7" scipy = ">=0.13.2" [[package]] name = "scipy" version = "1.6.1" description = "SciPy: Scientific Library for Python" category = "main" optional = false python-versions = ">=3.7" [package.dependencies] numpy = ">=1.16.5" [[package]] name = "scs" version = "2.1.2" description = "scs: splitting conic solver" category = "main" optional = false python-versions = "*" [package.dependencies] numpy = ">=1.7" scipy = ">=0.13.2" [metadata] lock-version = "1.1" python-versions = "3.7.3" content-hash = "14862b53acbc62c83549662af16d3b74d35a5035055362b0420b64421ccb54e7" ```

ibayer commented 3 years ago

related issue: https://github.com/alnurali/cvxstoc/issues/7

ibayer commented 3 years ago

@SteveDiamond I saw your commit regarding made cvxpy 1.1 compatible and your impressive work with cvxpy in general. Maybe you can give some advice here?

aisurfer commented 3 years ago

@SteveDiamond I've got exactly the same output and segfault as @ibayer above, Could you please show your setup, that allows to run code with recent changes? CC @alnurali

SteveDiamond commented 3 years ago

@aisurfer cvxstoch may be broken. I haven't used it in a long time. We'll look into it.