Solver 'SCS' failed when there is solver SCS in the cvxpy package.

[yonghyeokrhee]

machine: windows 10 python evn ; anaconda 5.4/ python 3.6 pip install fancyimpute conda install cvxpy

It keeps saying that SolverError: Solver 'SCS' failed. Try another solver or solve with verbose=True for more information. Try recentering the problem data around 0 and rescaling to reduce the dynamic range.

INSTALLED_SOLVERS
['ECOS', 'ECOS_BB', 'SCS', 'OSQP']

mat_restored = NuclearNormMinimization().fit_transform(mat_destroyed)
data = np.array([[3, 4, 1, 1, 5, 2, 3, 4, np.nan, 5],
                     [1, 5, 4, 2, 1, 3, 1, np.nan, 2, 1],
                     [1, 2, 4, 3, 2, np.nan, 2, 4, 4, 5],
                     [5, 4, 5, 5, 2, 2, 4, np.nan, 1, 3],
                     [1, np.nan, 2, 4, 2, np.nan, 1, 3, 5, 2],
                     [3,4,np.nan,1,np.nan,1,5,4,2,3],
                     [1,3,5,np.nan,3,np.nan,2,4,1,5],
                     [1,2,np.nan,4,np.nan,3,2,4,4,1],
                     [2,5,1,2,4,np.nan,2,1,np.nan,np.nan],
                     [4,1,3,2,5,3,np.nan,2,3,1]])

---------------------------------------------------------------------------
SolverError                               Traceback (most recent call last)
<ipython-input-25-123fc995ba03> in <module>
----> 1 complete_image(a)

<ipython-input-19-074d2c1581c1> in complete_image(mat_destroyed)
      1 def complete_image(mat_destroyed):
----> 2     mat_restored = NuclearNormMinimization().fit_transform(mat_destroyed)
      3     return mat_restored

~\AppData\Roaming\Python\Python36\site-packages\fancyimpute\solver.py in fit_transform(self, X, y)
    187                     type(X_filled)))
    188 
--> 189         X_result = self.solve(X_filled, missing_mask)
    190         if not isinstance(X_result, np.ndarray):
    191             raise TypeError(

~\AppData\Roaming\Python\Python36\site-packages\fancyimpute\nuclear_norm_minimization.py in solve(self, X, missing_mask)
    127             max_iters=self.max_iters,
    128             # use_indirect, see: https://github.com/cvxgrp/cvxpy/issues/547
--> 129             use_indirect=False)
    130         return S.value

~\AppData\Roaming\Python\Python36\site-packages\cvxpy\problems\problem.py in solve(self, *args, **kwargs)
    245         else:
    246             solve_func = Problem._solve
--> 247         return solve_func(self, *args, **kwargs)
    248 
    249     @classmethod

~\AppData\Roaming\Python\Python36\site-packages\cvxpy\problems\problem.py in _solve(self, solver, ignore_dcp, warm_start, verbose, parallel, **kwargs)
    360         solution = self._solving_chain.solve_via_data(self, data, warm_start, verbose,
    361                                                       kwargs)
--> 362         self.unpack_results(solution, self._solving_chain, inverse_data)
    363         return self.value
    364 

~\AppData\Roaming\Python\Python36\site-packages\cvxpy\problems\problem.py in unpack_results(self, solution, chain, inverse_data)
    470                 "Try another solver or solve with verbose=True for more information. " +
    471                 "Try recentering the problem data around 0 and rescaling " +
--> 472                 "to reduce the dynamic range."
    473             )
    474         self._status = solution.status

SolverError: Solver 'SCS' failed. Try another solver or solve with verbose=True for more information. Try recentering the problem data around 0 and rescaling to reduce the dynamic range.

[sergeyf]

Do all the fancyimpute tests pass locally on your machine?

[mabdelhack]

Same issue happened for me... It occurs with the test code as well of the nuclear_norm_minimization.

[sergeyf]

Can you paste the error trace?

[mabdelhack]

System: Windows 10 Python: 3.6 Inside a conda environment

~\Anaconda3\envs\pytorch\python.exe ~/Documents/GitHub/fancyimpute/test/test_nuclear_norm_minimization.py Using TensorFlow backend. Traceback (most recent call last):

SCS v2.1.2 - Splitting Conic Solver

File ~/Documents/GitHub/fancyimpute/test/test_nuclear_norm_minimization.py", line 57, in (c) Brendan O'Donoghue, Stanford University, 2012 test_rank1_convex_solver()

Lin-sys: sparse-direct, nnz in A = 228 File ~/Documents/GitHub/fancyimpute/test/test_nuclear_norm_minimization.py", line 34, in test_rank1_convex_solver eps = 1.00e-04, alpha = 1.50, max_iters = 50000, normalize = 1, scale = 1.00 XY_completed_rank1 = solver.fit_transform(XY_missing_rank1) acceleration_lookback = 0, rho_x = 1.00e-03 Variables n = 105, constraints m = 155 File "~\Documents\GitHub\fancyimpute\fancyimpute\solver.py", line 186, in fit_transform Cones: primal zero / dual free vars: 25 linear vars: 75 X_result = self.solve(X_filled, missing_mask) sd vars: 55, sd blks: 1 File "~\Documents\GitHub\fancyimpute\fancyimpute\nuclear_norm_minimization.py", line 127, in solve FATAL: Cannot solve SDPs with > 2x2 matrices without linked blas+lapack libraries use_indirect=False) Install blas+lapack and re-compile SCS with blas+lapack library locations ERROR: init_cone failure File "~\Anaconda3\envs\pytorch\lib\site-packages\cvxpy\problems\problem.py", line 290, in solve Setup time: 2.09e-04s return solve_func(self, *args, **kwargs) Failure:could not initialize work File "~\Anaconda3\envs\pytorch\lib\site-packages\cvxpy\problems\problem.py", line 575, in _solve self.unpack_results(solution, full_chain, inverse_data) File "~\Anaconda3\envs\pytorch\lib\site-packages\cvxpy\problems\problem.py", line 718, in unpack_results "Try another solver, or solve with verbose=True for more " cvxpy.error.SolverError: Solver 'SCS' failed. Try another solver, or solve with verbose=True for more information.

[sergeyf]

Weird. I guess something went wrong with CVXPY. We're not really supporting this package anymore unfortunately. You feel like mucking about with other solvers to see if the tests pass?

[mabdelhack]

I'll see what I can do. PS: sorry for the messed up traceback, I did not realize it is doing markdown formatting.

[mabdelhack]

I finally managed to make it run on windows. The way I managed to fix it was mainly by installing cvxpy with its dependencies from the unofficial Windows binaries except for SCS that I used the pip install method. It is still not performing as well as the one I installed on Linux. Apparently cvxpy overflows some shape variables when data is big while on Linux it just converts to a bigger variable size (or maybe it starts with a long in from the beginning. Hopefully the updated Windows binaries could handle this better. Currently, I am only able to solve very small data sets.

[sergeyf]

Well that sucks, sorry. Doesn't really seem like there's much we can do on our end except: (a) remove this algorithm entirely or (b) see if other solvers work better (not SCS).

[mabdelhack]

That's fine, when I actually tested this method on Linux on my simulated data, it takes too long while not outperforming other algorithms that I even removed it from my codes. Thanks a lot though for the discussion.

[sergeyf]

Sure, thanks for your investigation! I'll reopen this ticket so others can see in case they have issues.

[sergeyf]

We've officially replaced the SCS solver to CVXOPT in #129 . Please try to pip install the new version and see if it works.