embotech / ecos

A lightweight conic solver for second-order cone programming.
GNU General Public License v3.0
474 stars 123 forks source link

ECOS solver returns unsual results when used to train a soft-margin SVM #176

Closed prabathabey closed 5 years ago

prabathabey commented 5 years ago

CVXPY version: v1.0

Just thought of seeking some feedback on a bit of an unusual observation that I encountered while training a soft-margin SVM using the ECOS solver via Cvxpy framework. The implementation of the aforementioned soft-margin SVM goes below. The dataset I've used is a real-world dataset slightly modified to be able to train a binary SVM on it.

n = 50
C = Parameter(sign="positive")
C.value = 0.75
beta = cp.Variable(n, name="x")  # x in our terminology
v = cp.Variable(name="v")  # b in our terminology
Epsilon = cp.Variable(len(X_train), 1, name="Epsilon")  # punishing factor

objective = Minimize(sum_entries(square(beta)) + C * norm(Epsilon, 1))
g = C * cp.norm(Epsilon, 1)
for i in range(n):
  g = g + 0.5 * cp.square(beta[i])

constraints = [Epsilon >= 0]
for j in range(len(X_train)):
   constraints = constraints + [y_train[j] * (cp.sum_entries(cp.mul_elemwise(np.asmatrix(X_train[j]).T, beta)) + v) >= 1 - Epsilon[j]]
prob = Problem(Minimize(50*g), constraints)
try:
    prob.solve(solver=ECOS)
except SolverError as e:
    print(e)

I understand that different solvers use different ways of solving a given problem thereby returning different results. However, the results returned by the aforementioned implementation ranges around 98% for the given dataset, which is unusually high compared to the results returned by other implementations such as libsvm, etc, which were about 50%. Therefore, wondering if I might be missing something obvious.

Appreciate any input on this.

prabathabey commented 5 years ago

Turns out that the ECOS solver maybe solving a different problem than what I am looking at. Therefore, closing this ticket.