search

bbalasub1 / glmnet_python

GNU General Public License v3.0
200 stars 95 forks source link

ValueError: You need to supply a value for newx #23

Closed Visdoom closed 6 years ago

Visdoom commented 6 years ago

Hey there!

Sometimes I run into this error. 

Warning: Non-fatal error in glmnet library call: error code =  -82
Check results for accuracy. Partial or no results returned.
Warning: Non-fatal error in glmnet library call: error code =  -81
Check results for accuracy. Partial or no results returned.
Warning: Non-fatal error in glmnet library call: error code =  -81
Check results for accuracy. Partial or no results returned.
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-133-f0c494ec5f90> in <module>()
      9 foldid = scipy.random.choice(10, size = y.shape[0], replace = True)
     10 cvfit5 = cvglmnet(x = X_train.copy(), y = y.copy().astype(np.float64),ptype='deviance', 
---> 11                  family = 'binomial', foldid=foldid)

/usr/local/lib/python3.5/dist-packages/glmnet_python/cvglmnet.py in cvglmnet(x, y, family, ptype, nfolds, foldid, parallel, keep, grouped, **options)
    290         cvstuff = cvlognet(cpredmat, options['lambdau'], x, y \
    291                           , options['weights'], options['offset'] \
--> 292                           , foldid, ptype, grouped, keep)
    293     elif cpredmat[0]['class'] == 'multnet':
    294         cvstuff = cvmultnet(cpredmat, options['lambdau'], x, y \

/usr/local/lib/python3.5/dist-packages/glmnet_python/cvlognet.py in cvlognet(fit, lambdau, x, y, weights, offset, foldid, ptype, grouped, keep)
     64         else:
     65             off_sub = scipy.empty([0])
---> 66         preds = glmnetPredict(fitobj, x[which, ], scipy.empty([0]), 'response', False, off_sub)
     67         nlami = scipy.size(fit[i]['lambdau'])
     68         predmat[which, 0:nlami] = preds

/usr/local/lib/python3.5/dist-packages/glmnet_python/glmnetPredict.py in glmnetPredict(fit, newx, s, ptype, exact, offset)
    120 
    121     if newx.shape[0] == 0 and ptype != 'coefficients' and ptype != 'nonzero':
--> 122         raise ValueError('You need to supply a value for ''newx''')
    123 
    124     # python 1D arrays are not the same as matlab 1xn arrays

ValueError: You need to supply a value for newx
X_train = array([[ -5.40843620e+02,  -1.02225210e+03,  -9.55378815e+02,
         -2.36909150e+03,   1.40433027e+02,   4.49656681e+03,
         -2.29619051e+03,   1.62042649e+03,   2.69144447e+02,
          1.02409981e+03],
       [ -8.36097114e+02,  -3.81453719e+02,  -9.76317857e+02,
         -2.76014952e+03,  -1.39847652e+01,   1.61675552e+03,
         -1.12633253e+03,   1.46405497e+02,   3.74870789e+02,
         -8.73671544e+02],
       [ -3.29496293e+03,  -1.05882063e+03,   1.36507323e+03,
          2.01477622e+03,  -1.50741120e+03,   1.64081943e+03,
         -7.14132821e+01,  -3.91500280e+02,   7.04252159e+02,
         -1.69213489e+03],
       [  2.19903335e+03,   6.03864332e+03,  -3.57683306e+03,
          1.13912537e+03,   1.81812752e+03,   6.62883625e+02,
          8.16110863e+02,  -8.20431937e+00,  -5.50799204e+01,
          4.61625658e+01],
       [  1.46233126e+03,  -3.41969427e+03,  -2.40997944e+03,
         -3.77033987e+03,   3.48148300e+02,   1.36220896e+03,
         -1.00305255e+03,   7.66126812e+01,   7.67513544e+01,
         -4.50282974e+02],
       [  5.00228949e+03,  -1.89012074e+03,  -1.09295812e+03,
         -3.83595394e+03,  -3.10575697e+03,   9.00829930e+02,
          3.43770424e+03,  -1.98774551e+03,  -1.71300940e+03,
          1.03544293e+03],
       [ -4.69923932e+03,  -1.12382873e+03,   2.74094022e+03,
         -1.17222988e+03,   1.31776902e+03,  -8.94748058e+02,
          7.56585734e+02,  -8.78789297e+02,   5.92443242e+02,
          1.04592173e+03],
       [  7.94369448e+03,   8.80723934e+02,   5.95919534e+03,
          6.80743523e+02,   1.21636141e+03,  -3.60136695e+02,
         -1.70773168e+03,  -2.26612960e+02,   1.08977676e+02,
         -3.36302111e+02],
       [  4.34953770e+03,  -3.87207909e+03,  -4.61602535e+03,
         -2.79405389e+03,  -1.28918741e+03,  -3.42247555e+03,
         -1.71594000e+03,  -2.54542701e+02,   1.22761589e+03,
         -1.06290625e+03],
       [  6.28737181e+02,  -3.63911591e+03,  -1.61830619e+03,
          1.08822717e+03,   7.42246729e+02,  -5.58116984e+00,
          2.41222687e+02,   1.04780573e+02,  -5.96632952e+02,
         -3.57317633e+02],
       [  8.38702181e+03,   7.45201871e+02,   6.26846563e+03,
          6.12751249e+02,   1.25736317e+03,   8.24425640e+01,
         -2.40066976e+03,  -1.80153743e+03,  -9.05788327e+02,
          3.69977884e+02],
       [ -3.31687752e+03,   3.26373716e+03,   1.48250439e+03,
         -1.34061769e+03,  -1.99717419e+03,  -2.08123084e+03,
         -9.76937395e+02,   2.52949570e+02,   6.12224187e+02,
          2.54742082e+03],
       [ -3.00114441e+01,  -4.95156216e+03,  -2.15448705e+03,
         -5.14394440e+02,   3.39375791e+03,  -2.15308348e+03,
         -4.16008163e+02,   1.34334801e+03,  -5.41082476e+01,
          8.21498392e+02],
       [ -2.41085613e+02,   4.60771986e+03,  -1.51114701e+03,
          8.11852842e+02,  -1.29261361e+03,  -7.13556436e+00,
         -9.14088614e+02,   1.09465293e+03,  -7.80618126e+02,
         -4.79344251e+01],
       [  7.31768384e+03,   5.01534895e+02,   4.37954979e+03,
         -5.81161000e+02,  -9.62030594e+02,  -8.79412310e+01,
          3.70968224e+03,   3.50157475e+03,   1.61845849e+03,
         -4.35829855e+02],
       [  1.94243881e+03,   7.32604185e+03,  -3.92514023e+03,
          8.50881152e+02,   1.92852690e+03,   2.39259326e+01,
          7.64472194e+02,  -8.40252185e+01,  -2.65125554e+02,
          2.35512784e+02],
       [ -8.10164100e+02,   2.18996322e+03,  -1.19558758e+03,
          1.75215315e+01,   1.05429576e+03,   8.14930761e+02,
          3.94731571e+01,   2.58051371e+02,  -1.11249689e+02,
         -7.93591523e+02],
       [ -6.63682142e+02,  -3.76152941e+03,  -4.05981007e+02,
          1.61953431e+03,   1.47173872e+03,   3.40826704e+02,
          1.09923142e+03,  -1.18309452e+02,  -6.12414679e+02,
         -7.62163211e+02],
       [ -1.47544833e+03,  -2.85805201e+03,  -6.68908681e+01,
          4.13615600e+02,   1.03502293e+03,  -4.57390832e+02,
          3.01897601e+02,   7.95541713e+02,  -2.22159773e+03,
         -4.86088166e+02],
       [ -3.78308746e+03,   1.09558422e+03,   1.58352360e+03,
         -6.34232397e+02,  -9.84928833e+02,  -1.37589779e+03,
         -2.78411380e+02,   7.06474445e+00,  -5.55956917e+02,
         -3.07215986e+02],
       [ -3.99011334e+03,   8.01812627e+02,   1.60999031e+03,
         -1.45261523e+03,   6.71413408e+02,  -5.98102848e+02,
          3.19308838e+02,  -3.64840215e+02,  -1.58619190e+01,
         -6.64643730e+02],
       [ -2.66081373e+03,  -6.17617007e+02,   1.65952517e+03,
          1.05192725e+03,  -9.72639423e+02,   6.93250063e+01,
          5.21679842e+02,  -7.79449648e+02,   8.60091411e+02,
         -4.98618141e+02],
       [  3.02209164e+03,  -2.79499426e+03,  -2.66143756e+03,
          3.72849219e+03,  -2.12973619e+03,   1.77607134e+03,
          1.25690547e+02,  -1.28379402e+03,  -1.85887279e+02,
          1.01594674e+03],
       [  2.98672314e+03,  -4.17606942e+03,  -3.63874797e+03,
          4.09966888e+03,  -1.29678538e+03,  -1.75318123e+03,
         -3.92382072e+02,   1.73095212e+02,   8.04243813e+02,
          7.02176569e+02],
       [  5.73760322e+02,   5.72696872e+03,  -2.15588804e+03,
          3.62945504e+02,   5.72620121e+01,  -9.33565971e+02,
          1.66817040e+02,   3.05250103e+02,  -1.28948497e+03,
         -3.02285528e+02],
       [ -1.98252857e+03,  -7.14060143e+01,   2.48803912e+02,
          2.34941985e+03,  -1.94441658e+03,   9.57253803e+02,
         -6.18933437e+02,  -2.27377946e+02,   1.59754122e+03,
         -4.74274038e+02],
       [ -3.86718295e+03,  -2.27868876e+03,   1.74322617e+03,
          1.15949026e+03,   9.80542912e+02,   2.32127566e+02,
          5.36729449e+02,   6.33540398e+02,  -2.71765518e+02,
          2.44714206e+03],
       [ -3.26924168e+03,   2.04882321e+03,   1.65094363e+03,
         -7.06895324e+02,  -1.33390717e+03,  -1.68147273e+03,
          1.52656319e+02,  -5.46771092e+02,  -9.05308643e+02,
         -1.58889784e+03],
       [ -4.92294949e+03,  -1.22625367e+03,   2.74893721e+03,
         -4.64294510e+02,   7.88162923e+02,   1.04811023e+02,
          9.22172436e+02,  -8.35821024e+02,   6.24338078e+02,
         -5.11020561e+01],
       [  3.97841273e+00,   4.21539688e+03,  -1.96320880e+03,
         -1.00326595e+03,   1.93485649e+03,   2.98516901e+02,
          8.02575187e+02,  -1.99127059e+03,   2.51155083e+03,
          3.16330004e+02],
       [ -2.81913688e+03,  -3.00259378e+03,   7.61282049e+02,
          1.06994728e+03,   1.51577828e+03,   6.52899860e+02,
          8.15093108e+02,   2.68599495e+01,  -5.61778427e+02,
         -5.95493818e+02],
       [ -2.61585520e+03,   2.70397994e+03,   7.22354291e+02,
          3.28374945e+02,  -2.84123511e+03,  -2.21251727e+02,
         -1.61101153e+03,   1.44043770e+03,  -8.80835270e+02,
          1.73121426e+02]]) 
 y = array([1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0,
       0, 1, 1, 0, 1, 0, 1, 1, 1])

Funnily enough the issue only occurs sometimes.

Any ideas where that comes from?

bbalasub1 commented 6 years ago

Please provide data so we can reproduce this.

githubhnw commented 4 years ago

Hi,

I am running into the same issue. Sometimes I get this error. If I run again, all is good. And it's not a problem with a newx empty parameter.

Do you know what is the issue or how to overcome this 'sometimes' occuring error?

Thanks, Raluca

bbalasub1 commented 4 years ago

Not sure about this. Can you provide a reproducible example?

On Sun, Nov 8, 2020 at 4:11 PM githubhnw notifications@github.com wrote:

Hi,

I am running into the same issue. Sometimes I get this error. If I run again, all is good. And it's not a problem with a newx empty parameter.

Do you know what is the issue or how to overcome this 'sometimes' occuring error?

Thanks, Raluca

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/bbalasub1/glmnet_python/issues/23#issuecomment-723666484, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACVEGE47XQQ3HI5ECWHM4YDSO4CQRANCNFSM4EVQ3IMQ .

githubhnw commented 4 years ago

Unfortunately not. I'm getting this error from time to time. Now, I'm only worried about moving the script into production. What can I do to overcome the issue. Running the script until there is no error...

bbalasub1 commented 4 years ago

Can you try looping it until there is an error and dumping? Could be that the problem is sensitive to the random start value. You could try to fix the random starts and see what happens.

Hard to troubleshoot from my end without an example that shows the error.

On Mon, Nov 9, 2020, 4:40 AM githubhnw notifications@github.com wrote:

Unfortunately not. I'm getting this error from time to time. Now, I'm only worried about moving the script into production. What can I do to overcome the issue. Running the script until there is no error...

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/bbalasub1/glmnet_python/issues/23#issuecomment-723894408, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACVEGE5YHJKYB5GLFO4G27LSO62J5ANCNFSM4EVQ3IMQ .

githubhnw commented 4 years ago

Hello,

Below the error

image

I will set a seed before setting the foldid parameter.

Regards, Raluca

lovestat commented 2 years ago

For those who have the same problem,

this could be caused by the foldid ranged from [1, nfold].

Just set the foldid ranged in [0, nfold -1], and the error shall be solved.