ValueError: H contains Nan/Inf values

[MLopez-Ibanez]

Reproducible with the latest version in github. See code below for how to reproduce it.

/home/manu/.local/lib/python3.8/site-packages/pybobyqa/model.py:252: LinAlgWarning: Diagonal number 2 is exactly zero. Singular matrix.
  self.lu, self.piv = LA.lu_factor(A)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
~/.local/lib/python3.8/site-packages/trustregion/interface.py in _to_array(X, lbl)
     13     try:
---> 14         return np.asarray_chkfinite(X)
     15     except ValueError:

~/.local/lib/python3.8/site-packages/numpy/lib/function_base.py in asarray_chkfinite(a, dtype, order)
    487     if a.dtype.char in typecodes['AllFloat'] and not np.isfinite(a).all():
--> 488         raise ValueError(
    489             "array must not contain infs or NaNs")

ValueError: array must not contain infs or NaNs

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-1-9226e210f113> in <module>
----> 1 import codecs, os;__pyfile = codecs.open('''/tmp/pygzTqh2''', encoding='''utf-8''');__code = __pyfile.read().encode('''utf-8''');__pyfile.close();os.remove('''/tmp/pygzTqh2''');exec(compile(__code, '''/home/manu/work/gtoc11/poliastro/src/bobyqa_bug.py''', 'exec'));

~/work/gtoc11/poliastro/src/bobyqa_bug.py in <module>
    341 
    342 np.random.seed(42)
--> 343 result = pybobyqa.solve(fitness, x0 = fitness.x0, bounds = (fitness.lower, fitness.upper),
    344                                 maxfun=1000, seek_global_minimum=True,
    345                                 scaling_within_bounds=True)

~/.local/lib/python3.8/site-packages/pybobyqa/solver.py in solve(objfun, x0, args, bounds, npt, rhobeg, rhoend, maxfun, nsamples, user_params, objfun_has_noise, seek_global_minimum, scaling_within_bounds, do_logging, print_progress)
    809     nx = 0
    810     xmin, fmin, gradmin, hessmin, nsamples_min, nf, nx, nruns, exit_info, diagnostic_info = \
--> 811         solve_main(objfun, x0, args, xl, xu, npt, rhobeg, rhoend, maxfun, nruns, nf, nx, nsamples, params,
    812                     diagnostic_info, scaling_changes, do_logging=do_logging, print_progress=print_progress)
    813 

~/.local/lib/python3.8/site-packages/pybobyqa/solver.py in solve_main(objfun, x0, args, xl, xu, npt, rhobeg, rhoend, maxfun, nruns_so_far, nf_so_far, nx_so_far, nsamples, params, diagnostic_info, scaling_changes, f0_avg_old, f0_nsamples_old, do_logging, print_progress)
    638                 if params("restarts.use_restarts") and params("restarts.use_soft_restarts"):
    639                     number_of_samples = max(nsamples(control.delta, control.rho, current_iter, nruns_so_far), 1)
--> 640                     exit_info = control.soft_restart(number_of_samples, nruns_so_far, params,
    641                                                      x_in_abs_coords_to_save=None, f_to_save=None, nsamples_to_save=None)
    642                     if exit_info is not None:

~/.local/lib/python3.8/site-packages/pybobyqa/controller.py in soft_restart(self, number_of_samples, nruns_so_far, params, x_in_abs_coords_to_save, f_to_save, nsamples_to_save)
    537             # Using adelt=delta in fix_geometry (adelt determines the ball to max lagrange poly in altmov)
    538             # [Only reason actual 'delta' is needed in fix_geometry is for calling nsamples()]
--> 539             exit_info = self.geometry_step(knew, self.delta, number_of_samples, params)
    540             if exit_info is not None:
    541                 return exit_info

~/.local/lib/python3.8/site-packages/pybobyqa/controller.py in geometry_step(self, knew, adelt, number_of_samples, params)
    289             c, g, H = self.model.lagrange_polynomial(knew)  # based at xopt
    290             # Solve problem: bounds are sl <= xnew <= su, and ||xnew-xopt|| <= adelt
--> 291             xnew = trsbox_geometry(self.model.xopt(), c, g, H, self.model.sl, self.model.su, adelt)
    292         except LA.LinAlgError:
    293             exit_info = ExitInformation(EXIT_LINALG_ERROR, "Singular matrix encountered in geometry step")

~/.local/lib/python3.8/site-packages/pybobyqa/trust_region.py in trsbox_geometry(xbase, c, g, H, lower, upper, Delta, use_fortran)
    395     #   s.t.   lower <= xbase + s <= upper
    396     #          ||s|| <= Delta
--> 397     smin, gmin, crvmin = trsbox(xbase, g, H, lower, upper, Delta, use_fortran=use_fortran)  # minimise L(x)
    398     smax, gmax, crvmax = trsbox(xbase, -g, -H, lower, upper, Delta, use_fortran=use_fortran)  # maximise L(x)
    399     if abs(c + model_value(g, H, smin)) >= abs(c + model_value(g, H, smax)):  # take largest abs value

~/.local/lib/python3.8/site-packages/pybobyqa/trust_region.py in trsbox(xopt, g, H, sl, su, delta, use_fortran)
     64 def trsbox(xopt, g, H, sl, su, delta, use_fortran=USE_FORTRAN):
     65     if use_fortran:
---> 66         return trustregion.solve(g, H, delta,
     67                                  sl=np.minimum(sl - xopt, -ZERO_THRESH),
     68                                  su=np.maximum(su - xopt, ZERO_THRESH),

~/.local/lib/python3.8/site-packages/trustregion/interface.py in solve(g, H, delta, sl, su, verbose_output)
     54     # Convert to desired types
     55     g = _to_array(g, 'g')
---> 56     H = None if H is None else _to_array(H, 'H')
     57     try:
     58         delta = float(delta)

~/.local/lib/python3.8/site-packages/trustregion/interface.py in _to_array(X, lbl)
     14         return np.asarray_chkfinite(X)
     15     except ValueError:
---> 16         raise ValueError('%s contains Nan/Inf values' % lbl)
     17 
     18 

ValueError: H contains Nan/Inf values

import pybobyqa
import numpy as np

SEC_PER_DAY = 86400 # s

class VisitFitness:
    bounds = [(0., 730.),
              (3600./SEC_PER_DAY, 730.)]
    lower = np.array(bounds)[:,0]
    upper = np.array(bounds)[:,1]
    x0 = np.array([0., 30.])

    f_values = np.array([10.11572753649087985650,
 30.66843985569197883478,
 12.10014994129142351653,
 55.85847358731629697104,
 5383.72631231060131540289,
 10.16117834002221265166,
 12.36951055457960535477,
 9.57702202652344780631,
 10.44058095842921929375,
 9.55662933393137770111,
 9.54123152986866607250,
 9.54117776421004037957,
 9.61797191385653427176,
 9.54085321652559237293,
 9.62243765585130717000,
 9.54087512984960817164,
 9.55012014888882809771,
 9.54629142950483711161,
 9.54102407400895735634,
 9.54085319225582217939,
 9.54178767704779851044,
 9.54146041017130563944,
 9.54085319026665068520,
 9.54085319261752751174,
 9.54094457154279140809,
 9.54090227541482782669,
 9.54085565247380529286,
 9.54086452803617568463,
 9.54085622465918703483,
 9.54085984350153992750,
 9.54085323729991507946,
 9.54085319026614442350,
 9.54085319027010214654,
 9.54085319026819789201,
 9.54085368026597180346,
 11155.96339733171589614358,
 24.36364878547908219275,
 44.51039287958609236284,
 30.18900737512171872368,
 29.07508529780710659907,
 22.71350117409208735353,
 26.08977603131518563373,
 21.36945044514498093235,
 20.60464023994765270231,
 19.33053672389414856525,
 20.66601676909562712581,
 18.96139849632065832452,
 18.71603435330364462175,
 19.03425084057550975558,
 18.58851016322796567692,
 18.49460631757657225194,
 18.37161530664230468801,
 18.26029149604252310723,
 17.81998266212578485579,
 16.38373183343565742121,
 11.70441722102565584862,
 11.18539420028015207720,
 11.13716917742398848645,
 11.21414528574188551602,
 11.15859356600115326330,
 11.20768478052635686026,
 11.13362352598097437806,
 11.14933518384334121265,
 11.11845470714375139210,
 11.10173994023707777501,
 11.05358609093479316243,
 11.01432291271084551454,
 10.85936119573387514947,
 10.28190595123253103793,
 11.49664035760720537382,
 9.78109636812332894351,
 9.76591380443313639148,
 10.11144260346175904886,
 9.74385914376990491803,
 10.19446569155174842081,
 9.99550972291460837482,
 9.68014779302152206242,
 9.70163103007076443873,
 9.62021989456571269272,
 9.57084784013245126744,
 9.59032232177467136580,
 9.59151823136871328757,
 9.57418555455031139445,
 9.58203089552088016490,
 9.56766843572178515842,
 9.55659203299698845058,
 9.54199109706627623950,
 11.13120197643271680477,
 9.69920606079684688439,
 9.54088560591848278136,
 9.54093550355950270614,
 9.54855139789792062288,
 9.54085923772327149095,
 9.54085319099968387491,
 9.55009846759046610032,
 9.54156436862724888215,
 9.54085339890114347838,
 9.54158967345539466010,
 9.54085331258791491393,
 9.54085319026588152269,
 9.54095618946305812358,
 9.54088744369884622643,
 9.54085319187876024216,
 9.54085319040163781779,
 9.54086462284087488683,
 9.54086127504193548532,
 9.54085319028275868902,
 9.54085319026783018614,
 9.54085319026354738980,
 9.54085319026781242258,
 9.54085433208056699073,
 9.54085319026875211534,
 9.54085368537992906113,
 9.54085319026616573979,
 12.91947006185212032392,
 18.27718500351339514509,
 21.99131003303629228185,
 20.55747142698114160453,
 15.41920095812745294950,
 12.47876185640576807145,
 14.56257465282083174429,
 12.04614324377087086759,
 12.26974007425033441621,
 11.83351883613651267524,
 11.01644679943664684174,
 10.78963299558271060619,
 12.27759358224415997540,
 9.63568859982660441688,
 9.89182808484722109199,
 10.39629879322508543282,
 9.54212087803402653208,
 9.55248917578839495945,
 9.71168788291488382924,
 9.54197943351636013176,
 9.61573795221762495089,
 9.54085590633266811267,
 9.54933594178939770813,
 9.54086338875514705649,
 9.54735859275178633254,
 9.54085319058652103763,
 9.54171097185607486324,
 9.54085319026920508634,
 9.54096823376606018030,
 9.54085319245253948850,
 9.54092794167742397349,
 9.54085319026203748649,
 9.54086368306339593914,
 9.54085847790977936711,
 9.54085319028604317282,
 9.54085319026441247559,
 9.54085319026786748964,
 9.54085395414431225447,
 9.54085393884466981262,
 15.99818655102091113918,
 16.63796548385391460556,
 25.03290237554396568953,
 26.11099316511672796537,
 12.30081127595430956490,
 13.31706926057203688174,
 13.40771238056107605985,
 11.87208637125209875762,
 12.22053992871303762513,
 11.56294365785527133994,
 11.44917957469806779613,
 11.00633904305782451161,
 9.63725660294579000720,
 5383.72631231060131540289,
 11.24347719515458976502,
 10.34246429232709552082,
 10.30502583088494361618,
 9.55320039759946837421,
 9.58204638468711067389,
 9.84990696403078302978,
 9.54235215569945083303,
 9.54419860861445279454,
 9.57675127907513257242,
 9.54090090549254377095,
 9.54085405721296275772,
 9.55209172565533037869,
 9.54087769176860511777,
 9.54085319038920331991,
 9.54184182177231754451,
 9.54085343530124774247,
 9.54140802389881592660,
 9.54101767076124218647,
 9.54085319701965062222,
 9.54096698537731491285,
 9.54087518540971046832,
 9.54092359855541616298,
 9.54085319026758860161,
 9.54085319029051426298,
 9.54086436318687347580,
 9.54085916689485280529,
 9.54085401382196529596,
 9.54085319026650324759,
 9.54085319026653699837,
 9.54085328305010627048,
 9.54085319026443556822,
 9.54085413704370211008,
 5383.72631231060131540289,
 536252.92010295542422682047,
 7876.84376734635407046881,
 5169.72460685076202935306,
 15.38039624480350653357,
 19.31591368033434719109,
 14.90903903208799619051,
 12.62088529715983753476,
 11.42893302044527281680,
 11.09385677290211091872,
 11.90649775009120858726,
 11.77288645193038441050,
 10.98416277038281485545,
 10.72580409547348168076,
 10.44183511982485512704,
 9.55656814376146357404,
 9.55173220928139699026,
 9.70613412443892364934,
 9.54087523328895414920,
 9.54086906929717670778,
 9.56151411080746349569,
 9.54085319068917669938,
 9.55130239871468411650,
 9.54207999633450221211,
 9.54160029689607469550,
 9.54085342786442502927,
 9.54085333495666532144,
 9.54085319026419931276,
 9.54097721905542073273,
 9.54093155378507162823,
 9.54085319263662867684,
 9.54085485457341953008,
 9.54086515551877312191,
 9.54086121586855639976,
 9.54085319028884804027,
 9.54085319028048672862,
 9.54085432820503065443,
 9.54085392012331290346,
 9.54085319026286882149,
 9.54085319026943956544,
 12988.59128619515649916138,
 130.95851733875338140933,
 39.01491957376705954630,
 63.90901040001111255151,
 32.10726523098355755792,
 30.15011295236418575882,
 33.28171296700336512231,
 29.84798168828263342789,
 31.11831352677140571927,
 29.25426493195438837347,
 29.04704750706764926349,
 29.35778370484440813470,
 28.92887298892073388856,
 28.80053260929373948329,
 28.29412143754063535539,
 27.80939603188407716061,
 27.35353821251900896527,
 26.86781462897169703297,
 25.03220641025961157311,
 18.42255680452024790839,
 14.31599143660539930067,
 14.20266007336414304518,
 14.25251905193321455556,
 14.27632286357395408061,
 14.18225281605712595479,
 14.23214869949086747170,
 14.20351865093046583866,
 14.17451412633179330669,
 14.16866830554972267464,
 14.17231391902840975661,
 14.16047737919050319988,
 14.13296932603068611911,
 14.05699444029977840387,
 13.95501660420300638066,
 13.61537961741653468550,
 12.62985452286975629477,
 9.56629334266571795808,
 5383.72631231060131540289,
 20.75092278438621917758,
 10.61369854456642691787,
 12.54431185770854817463,
 9.80957740812961809240,
 9.90030372862304730575,
 9.80333106918157959342,
 9.55031921450572696131,
 9.65561877693923342747,
 9.62556644776771186400,
 9.55842894097626327721,
 9.61726916430501965749,
 9.58135262170341839294,
 9.54480601552343266292,
 9.54170408817461535023,
 9.54086963229641682460,
 9.56308618555887690604,
 9.54105070854513215295,
 9.54802359504166098247,
 9.54085344135743085303,
 9.54085319067923798286,
 9.54211149690140025825,
 9.54168644217759620574,
 9.54085333662522749876,
 9.54085319026384581775,
 9.54097769913793669616,
 9.54093087442718790214,
 9.54085319276976306924,
 9.54085319173421630978,
 9.54086512499509353802,
 9.54086087605103472242,
 9.54085319028929035312,
 9.54085319028204992264,
 9.54085319026321876379,
 9.54085411112136227985,
 9.54085383055536873087,
 9.54085319026324363278,
 9.54085319026514078189,
 13.86462183107111023617,
 13.86462183107111023617])

    def __init__(self):
        self.step = 0

    def __call__(self, x):
        f = self.f_values[self.step]
        print(f'{f}:{x}:{self.step}')
        self.step += 1
        return f

fitness = VisitFitness()

np.random.seed(42)
result = pybobyqa.solve(fitness, x0 = fitness.x0, bounds = (fitness.lower, fitness.upper),
                                maxfun=1000, seek_global_minimum=True,
                                scaling_within_bounds=True)

[lindonroberts]

Thanks for posting - this is the same underlying problem as in #14, which was fixed in https://github.com/numericalalgorithmsgroup/pybobyqa/commit/3b65e18508a37eed3366d61097d38ddbbd0ee3e9. I haven't made a new release - would it be helpful to release this soon?

[MLopez-Ibanez]

I just reinstalled from git and it still fails. Does it work for you?

numba==0.54.1 numpy==1.20.3 pandas==1.3.4 scipy==1.7.1 joblib==1.1.0 scikit-learn==1.0 trustregion==1.2

[lindonroberts]

I tried with those package versions (and the latest Py-BOBYQA from git), with Python 3.8 on Ubuntu, but still can't replicate this error.

However, looking more carefully at your traceback, you were getting the same error as #14 but from a different place in the code. I have hopefully addressed that in https://github.com/numericalalgorithmsgroup/pybobyqa/commit/7a87380bdd27ed1e17c41449644dc21476a58ef1 (Py-BOBYQA should now finish solving and return an error message, rather than raising an error)

[MLopez-Ibanez]

Thanks! The error is different now:

9.540854111121362:[8.76501050e-06 4.36792357e+01]:310
9.540853830555369:[6.09449153e-06 4.36792200e+01]:311
9.540853190263244:[ 0.        43.6792183]:312
9.54085319026514:[ 0.         43.67922428]:313
13.86462183107111:[  0.         132.00418746]:314
13.86462183107111:[  0.         132.00418746]:315
/home/manu/.local/lib/python3.8/site-packages/pybobyqa/model.py:252: LinAlgWarning: Diagonal number 2 is exactly zero. Singular matrix.
  self.lu, self.piv = LA.lu_factor(A)

I'm using Python 3.8.10 with ubuntu 20.04.

Not sure if there is a way to dump more debugging output out of Py-BOBYQA.

[MLopez-Ibanez]

I also get:

/home/manu/.local/lib/python3.8/site-packages/pybobyqa/solver.py:797: RuntimeWarning: x0 below lower bound, adjusting warnings.warn("x0 below lower bound, adjusting", RuntimeWarning)

But this is not true.

[lindonroberts]

For the "x0 below lower bound" warning, this is generated when x0 <= lower in any entry (not just strictly). In your case you have x0=lower for one entry, I think, which explains this warning. I admit, it probably shouldn't show a warning in that situation (or at least be clearer). I have opened #24 for this.

For the other issue, it seems that the code might not be detecting a singular interpolation matrix properly (usually this is done by catching a LinAlgError). There are two things to try to get more information from the solver:

• After running, do print(result) to see the termination message
• Add import logging; logging.basicConfig(level=logging.DEBUG) before you call pybobyqa.solve(...)

[MLopez-Ibanez]

See attached log.

bobyqa-log.txt