search

esa / pygmo2

A Python platform to perform parallel computations of optimisation tasks (global and local) via the asynchronous generalized island model.
https://esa.github.io/pygmo2/
Mozilla Public License 2.0
422 stars 57 forks source link

[BUG] Some tests fail: Optimization stopped because xtol_rel or xtol_abs was reached), etc. #86

Closed yurivict closed 2 years ago

yurivict commented 2 years ago

Describe the bug

===>   py38-pygmo2-2.18.0 depends on file: /usr/local/bin/python3.8 - found
runTest (pygmo.test.core_test_case) ... ok
runTest (pygmo._bfe_test.bfe_test_case) ... ok
runTest (pygmo._bfe_test.thread_bfe_test_case) ... ok
runTest (pygmo._bfe_test.member_bfe_test_case) ... ok
runTest (pygmo._bfe_test.mp_bfe_test_case) ... ok
runTest (pygmo._bfe_test.ipyparallel_bfe_test_case) ... ok
runTest (pygmo._bfe_test.default_bfe_test_case) ... ok
runTest (pygmo.test.archipelago_test_case) ... ERROR
runTest (pygmo._island_test.island_test_case) ... ok
runTest (pygmo._s_policy_test.s_policy_test_case) ... ok
runTest (pygmo._r_policy_test.r_policy_test_case) ... ok
runTest (pygmo._topology_test.topology_test_case) ... ok
runTest (pygmo.test.fair_replace_test_case) ... ok
runTest (pygmo.test.select_best_test_case) ... ok
runTest (pygmo.test.unconnected_test_case) ... ok
runTest (pygmo.test.ring_test_case) ... ok
runTest (pygmo.test.free_form_test_case) ... ok
runTest (pygmo.test.fully_connected_test_case) ... ok
runTest (pygmo.test.thread_island_test_case) ... ok
runTest (pygmo.test.thread_island_torture_test_case) ... ok
runTest (pygmo._problem_test.problem_test_case) ... ok
runTest (pygmo._algorithm_test.algorithm_test_case) ... /usr/local/lib/python3.8/site-packages/scipy/optimize/_minimize.py:524: RuntimeWarning: Method L-BFGS-B does not use Hessian information (hess).
  warn('Method %s does not use Hessian information (hess).' % method,
/usr/local/lib/python3.8/site-packages/scipy/optimize/_trustregion_constr/projections.py:181: UserWarning: Singular Jacobian matrix. Using SVD decomposition to perform the factorizations.
  warn('Singular Jacobian matrix. Using SVD decomposition to ' +
/usr/local/lib/python3.8/site-packages/scipy/optimize/_hessian_update_strategy.py:182: UserWarning: delta_grad == 0.0. Check if the approximated function is linear. If the function is linear better results can be obtained by defining the Hessian as zero instead of using quasi-Newton approximations.
  warn('delta_grad == 0.0. Check if the approximated '
/usr/local/lib/python3.8/site-packages/pygmo/_py_algorithms.py:527: UserWarning: Problem Hock Schittkowsky 71 has constraints and hessians, but trust-constr requires the callable to also accept lagrange multipliers. Thus, hessians of constraints are ignored.
  warnings.warn(
/usr/local/lib/python3.8/site-packages/scipy/optimize/_minimize.py:524: RuntimeWarning: Method SLSQP does not use Hessian information (hess).
  warn('Method %s does not use Hessian information (hess).' % method,
ok
runTest (pygmo._island_test.mp_island_test_case) ... ok
runTest (pygmo._island_test.ipyparallel_island_test_case) ... ok
runTest (pygmo.test.golomb_ruler_test_case) ... ok
runTest (pygmo.test.lennard_jones_test_case) ... ok
runTest (pygmo.test.de_test_case) ... ok
runTest (pygmo.test.nsga2_test_case) ... ok
runTest (pygmo.test.gaco_test_case) ... ok
runTest (pygmo.test.gwo_test_case) ... ok
runTest (pygmo.test.de1220_test_case) ... ok
runTest (pygmo.test.sea_test_case) ... ok
runTest (pygmo.test.pso_test_case) ... ok
runTest (pygmo.test.pso_gen_test_case) ... ok
runTest (pygmo.test.bee_colony_test_case) ... ok
runTest (pygmo.test.compass_search_test_case) ... ok
runTest (pygmo.test.sa_test_case) ... ok
runTest (pygmo.test.moead_test_case) ... ok
runTest (pygmo.test.sga_test_case) ... ok
runTest (pygmo.test.ihs_test_case) ... ok
runTest (pygmo.test.population_test_case) ... ok
runTest (pygmo.test.null_problem_test_case) ... ok
runTest (pygmo.test.hypervolume_test_case) ... ok
runTest (pygmo.test.mo_utils_test_case) ... ok
runTest (pygmo.test.con_utils_test_case) ... ok
runTest (pygmo.test.global_rng_test_case) ... ok
runTest (pygmo.test.estimate_sparsity_test_case) ... ok
runTest (pygmo.test.estimate_gradient_test_case) ... ok
runTest (pygmo.test.random_decision_vector_test_case) ... ok
runTest (pygmo.test.batch_random_decision_vector_test_case) ... ok
runTest (pygmo.test.cmaes_test_case) ... ok
runTest (pygmo.test.xnes_test_case) ... ok
runTest (pygmo.test.dtlz_test_case) ... ok
runTest (pygmo.test.cec2006_test_case) ... ok
runTest (pygmo.test.cec2009_test_case) ... ok
runTest (pygmo.test.cec2013_test_case) ... ok
runTest (pygmo.test.cec2014_test_case) ... ok
runTest (pygmo.test.luksan_vlcek1_test_case) ... ok
runTest (pygmo.test.minlp_rastrigin_test_case) ... ok
runTest (pygmo.test.rastrigin_test_case) ... ok
runTest (pygmo.test.translate_test_case) ... ok
runTest (pygmo.test.decompose_test_case) ... ok
runTest (pygmo.test.unconstrain_test_case) ... ok
runTest (pygmo.test.mbh_test_case) ... ok
runTest (pygmo.test.cstrs_self_adaptive_test_case) ... ok
runTest (pygmo.test.decorator_problem_test_case) ... ok
runTest (pygmo.test.wfg_test_case) ... ok
runTest (pygmo.test.nlopt_test_case) ... 
 objevals:        objval:      violated:    viol. norm:
         1         146149             18        691.455 i
         6        73016.4             18         2265.5 i
        11        691.388             18        18.8091 i
        16        12.8864             12      0.0650723 i
        21        6.23246              0              0

Optimisation return status: NLOPT_XTOL_REACHED (value = 4, Optimization stopped because xtol_rel or xtol_abs was reached)

 objevals:        objval:      violated:    viol. norm:
         1         192142             18        788.428 i
         6        1058.66             18        23.5218 i
        11        40.2466             12       0.423955 i
        16       0.512006              5      0.0350398 i
        21    1.55921e-13              0              0

Optimisation return status: NLOPT_XTOL_REACHED (value = 4, Optimization stopped because xtol_rel or xtol_abs was reached)
ok
runTest (pygmo.test.ipopt_test_case) ... 
******************************************************************************
This program contains Ipopt, a library for large-scale nonlinear optimization.
 Ipopt is released as open source code under the Eclipse Public License (EPL).
         For more information visit http://projects.coin-or.org/Ipopt
******************************************************************************

 objevals:        objval:      violated:    viol. norm:
         1         143465             18        723.615 i
         6        2069.31             18        43.8783 i
        11        300.481              7        13.6892 i
        16        7.37596              3      0.0130333 i
        21        6.23246              0              0

Optimisation return status: Solve_Succeeded (value = 0)

 objevals:        objval:      violated:    viol. norm:
         1         137229             18        441.927 i
         6        169.094             18         6.2434 i
        11    1.70233e-08              1    2.59369e-06 i

Optimisation return status: Solve_Succeeded (value = 0)
ok

======================================================================
ERROR: runTest (pygmo.test.archipelago_test_case)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/pygmo/test.py", line 878, in runTest
    self.run_torture_test_0()
  File "/usr/local/lib/python3.8/site-packages/pygmo/test.py", line 1348, in run_torture_test_0
    archi2 = archipelago(n=1000, algo=de(
  File "/usr/local/lib/python3.8/site-packages/pygmo/__init__.py", line 575, in _archi_init
    self.push_back(**kwargs)
  File "/usr/local/lib/python3.8/site-packages/pygmo/__init__.py", line 620, in _archi_push_back
    self._push_back(island(**kwargs))
RuntimeError: thread constructor failed: Resource temporarily unavailable

----------------------------------------------------------------------
Ran 69 tests in 63.152s

FAILED (errors=1)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.8/site-packages/pygmo/test.py", line 2957, in run_test_suite
    raise RuntimeError('One or more tests failed.')
RuntimeError: One or more tests failed.
*** Error code 1

Version: 2.18.0 Python: 3.8 OS: FreeBSD 13

bluescarni commented 2 years ago

Hi @yurivict

I believe the issue here is that the test suite is creating too many threads in a torture test, hitting some sort of OS limit. See the error message:

RuntimeError: thread constructor failed: Resource temporarily unavailable

Could you try to raise the thread limit number via ulimit or similar?

Alternatively, I think that if you invoke the test suite with pygmo.test.run_test_suite(1) rather than pygmo.test.run_test_suite(), the torture tests will be skipped.

yurivict commented 2 years ago

Increasing the thread limit from 1500 to 15000 helped.

Thanks!