search

spyder-ide / spyder

Official repository for Spyder - The Scientific Python Development Environment
https://www.spyder-ide.org
MIT License
8.2k stars 1.59k forks source link

Spyder buffers scipy.optimize.minimize() convergence messages between runs #18039

Open nbfazel opened 2 years ago

nbfazel commented 2 years ago

Description

What steps will reproduce the problem?

I'm using scipy.optimize.minimize() as follows:

OptimizeResult = optimize.minimize(self.Z_f, x0, args=parameters, method='L-BFGS-B', jac=self.dZ_f, options={'disp': True})

The "options={'disp': True}" argument tells optimize.minimize() to print optimization convergence messages. However, when I run the program in Spyder with a new kernel, during the very first run, convergence messages are suppressed. When I run again (same kernel), the messages from the previous run are printed along with some of the messages from the current run. If I run a 3rd time, the remaining messages from the previous run and some or all messages from the 3rd run are printed, and so on. When I restart the kernel between runs, convergence messages are suppressed and never printed no matter how many times I run the program.

For example, the following optimization reproducibly converges in 27 iterations each time it's run. The first run printed no convergence messages at all. The 2nd run, reproduced below, printed iterations 0 to 27 from the first run and iterations 0 to 23 (below it) from the 2nd run. (Iteration 24-27 for the 2nd run are missing and would be printed in the 3rd run if I ran again.)

It seems messages are buffered and not flushed out in a timely manner.

Thanks, Neil

Convergence messages from optimize.minimize(). Convergence is alway after 27 iterations

RUNNING THE L-BFGS-B CODE

       * * *

Machine precision = 2.220D-16 N = 3 M = 10

At X0 0 variables are exactly at the bounds

At iterate 0 f= 7.03337D+05 |proj g|= 4.00718D+06

At iterate 1 f= 8.25333D+03 |proj g|= 1.19944D+04

At iterate 2 f= 7.30713D+03 |proj g|= 8.86422D+03

At iterate 3 f= 5.37744D+03 |proj g|= 3.97451D+03

At iterate 4 f= 4.21818D+03 |proj g|= 2.18222D+03

At iterate 5 f= 3.04832D+03 |proj g|= 1.16546D+03

At iterate 6 f= 2.16928D+03 |proj g|= 6.58421D+02

At iterate 7 f= 1.59417D+03 |proj g|= 3.55064D+02

At iterate 8 f= 1.25258D+03 |proj g|= 1.87290D+02

At iterate 9 f= 1.02133D+03 |proj g|= 1.06669D+02

At iterate 10 f= 8.83831D+02 |proj g|= 4.63954D+02

At iterate 11 f= 8.64742D+02 |proj g|= 9.12474D+01

At iterate 12 f= 8.41545D+02 |proj g|= 4.92149D+01

At iterate 13 f= 8.24391D+02 |proj g|= 1.17475D+02

At iterate 14 f= 8.19805D+02 |proj g|= 3.12590D+01

At iterate 15 f= 8.17839D+02 |proj g|= 8.14758D+00

At iterate 16 f= 8.17382D+02 |proj g|= 8.03946D+00

At iterate 17 f= 8.16350D+02 |proj g|= 9.63922D+00

At iterate 18 f= 8.08699D+02 |proj g|= 1.56648D+01

At iterate 19 f= 7.88834D+02 |proj g|= 2.43618D+01

At iterate 20 f= 7.82030D+02 |proj g|= 1.11805D+01

At iterate 21 f= 7.76987D+02 |proj g|= 2.00426D+01

At iterate 22 f= 7.71927D+02 |proj g|= 1.64120D+01

At iterate 23 f= 7.71681D+02 |proj g|= 1.07082D+01

At iterate 24 f= 7.71619D+02 |proj g|= 7.11228D-01

At iterate 25 f= 7.71618D+02 |proj g|= 8.72199D-02

At iterate 26 f= 7.71618D+02 |proj g|= 3.19921D-02

At iterate 27 f= 7.71618D+02 |proj g|= 2.02636D-04

       * * *

Tit = total number of iterations Tnf = total number of function evaluations Tnint = total number of segments explored during Cauchy searches Skip = number of BFGS updates skipped Nact = number of active bounds at final generalized Cauchy point Projg = norm of the final projected gradient F = final function value

       * * *

N Tit Tnf Tnint Skip Nact Projg F 3 27 31 1 0 0 2.026D-04 7.716D+02 F = 771.61838501664488

CONVERGENCE: REL_REDUCTION_OFF<=_FACTR*EPSMCH
RUNNING THE L-BFGS-B CODE

       * * *

Machine precision = 2.220D-16 N = 3 M = 10

At X0 0 variables are exactly at the bounds

At iterate 0 f= 7.03337D+05 |proj g|= 4.00718D+06

At iterate 1 f= 8.25333D+03 |proj g|= 1.19944D+04

At iterate 2 f= 7.30713D+03 |proj g|= 8.86422D+03

At iterate 3 f= 5.37744D+03 |proj g|= 3.97451D+03

At iterate 4 f= 4.21818D+03 |proj g|= 2.18222D+03

At iterate 5 f= 3.04832D+03 |proj g|= 1.16546D+03

At iterate 6 f= 2.16928D+03 |proj g|= 6.58421D+02

At iterate 7 f= 1.59417D+03 |proj g|= 3.55064D+02

At iterate 8 f= 1.25258D+03 |proj g|= 1.87290D+02

At iterate 9 f= 1.02133D+03 |proj g|= 1.06669D+02

At iterate 10 f= 8.83831D+02 |proj g|= 4.63954D+02

At iterate 11 f= 8.64742D+02 |proj g|= 9.12474D+01

At iterate 12 f= 8.41545D+02 |proj g|= 4.92149D+01

At iterate 13 f= 8.24391D+02 |proj g|= 1.17475D+02

At iterate 14 f= 8.19805D+02 |proj g|= 3.12590D+01

At iterate 15 f= 8.17839D+02 |proj g|= 8.14758D+00

At iterate 16 f= 8.17382D+02 |proj g|= 8.03946D+00

At iterate 17 f= 8.16350D+02 |proj g|= 9.63922D+00

At iterate 18 f= 8.08699D+02 |proj g|= 1.56648D+01

At iterate 19 f= 7.88834D+02 |proj g|= 2.43618D+01

At iterate 20 f= 7.82030D+02 |proj g|= 1.11805D+01

At iterate 21 f= 7.76987D+02 |proj g|= 2.00426D+01

At iterate 22 f= 7.71927D+02 |proj g|= 1.64120D+01

At iterate 23 f= 7.71681D+02 |proj g|= 1.07082D+01

Versions

Dependencies

# Mandatory:
applaunchservices >=0.1.7     :  0.3.0 (OK)
atomicwrites >=1.2.0          :  1.4.0 (OK)
chardet >=2.0.0               :  4.0.0 (OK)
cloudpickle >=0.5.0           :  2.1.0 (OK)
cookiecutter >=1.6.0          :  1.7.3 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree >=3.0.2          :  3.0.2 (OK)
IPython >=7.31.1;<8.0.0       :  7.33.0 (OK)
jedi >=0.17.2;<0.19.0         :  0.18.1 (OK)
jellyfish >=0.7               :  0.9.0 (OK)
jsonschema >=3.2.0            :  4.5.1 (OK)
keyring >=17.0.0              :  23.5.1 (OK)
nbconvert >=4.0               :  6.5.0 (OK)
numpydoc >=0.6.0              :  1.3.1 (OK)
parso >=0.7.0;<0.9.0          :  0.8.3 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.9.1 (OK)
pygments >=2.0                :  2.12.0 (OK)
pylint >=2.5.0                :  2.13.9 (OK)
pyls_spyder >=0.4.0           :  0.4.0 (OK)
pylsp >=1.4.1;<1.5.0          :  1.4.1 (OK)
pylsp_black >=1.2.0           :  1.2.1 (OK)
qdarkstyle >=3.0.2;<3.1.0     :  3.0.3 (OK)
qstylizer >=0.1.10            :  0.2.1 (OK)
qtawesome >=1.0.2             :  1.1.1 (OK)
qtconsole >=5.3.0;<5.4.0      :  5.3.0 (OK)
qtpy >=2.1.0                  :  2.1.0 (OK)
rtree >=0.9.7                 :  1.0.0 (OK)
setuptools >=49.6.0           :  62.3.2 (OK)
sphinx >=0.6.6                :  4.5.0 (OK)
spyder_kernels >=2.3.1;<2.4.0 :  2.3.1 (OK)
textdistance >=4.2.0          :  4.2.2 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog >=0.10.3             :  2.1.8 (OK)
zmq >=22.1.0                  :  23.0.0 (OK)

# Optional:
cython >=0.21                 :  None (NOK)
matplotlib >=3.0.0            :  3.5.2 (OK)
numpy >=1.7                   :  1.22.4 (OK)
pandas >=1.1.1                :  1.4.2 (OK)
scipy >=0.17.0                :  1.8.1 (OK)
sympy >=0.7.3                 :  None (NOK)
ccordoba12 commented 2 years ago

Hey @nbfazel, thanks for reporting. Please post a simple code example that generates this problem so we can try to reproduce it on our side.

nbfazel commented 2 years ago

Thank you. I will do that.

Neil

On Sat, May 28, 2022 at 12:08 PM Carlos Cordoba @.***> wrote:

Hey @nbfazel https://github.com/nbfazel, thanks for reporting. Please post a simple code example that generates this problem so we can try to reproduce it on our side.

— Reply to this email directly, view it on GitHub https://github.com/spyder-ide/spyder/issues/18039#issuecomment-1140290075, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADML47EY55JH727FXLAWEPLVMJAIHANCNFSM5XGCZGFQ . You are receiving this because you were mentioned.Message ID: @.***>

nbfazel commented 2 years ago

Thanks, again. Here's the sample code (I couldn't attach a .py file):

#!/usr/bin/env python3
from scipy import optimize

# used for run 1
x0 = [1.3, 0.7, 0.8, 1.9, 1.2]

# used for runs 2-4
# x0 = [2.3, 1.7, 1.8, 2.9, 2.2]

print("Sample code: Optimization starting...")
OptimizeResult = optimize.minimize(optimize.rosen, x0, method='L-BFGS-B', jac=optimize.rosen_der, options={'disp': True})
print("Sample code: Optimization ended, printing the results next...")
print(OptimizeResult)

Here's are the convergence messages that were actually displayed for each run:

Run 1 (24 iterations): No convergence messages were displayed. Run 2 (33 iterations): Convergences messages for iterations 0-24 from run 1 and iterations 0-26 from run 2 are displayed. Run 3 (33 iterations): No convergence messages were displayed. Run 4 (33 iterations): Convergences messages for iterations 27-33 from run 2, iterations 0-33 from run 3, and iterations 0-1 from run 4 are displayed.

issue_18039_sample_code_output.pdf

I hope this helps diagnose the issue.

Thanks, Neil

nbfazel commented 2 years ago

Hi Carlos,

I noticed the status is still "Awaiting Followup". Is there any other information I should provide in addition to the above code sample?

Thanks, Neil

ccordoba12 commented 2 years ago

I'm sorry, I just haven't had time to test your code sample locally (thanks a lot for it, by the way). I hope to do it before the end of the week.

nbfazel commented 2 years ago

Great, thank you so much.

On Mon, Jun 6, 2022 at 12:24 PM Carlos Cordoba @.***> wrote:

I'm sorry, I just haven't had time to test your code sample locally (thanks a lot for it, by the way). I hope to do it before the end of the week.

— Reply to this email directly, view it on GitHub https://github.com/spyder-ide/spyder/issues/18039#issuecomment-1147638342, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADML47GTCGK3VOWWG3XJPKLVNYQ33ANCNFSM5XGCZGFQ . You are receiving this because you were mentioned.Message ID: @.***>

nbfazel commented 1 year ago

Hi Carlos. I was wondering if there were any updates on this. Thanks.