Closed dannyopts closed 2 months ago
odow
commented
2 months ago Can you provide a log of the solution process?
There are no hooks to muck with the cuts. In general, you can ignore this warning, but it often means that some part of your model could be improved.
dannyopts
commented
2 months ago I can't replicate the exact warning I got earlier right now, but here is an example where I train a model, then interupt it and then run it again, you can see there was no warnings during the training, but when I resume training the rhs has some very small values.
-------------------------------------------------------------------
SDDP.jl (c) Oscar Dowson and contributors, 2017-23
-------------------------------------------------------------------
problem
nodes : 53
state variables : 5
scenarios : Inf
existing cuts : false
options
solver : serial mode
risk measure : SDDP.Expectation()
sampling scheme : SDDP.InSampleMonteCarlo
subproblem structure
VariableRef : [11, 391]
AffExpr in MOI.EqualTo{Float64} : [217, 217]
AffExpr in MOI.LessThan{Float64} : [1, 169]
VariableRef in MOI.GreaterThan{Float64} : [6, 386]
VariableRef in MOI.LessThan{Float64} : [5, 385]
numerical stability report
matrix range [1e-03, 4e+00]
objective range [1e+00, 4e+03]
bounds range [3e+00, 6e+01]
rhs range [3e+00, 3e+00]
-------------------------------------------------------------------
iteration simulation bound time (s) solves pid
-------------------------------------------------------------------
1 3.765385e+03 1.209591e+02 6.881621e-01 1462 1
2 4.484024e+03 6.001353e+01 8.759099e+00 13919 1
3 3.825398e+03 1.673981e+02 9.335534e+00 15381 1
4 1.614157e+03 1.996049e+02 9.777245e+00 16843 1
5 7.593029e+02 2.088820e+02 1.034131e+01 18305 1
6 1.372024e+03 2.129336e+02 1.077601e+01 19767 1
7 1.606268e+03 2.174668e+02 1.121425e+01 21229 1
8 5.006371e+02 2.129940e+02 1.166923e+01 22691 1
9 5.864301e+02 2.278504e+02 1.219528e+01 24153 1
10 6.372647e+02 2.294004e+02 1.263317e+01 25615 1
11 3.763074e+02 2.311329e+02 1.301317e+01 27077 1
12 4.503891e+02 2.370776e+02 1.340642e+01 28539 1
13 2.980993e+02 2.416176e+02 1.378349e+01 30001 1
14 2.816669e+02 2.421145e+02 1.417128e+01 31463 1
15 2.505686e+02 2.434508e+02 1.456582e+01 32925 1
16 3.546565e+02 2.456188e+02 1.497626e+01 34387 1
17 2.718219e+02 2.462821e+02 1.536984e+01 35849 1
18 3.288580e+02 2.494258e+02 1.575462e+01 37311 1
19 2.870288e+02 2.513201e+02 1.615366e+01 38773 1
20 3.057074e+02 2.540069e+02 1.654961e+01 40235 1
21 3.025645e+02 2.545979e+02 1.694901e+01 41697 1
22 2.855894e+02 2.561419e+02 1.734401e+01 43159 1
23 2.625401e+02 2.587836e+02 1.771740e+01 44621 1
24 2.601189e+02 2.601611e+02 1.810583e+01 46083 1
25 2.940693e+02 2.602913e+02 1.850237e+01 47545 1
26 2.765799e+02 2.606613e+02 1.889195e+01 49007 1
27 2.815766e+02 2.609637e+02 1.926868e+01 50469 1
28 2.690177e+02 2.604475e+02 1.964259e+01 51931 1
29 2.649696e+02 2.627237e+02 2.022883e+01 53393 1
30 3.768636e+02 2.629391e+02 2.064997e+01 54855 1
31 2.964850e+02 2.631215e+02 2.106545e+01 56317 1
32 2.691314e+02 2.633649e+02 2.143886e+01 57779 1
33 2.783447e+02 2.641154e+02 2.190281e+01 59241 1
34 2.657391e+02 2.646819e+02 2.235102e+01 60703 1
35 2.649729e+02 2.648500e+02 2.285591e+01 62165 1
36 2.792809e+02 2.651775e+02 2.326710e+01 63627 1
37 2.786219e+02 2.652685e+02 2.364356e+01 65089 1
38 2.773879e+02 2.652966e+02 2.401850e+01 66551 1
39 2.670780e+02 2.658636e+02 2.440214e+01 68013 1
40 2.711787e+02 2.658767e+02 2.478274e+01 69475 1
41 2.851768e+02 2.658861e+02 2.519590e+01 70937 1
42 2.779956e+02 2.658937e+02 2.565985e+01 72399 1
43 2.750167e+02 2.659769e+02 2.607883e+01 73861 1
44 2.800650e+02 2.658433e+02 2.674562e+01 75323 1
45 2.658433e+02 2.663431e+02 3.152516e+01 87780 1
46 2.767353e+02 2.669034e+02 3.196999e+01 89242 1
47 2.691588e+02 2.669070e+02 3.238416e+01 90704 1
48 2.786554e+02 2.669403e+02 3.282495e+01 92166 1
49 2.655353e+02 2.670684e+02 3.322972e+01 93628 1
50 2.650161e+02 2.676918e+02 3.365041e+01 95090 1
51 2.702117e+02 2.678317e+02 3.409735e+01 96552 1
52 2.768484e+02 2.681569e+02 3.456637e+01 98014 1
53 2.696476e+02 2.682343e+02 3.502870e+01 99476 1
54 2.764791e+02 2.682983e+02 3.551073e+01 100938 1
55 2.691652e+02 2.684335e+02 3.598459e+01 102400 1
56 2.658602e+02 2.684915e+02 3.639239e+01 103862 1
57 2.687974e+02 2.684987e+02 3.677373e+01 105324 1
58 2.680208e+02 2.685705e+02 3.722237e+01 106786 1
59 2.676513e+02 2.686306e+02 3.761540e+01 108248 1
60 2.751999e+02 2.685767e+02 3.804022e+01 109710 1
61 2.688262e+02 2.685967e+02 3.841834e+01 111172 1
62 2.769281e+02 2.689271e+02 3.881748e+01 112634 1
63 2.705103e+02 2.690077e+02 3.921606e+01 114096 1
64 2.675243e+02 2.691472e+02 3.960272e+01 115558 1
65 2.686786e+02 2.692894e+02 4.005919e+01 117020 1
66 2.763772e+02 2.694300e+02 4.045930e+01 118482 1
67 2.751971e+02 2.695532e+02 4.084502e+01 119944 1
68 2.679343e+02 2.695885e+02 4.121982e+01 121406 1
69 2.758892e+02 2.697565e+02 4.160343e+01 122868 1
70 2.690810e+02 2.698380e+02 4.200077e+01 124330 1
71 2.681919e+02 2.700997e+02 4.238875e+01 125792 1
72 2.681344e+02 2.702472e+02 4.277824e+01 127254 1
73 2.676911e+02 2.710513e+02 4.323111e+01 128716 1
74 2.624346e+02 2.710539e+02 4.364274e+01 130178 1
75 2.685774e+02 2.710573e+02 4.405928e+01 131640 1
76 2.692659e+02 2.710723e+02 4.444195e+01 133102 1
77 2.672106e+02 2.712881e+02 4.484112e+01 134564 1
78 2.723427e+02 2.712971e+02 4.521921e+01 136026 1
79 2.684964e+02 2.711859e+02 4.560259e+01 137488 1
80 2.701191e+02 2.713109e+02 4.600906e+01 138950 1
81 2.682819e+02 2.715058e+02 4.639942e+01 140412 1
82 2.776739e+02 2.715497e+02 4.680563e+01 141874 1
83 2.699187e+02 2.715681e+02 4.720196e+01 143336 1
84 2.693063e+02 2.700326e+02 4.760862e+01 144798 1
85 2.767555e+02 2.708361e+02 4.805503e+01 146260 1
86 2.709254e+02 2.715771e+02 4.847322e+01 147722 1
87 2.786338e+02 2.718689e+02 4.886669e+01 149184 1
88 2.739065e+02 2.721587e+02 4.924716e+01 150646 1
89 2.721657e+02 2.722349e+02 4.964230e+01 152108 1
90 2.785528e+02 2.723331e+02 5.005863e+01 153570 1
91 2.699327e+02 2.723823e+02 5.047705e+01 155032 1
92 2.698823e+02 2.724267e+02 5.087142e+01 156494 1
93 2.746270e+02 2.724869e+02 5.129617e+01 157956 1
94 2.742251e+02 2.725327e+02 5.169067e+01 159418 1
95 2.692033e+02 2.725574e+02 5.207172e+01 160880 1
96 2.741163e+02 2.725579e+02 5.248780e+01 162342 1
97 2.702283e+02 2.725667e+02 5.286361e+01 163804 1
98 2.740060e+02 2.725887e+02 5.331959e+01 165266 1
99 2.699776e+02 2.727078e+02 5.376565e+01 166728 1
100 2.716149e+02 2.727895e+02 5.425564e+01 168190 1
101 2.767697e+02 2.728132e+02 5.471982e+01 169652 1
102 2.716356e+02 2.728351e+02 5.515811e+01 171114 1
103 2.721697e+02 2.728505e+02 5.561749e+01 172576 1
104 2.715830e+02 2.728646e+02 5.607587e+01 174038 1
105 2.744227e+02 2.728919e+02 5.654617e+01 175500 1
106 2.720827e+02 2.729322e+02 5.700192e+01 176962 1
107 2.713051e+02 2.730258e+02 5.742817e+01 178424 1
108 2.701360e+02 2.730560e+02 5.786613e+01 179886 1
109 2.697768e+02 2.730563e+02 5.833459e+01 181348 1
110 2.699177e+02 2.730575e+02 5.881011e+01 182810 1
111 2.706733e+02 2.730590e+02 5.925293e+01 184272 1
112 2.699624e+02 2.730619e+02 5.966395e+01 185734 1
113 2.704701e+02 2.730700e+02 6.008306e+01 187196 1
114 2.702547e+02 2.730706e+02 6.048220e+01 188658 1
115 2.740173e+02 2.731642e+02 6.088892e+01 190120 1
116 2.732193e+02 2.731812e+02 6.129532e+01 191582 1
117 2.737857e+02 2.732277e+02 6.170928e+01 193044 1
118 2.722182e+02 2.732334e+02 6.213349e+01 194506 1
119 2.732623e+02 2.732336e+02 6.258349e+01 195968 1
120 2.732493e+02 2.732432e+02 6.301085e+01 197430 1
121 2.732327e+02 2.732432e+02 6.340045e+01 198892 1
122 2.732065e+02 2.732437e+02 6.381475e+01 200354 1
123 2.727812e+02 2.732461e+02 6.420399e+01 201816 1
124 2.732397e+02 2.732517e+02 6.458799e+01 203278 1
125 2.741615e+02 2.732521e+02 6.502784e+01 204740 1
126 2.731754e+02 2.732502e+02 6.547302e+01 206202 1
127 2.732913e+02 2.732566e+02 6.592271e+01 207664 1
128 2.731266e+02 2.732662e+02 6.634441e+01 209126 1
129 2.732701e+02 2.732675e+02 6.676560e+01 210588 1
130 2.732296e+02 2.732697e+02 6.725525e+01 212050 1
131 2.732426e+02 2.732702e+02 6.767421e+01 213512 1
132 2.732952e+02 2.732716e+02 6.808930e+01 214974 1
133 2.731296e+02 2.732719e+02 6.848047e+01 216436 1
134 2.741880e+02 2.732742e+02 6.887538e+01 217898 1
135 2.732304e+02 2.732733e+02 6.930076e+01 219360 1
136 2.734006e+02 2.732767e+02 6.972888e+01 220822 1
137 2.732713e+02 2.732783e+02 7.018592e+01 222284 1
138 2.733327e+02 2.732783e+02 7.063411e+01 223746 1
139 2.731752e+02 2.732787e+02 7.109099e+01 225208 1
140 2.731935e+02 2.732788e+02 7.150876e+01 226670 1
141 2.733447e+02 2.732810e+02 7.190082e+01 228132 1
142 2.732282e+02 2.732907e+02 7.233107e+01 229594 1
143 2.733367e+02 2.732915e+02 7.280462e+01 231056 1
144 2.733540e+02 2.732929e+02 7.321587e+01 232518 1
145 2.732191e+02 2.732940e+02 7.787809e+01 244975 1
146 2.733941e+02 2.732958e+02 7.827377e+01 246437 1
147 2.734007e+02 2.732961e+02 7.865291e+01 247899 1
148 2.732955e+02 2.732970e+02 7.904198e+01 249361 1
149 2.732608e+02 2.732992e+02 7.945475e+01 250823 1
150 2.732483e+02 2.732994e+02 7.983934e+01 252285 1
151 2.732633e+02 2.733017e+02 8.070554e+01 253747 1
152 2.732395e+02 2.733023e+02 8.136021e+01 255209 1
153 2.732461e+02 2.733030e+02 8.189419e+01 256671 1
154 2.732517e+02 2.733040e+02 8.238376e+01 258133 1
155 2.732127e+02 2.733057e+02 8.277345e+01 259595 1
156 2.732516e+02 2.733057e+02 8.317172e+01 261057 1
157 2.732410e+02 2.733071e+02 8.382510e+01 262519 1
158 2.732560e+02 2.733075e+02 8.438923e+01 263981 1
159 2.732609e+02 2.733076e+02 8.478089e+01 265443 1
160 2.732356e+02 2.733101e+02 8.516757e+01 266905 1
161 2.733541e+02 2.733108e+02 8.557684e+01 268367 1
162 2.732640e+02 2.733117e+02 8.600450e+01 269829 1
163 2.732554e+02 2.733131e+02 8.641938e+01 271291 1
164 2.732668e+02 2.733140e+02 8.695604e+01 272753 1
165 2.732148e+02 2.733133e+02 8.734813e+01 274215 1
166 2.732172e+02 2.733146e+02 8.775539e+01 275677 1
167 2.732633e+02 2.733178e+02 8.837479e+01 277139 1
168 2.732569e+02 2.733197e+02 8.890979e+01 278601 1
169 2.732485e+02 2.733202e+02 8.931189e+01 280063 1
170 2.732282e+02 2.733204e+02 8.970530e+01 281525 1
171 2.732673e+02 2.733204e+02 9.009976e+01 282987 1
172 2.734463e+02 2.733206e+02 9.048857e+01 284449 1
173 2.733240e+02 2.733213e+02 9.087605e+01 285911 1
174 2.732572e+02 2.733215e+02 9.126965e+01 287373 1
175 2.732490e+02 2.733238e+02 9.164772e+01 288835 1
176 2.732605e+02 2.733253e+02 9.207615e+01 290297 1
177 2.732561e+02 2.733291e+02 9.245411e+01 291759 1
178 2.732427e+02 2.733298e+02 9.282802e+01 293221 1
179 2.732690e+02 2.733315e+02 9.320067e+01 294683 1
180 2.733610e+02 2.733334e+02 9.357569e+01 296145 1
181 2.732791e+02 2.733344e+02 9.403883e+01 297607 1
182 2.732684e+02 2.733346e+02 9.442919e+01 299069 1
183 2.734138e+02 2.733370e+02 9.488480e+01 300531 1
184 2.731826e+02 2.733371e+02 9.528432e+01 301993 1
185 2.731728e+02 2.733366e+02 9.566486e+01 303455 1
186 2.735640e+02 2.733374e+02 9.604556e+01 304917 1
187 2.732083e+02 2.733379e+02 9.644976e+01 306379 1
188 2.731897e+02 2.733402e+02 9.681989e+01 307841 1
189 2.732752e+02 2.733403e+02 9.720872e+01 309303 1
190 2.732142e+02 2.733431e+02 9.758789e+01 310765 1
191 2.731434e+02 2.733450e+02 9.796999e+01 312227 1
192 2.733890e+02 2.733453e+02 9.837913e+01 313689 1
193 2.734131e+02 2.733463e+02 9.876280e+01 315151 1
194 2.733017e+02 2.733473e+02 9.916037e+01 316613 1
-------------------------------------------------------------------
status : interrupted
total time (s) : 9.916037e+01
total solves : 316613
best bound : 2.733473e+02
simulation ci : 3.605292e+02 ± 6.902418e+01
numeric issues : 0
-------------------------------------------------------------------continue training
-------------------------------------------------------------------
SDDP.jl (c) Oscar Dowson and contributors, 2017-23
-------------------------------------------------------------------
problem
nodes : 53
state variables : 5
scenarios : Inf
existing cuts : true
options
solver : serial mode
risk measure : SDDP.Expectation()
sampling scheme : SDDP.InSampleMonteCarlo
subproblem structure
VariableRef : [11, 391]
AffExpr in MOI.EqualTo{Float64} : [217, 217]
AffExpr in MOI.GreaterThan{Float64} : [6, 20]
AffExpr in MOI.LessThan{Float64} : [1, 169]
VariableRef in MOI.EqualTo{Float64} : [5, 5]
VariableRef in MOI.GreaterThan{Float64} : [6, 386]
VariableRef in MOI.LessThan{Float64} : [5, 385]
numerical stability report
matrix range [1e-03, 2e+05]
objective range [1e+00, 4e+03]
bounds range [3e+00, 6e+01]
rhs range [5e-15, 5e+03]
WARNING: numerical stability issues detected
- rhs range contains small coefficients
Very large or small absolute values of coefficients
can cause numerical stability issues. Consider
reformulating the model.
-------------------------------------------------------------------
iteration simulation bound time (s) solves pid
-------------------------------------------------------------------
1 2.731142e+02 2.733487e+02 6.223300e-01 318549 1
2 2.731659e+02 2.733536e+02 4.812463e+00 329560 1
3 2.733208e+02 2.733552e+02 5.274251e+00 331022 1
4 2.731880e+02 2.733577e+02 5.857349e+00 332484 1Do you have some ideas on what about my model might lead to this? I can just ignore this, but keen to try and improve my model if I can :)
odow
commented
2 months ago Oh, I would just ignore this. It's not causing any problems.
There's the line at the bottom of the first training:
numeric issues : 0When you resume, you have some cuts with small intercepts, but that doesn't matter. It's really only a problem if they are in the matrix.
dannyopts
commented
2 months ago Ok, will do. thank you :)
Hi,
I am getting some warnings about cuts with really small coefficents (1e-10 and smaller), but all of my state vars have quite small bounds (>= 0, <=10) and my objective is generally in the range 250.
My original sub problems are not ill conditioned, but as cuts get added I get these warnings, I presume that the problem begins to become almost flat in some regions. I'm not sure how to reason about the smallest duals which will be produced after many iterations from looking at the original sub problems? I know you mention rounding input values (not measuring water inflow to 15 decimal places for example), but this doesnt entirely remove the adding of a small coefficent.
In such cases I am thinking we could either move the cut down by the maximum that this coefficent could matter (1e-9 in this case), or just discard this coefficent which might induce a slight suboptimality but shouldnt really matter to the overall optimisation?
Moving down by the constant will still end up adding small values to the model and give numeric issues, so the second approach seems better.
Does this idea make sense?
Is there a hook or similar which I could utilise to do something like this within this library?
Once again, thank you for this brilliant tool!!!!