Closed TohokuEva closed 1 year ago
FabianHofmann
commented
1 year ago Hey @TohokuEva, I am wondering what your model looks like that you have 10425 different arrays of constraints. This looks suboptimal, and part of the reason why the writing time is high. Perhaps you make sure that define arrays on constraints with coordinates? This should speed up things.
TohokuEva
commented
1 year ago Thank you very mush for your answer, @FabianHofmann.
While defining KCL and KVL constraints I have used loops. This wad indeed the origin of the problem.
My previous code was this one:
for i in coords_nodes:
for t in coords_time:
lhs = ((var_gen_dispatch.sel(dim_nodes=i,dim_time=t)).sum()
+ (var_stor_discharging.sel(dim_nodes=i,dim_time=t)).sum()
- (var_stor_charging.sel(dim_nodes=i,dim_time=t)).sum()
+ (network_matrix_K.sel(dim_nodes=i) * var_line_flow.sel(dim_time=t)).sum())
model.add_constraints(lhs, '=', demand.sel(dim_nodes=i,dim_time=t),
name = 'KCL_{}_{}'.format(i,t))
for c in coords_cycles:
for t in coords_time:
lhs = (network_matrix_C.sel(dim_cycles=c) * line_reactance * var_line_flow.sel(dim_time=t)).sum()
model.add_constraints(lhs, '=', 0,
name = 'KVL_{}_{}'.format(c,t))After you pointed the problem I have tried to fix it by deleting the loops and specifying on which dimension the variables should be summed. I did not really know what is the good formulation, so I am not sure if I did it correctly. The new code is this one:
lhs = ((var_gen_dispatch).sum('dim_gen_techno')
+ (var_stor_discharging).sum('dim_stor_techno')
- (var_stor_charging).sum('dim_stor_techno')
+ (network_matrix_K * var_line_flow).sum('dim_lines'))
model.add_constraints(lhs, '=', demand, name = 'KCL')
lhs = (network_matrix_C * var_line_flow * line_reactance).sum('dim_lines')
model.add_constraints(lhs, '=', 0, name = 'KVL')I obtained the same result, but with a number of constraints drastically decreased:
Writing constraints.: 100%|██████████| 11/11 [00:00<00:00, 37.37it/s]
Writing variables.: 100%|██████████| 8/8 [00:00<00:00, 121.19it/s]Read LP format model from file C:\Users\Eva\AppData\Local\Temp\linopy-problem-tx0z8oe8.lp
Reading time = 0.20 seconds
obj: 126481 rows, 91595 columns, 367979 nonzeros
Gurobi Optimizer version 10.0.0 build v10.0.0rc2 (win64)
CPU model: 12th Gen Intel(R) Core(TM) i9-12900K, instruction set [SSE2|AVX|AVX2]
Thread count: 16 physical cores, 24 logical processors, using up to 24 threads
Optimize a model with 126481 rows, 91595 columns and 367979 nonzeros
Model fingerprint: 0xaa3f369a
Coefficient statistics:
Matrix range [1e-06, 2e+02]
Objective range [1e-02, 1e+05]
Bounds range [4e+02, 1e+07]
RHS range [6e+02, 6e+07]
Presolve removed 7713 rows and 7715 columns
Presolve time: 0.15s
Presolved: 118768 rows, 83880 columns, 354826 nonzeros
Concurrent LP optimizer: primal simplex, dual simplex, and barrier
Showing barrier log only...
Ordering time: 0.01s
Barrier statistics:
Dense cols : 81
Free vars : 6695
AA' NZ : 3.834e+05
Factor NZ : 1.991e+06 (roughly 100 MB of memory)
Factor Ops : 1.134e+08 (less than 1 second per iteration)
Threads : 14
Objective Residual
Iter Primal Dual Primal Dual Compl Time
0 2.05908200e+14 -3.84291651e+15 6.96e+10 4.15e+03 5.26e+12 0s
1 2.37145030e+14 -2.07731861e+15 5.70e+10 2.37e+05 2.48e+12 0s
2 1.39281994e+14 -1.68120405e+15 3.02e+10 8.45e+04 1.17e+12 0s
3 7.48499079e+13 -1.75250575e+15 1.44e+10 4.79e+04 6.36e+11 0s
4 3.63519649e+13 -1.80307024e+15 4.90e+09 1.02e+04 1.71e+11 0s
5 2.55046947e+13 -1.28282703e+15 2.44e+09 4.28e+03 8.11e+10 1s
6 2.08641032e+13 -9.86927082e+14 1.45e+09 2.37e+03 4.72e+10 1s
7 1.79799724e+13 -6.62212421e+14 8.66e+08 9.19e+02 2.36e+10 1s
8 1.62996655e+13 -5.56145461e+14 5.71e+08 6.13e+02 1.58e+10 1s
9 1.48920139e+13 -4.10745223e+14 3.43e+08 2.68e+02 8.71e+09 1s
10 1.27036089e+13 -2.36710884e+14 5.17e+07 7.19e+01 2.35e+09 1s
11 1.08221505e+13 -2.72790232e+13 1.94e+07 4.03e+00 3.41e+08 1s
12 7.81385424e+12 -1.25459861e+13 7.89e+06 1.70e+00 1.56e+08 1s
13 4.64090193e+12 -9.21035431e+12 1.41e+06 1.26e+00 8.41e+07 1s
14 3.66071977e+12 -7.80788460e+12 9.67e+05 2.21e+00 6.75e+07 1s
15 2.15706917e+12 -2.33085366e+12 3.45e+05 3.11e+00 2.49e+07 1s
16 1.25829759e+12 -1.34067622e+12 1.44e+05 1.40e+00 1.40e+07 1s
17 5.96802858e+11 -6.02885956e+11 5.01e+04 1.09e+00 6.36e+06 1s
18 3.09261641e+11 -1.91234391e+11 3.61e+03 4.14e-01 2.63e+06 1s
19 1.95736031e+11 -1.03025108e+11 4.22e+00 2.24e-01 1.57e+06 1s
20 1.27169044e+11 -7.91083101e+10 2.62e+00 3.63e-01 1.08e+06 1s
21 7.53853579e+10 -2.61494560e+10 1.44e+00 3.89e-01 5.32e+05 1s
22 5.26230399e+10 -6.21750537e+09 9.20e-01 3.65e-01 3.08e+05 1s
23 3.10837474e+10 1.47160160e+09 3.70e-01 2.20e-01 1.55e+05 1s
24 1.77664833e+10 2.31139980e+09 1.35e-01 8.76e-02 8.08e+04 1s
25 1.38019323e+10 5.99811191e+09 7.58e-02 3.60e-02 4.08e+04 1s
26 1.17194986e+10 6.98805479e+09 4.39e-02 2.79e-02 2.47e+04 1s
27 1.08168370e+10 7.57637732e+09 2.98e-02 6.76e-02 1.69e+04 1s
28 1.05474688e+10 7.79530339e+09 2.54e-02 5.62e-02 1.44e+04 1s
29 1.02049470e+10 7.98688715e+09 2.01e-02 4.73e-02 1.16e+04 1s
30 1.01387929e+10 8.07818319e+09 1.89e-02 4.35e-02 1.08e+04 1s
31 1.00498951e+10 8.25265912e+09 1.70e-02 3.60e-02 9.40e+03 1s
32 9.90306459e+09 8.39870399e+09 1.44e-02 4.86e-02 7.87e+03 1s
33 9.83678361e+09 8.44772011e+09 1.31e-02 4.52e-02 7.26e+03 2s
34 9.78729955e+09 8.50823320e+09 1.22e-02 4.19e-02 6.69e+03 2s
35 9.68576617e+09 8.66392022e+09 1.03e-02 3.33e-02 5.34e+03 2s
36 9.52086005e+09 8.74498175e+09 7.33e-03 2.71e-02 4.06e+03 2s
37 9.39270813e+09 8.81342891e+09 5.24e-03 2.11e-02 3.03e+03 2s
38 9.31977537e+09 8.89093165e+09 4.05e-03 1.42e-02 2.24e+03 2s
39 9.28747268e+09 8.93515235e+09 3.52e-03 1.50e-02 1.84e+03 2s
40 9.19958341e+09 8.95987749e+09 2.13e-03 1.39e-02 1.25e+03 2s
41 9.18346274e+09 9.00597495e+09 1.88e-03 7.92e-03 9.28e+02 2s
42 9.15370864e+09 9.01542616e+09 1.40e-03 5.50e-03 7.23e+02 2s
43 9.13921012e+09 9.02475199e+09 1.17e-03 6.53e-03 5.99e+02 2s
44 9.12361713e+09 9.03483949e+09 9.28e-04 6.32e-03 4.64e+02 2s
45 9.10934181e+09 9.04070645e+09 7.15e-04 5.89e-03 3.59e+02 2s
46 9.10230029e+09 9.04544674e+09 5.92e-04 5.14e-03 2.97e+02 2s
47 9.09216631e+09 9.04981166e+09 4.21e-04 4.29e-03 2.21e+02 2s
48 9.09061697e+09 9.05313908e+09 3.97e-04 3.69e-03 1.96e+02 2s
49 9.08751203e+09 9.05537159e+09 3.45e-04 3.11e-03 1.68e+02 2s
50 9.08094569e+09 9.05785298e+09 2.36e-04 2.60e-03 1.21e+02 2s
51 9.07781389e+09 9.06134599e+09 1.85e-04 2.08e-03 8.61e+01 2s
52 9.07379618e+09 9.06239848e+09 1.26e-04 1.89e-03 5.96e+01 2s
53 9.07197046e+09 9.06312530e+09 9.57e-05 1.65e-03 4.63e+01 2s
54 9.07025077e+09 9.06423894e+09 6.54e-05 1.27e-03 3.14e+01 2s
55 9.06867877e+09 9.06500298e+09 9.89e-04 9.11e-04 1.92e+01 2s
56 9.06777841e+09 9.06548237e+09 3.38e-03 5.81e-04 1.20e+01 2s
57 9.06723025e+09 9.06563339e+09 4.55e-03 4.88e-04 8.35e+00 2s
58 9.06704033e+09 9.06573719e+09 3.70e-03 3.80e-04 6.82e+00 3s
59 9.06692864e+09 9.06581696e+09 3.30e-03 2.92e-04 5.82e+00 3s
60 9.06667971e+09 9.06594734e+09 2.32e-03 3.39e-04 3.83e+00 3s
61 9.06658607e+09 9.06598391e+09 1.88e-03 3.72e-04 3.15e+00 3s
62 9.06644750e+09 9.06602358e+09 1.25e-03 4.01e-04 2.22e+00 3s
63 9.06636073e+09 9.06608835e+09 8.89e-04 4.11e-04 1.43e+00 3s
64 9.06629124e+09 9.06610711e+09 6.18e-04 4.12e-04 9.63e-01 3s
65 9.06624721e+09 9.06612531e+09 4.15e-04 3.51e-04 6.38e-01 3s
66 9.06622989e+09 9.06613225e+09 3.26e-04 3.04e-04 5.11e-01 3s
67 9.06621536e+09 9.06613682e+09 2.52e-04 2.65e-04 4.11e-01 3s
68 9.06619440e+09 9.06614879e+09 1.48e-04 1.61e-04 2.39e-01 3s
69 9.06618170e+09 9.06615589e+09 8.75e-05 2.05e-04 1.35e-01 3s
70 9.06617465e+09 9.06615858e+09 5.45e-05 2.31e-04 8.41e-02 3s
71 9.06616944e+09 9.06616019e+09 3.05e-05 2.51e-04 4.84e-02 3s
72 9.06616482e+09 9.06616151e+09 9.28e-06 2.50e-04 1.73e-02 3s
73 9.06616330e+09 9.06616218e+09 3.03e-06 2.09e-04 5.85e-03 4s
74 9.06616274e+09 9.06616242e+09 9.11e-07 1.58e-04 1.69e-03 4s
75 9.06616248e+09 9.06616245e+09 5.20e-08 1.71e-04 1.66e-04 4s
76 9.06616245e+09 9.06616246e+09 2.44e-07 1.80e-04 7.73e-07 4s
77 9.06616245e+09 9.06616246e+09 1.25e-08 1.83e-04 2.84e-10 4s
Barrier solved model in 77 iterations and 3.83 seconds (4.14 work units)
Optimal objective 9.06616245e+09
Crossover log...
60897 DPushes remaining with DInf 2.3298577e-03 4s
0 DPushes remaining with DInf 2.3298577e-03 4s
Warning: Markowitz tolerance tightened to 0.5
5854 PPushes remaining with PInf 0.0000000e+00 4s
0 PPushes remaining with PInf 0.0000000e+00 4s
Push phase complete: Pinf 0.0000000e+00, Dinf 3.6874893e-10 4s
Solved with barrier
Iteration Objective Primal Inf. Dual Inf. Time
47816 9.0661625e+09 0.000000e+00 0.000000e+00 4s
Solved in 47816 iterations and 4.43 seconds (5.24 work units)
Optimal objective 9.066162454e+09And this time, the real solving time (I calculated with time.time()) is 5.6 seconds instead of more than one hour. Thank you very much for your help!
FabianHofmann
commented
1 year ago Great! There you go :)
FabianHofmann
commented
1 year ago I'll close this, reopen if anything related comes up again.
linopy version: 0.0.15 Gurobi Optimizer version: 10.0.0
Describe the bug
I was solving an energy model with 10 nodes, 744 time steps; gas, solar, wind onshore and wind offshore production; battery and hydrogen storage.
The information returned in the console about time solving is: "Barrier solved model in 77 iterations and 4.07 seconds (4.14 work units) Optimal objective 9.06616245e+09 [...] Solved in 47816 iterations and 4.59 seconds (5.24 work units) Optimal objective 9.066162454e+09"
But when I measure real time solving by using time.time() in the line just before and the line just after model.solve(), I obtain 4442 seconds, which seems to me really different from the information returned in the console.
I also tried with 24, 120, 240 and 480 time steps, and I had same problem.
Solving information