Optionally release memory during solving process

PyPSA / linopy

Linear optimization with N-D labeled arrays in Python

https://linopy.readthedocs.io

MIT License

154 stars 42 forks source link

Optionally release memory during solving process #219

Open FabianHofmann opened 5 months ago

FabianHofmann commented 5 months ago

We could optionally write the model references to a NetCDF file just before the solving process and reassign afterwards. This could reduce the total memory usage by 30%. (Example: for a 112 GB peak memory example solving, 31 GB was held by linopy and 81 GB by gurobi.)

tgi-climact commented 1 month ago

I am testing https://github.com/PyPSA/linopy/pull/281 on larger problems of PyPSA-Eur with multiple planning horizons and scenarios. While waiting for a token, I have a lot of memory allocated for each waiting solve_sector_network_myopic. Could my problem be related to this issue ? Thanks