e2nIEE / pandapower

Convenient Power System Modelling and Analysis based on PYPOWER and pandas
https://www.pandapower.org
Other
890 stars 486 forks source link

Huge performance decrease for "deepcopy" #631

Closed BDonnot closed 4 years ago

BDonnot commented 4 years ago

Hello, i noticed a huge performance decrease in copying pandapower grid. This performance issue arises when copying the powergrid (operator deep copy).

With the following script:

import pandapower as pp
import pandapower.networks as pn
import time
import copy
print("Pandapowerversion: {}".format(pp.__version__))
nb_powerflow = 1000
beg_ = time.time()
grid = pn.case14()
for i in range(nb_powerflow):
    # pp.runpp(grid, check_connectivity=False, init="results")
    copy.deepcopy(grid)
print("time to copy {} grid: {:.2f}s".format(nb_powerflow, time.time() - beg_))

it get the following results with pandapower 2.1.0 vs 2.2.0:

Pandapowerversion: 2.1.0
time to copy 1000 grid: 5.98s

versus

Pandapowerversion: 2.2.0
time to copy 1000 grid: 275.52s

This problem arises on both Ubuntu 16.04 and Fedora 27. I tested it on python 3.6

I think this issue is related to the #620 : deep copy is now done by serializing / de serializing json leading to large overhead.

dlohmeier commented 4 years ago

Yes, this issue arises from the json overload. Unfortunately, we have not yet found a solution of resolving circular references in the usual case of deepcopy for controllers, as the following test would fail:

import copy
net2 = copy.deepcopy(net)
assert net2.controller.object.at[0].net == net2

Currently, this is only possible with the help of the json overload. We are working on a solution (also referenced in #577 ).

rbolgaryn commented 4 years ago

The pull request #676 solves the issue mentioned in https://github.com/e2nIEE/pandapower/issues/631#issuecomment-579188278

copy.deepcopy can be used again after the pull request is merged