line loading w/ and w/o network clustering differs significantly

[nesnoj]

This one came up in #137 and deserves a separate issue. Sorry for posting figures without legend in #137.. here are the full ones:

Full param set:

args = {# Setup and Configuration:
        'db': 'oedb_remote', # db session
        'gridversion': None, # None for model_draft or Version number (e.g. v0.2.11) for grid schema
        'method': 'lopf', # lopf or pf
        'pf_post_lopf': False, # state whether you want to perform a pf after a lopf simulation
        'start_snapshot': 1,
        'end_snapshot' : 2,
        'scn_name': 'Status Quo', # state which scenario you want to run: Status Quo, NEP 2035, eGo100
        'solver': 'gurobi', # glpk, cplex or gurobi
        # Export options:
        'lpfile': False, # state if and where you want to save pyomo's lp file: False or /path/tofolder
        'results': 'results', # state if and where you want to save results as csv: False or /path/tofolder
        'export': False, # state if you want to export the results back to the database
        # Settings:        
        'storage_extendable':False, # state if you want storages to be installed at each node if necessary.
        'generator_noise':False, # state if you want to apply a small generator noise
        'reproduce_noise': False, # state if you want to use a predefined set of random noise for the given scenario. if so, provide path, e.g. 'noise_values.csv'
        'minimize_loading':False,
        # Clustering:
        'k_mean_clustering': False, # state if you want to perform a k-means clustering on the given network. State False or the value k (e.g. 20).
        'network_clustering': [VARIES, SEE TEXT BELOW], # state if you want to perform a clustering of HV buses to EHV buses.
        # Simplifications:
        'parallelisation':False, # state if you want to run snapshots parallely.
        'skip_snapshots':False,
        'line_grouping': False, # state if you want to group lines running between the same buses.
        'branch_capacity_factor': 0.7, # globally extend or lower branch capacities
        'load_shedding':False, # meet the demand at very high cost; for debugging purposes.
        'comments':None }

A. Network clustering only ('network_clustering' = True)

B. No clustering ('network_clustering' = False) As u can see, the loading in B ranges from -0.1..0.1%.

Questions:

1. Why is the loading in B this low compared to A?
2. How come loading is negative? (shouldn't this be an absolute value?!)

[nesnoj]

Uh, my bad! I missed this output:

...
INFO:keyring.backend:Loading windows
No data for StoragePqSet in column p_set.
Series p_set of component StorageUnit could not be imported
INFO:pypsa.pf:Slack bus for sub-network 0 is 28431
INFO:pypsa.opf:Performed preliminary steps
INFO:pypsa.opf:Building pyomo model using `angles` formulation
INFO:pypsa.opf:Solving model using gurobi
WARNING: Loading a SolverResults object with a warning status into
WARNING:pyomo.core:Loading a SolverResults object with a warning status into model=Linear Optimal Power Flow;
    model=Linear Optimal Power Flow;
        message from solver=Model was proven to be infeasible.
    message from solver=Model was proven to be infeasible.
ERROR:pypsa.opf:Optimisation failed with status warning and terminal condition infeasible
INFO:pypsa.io:Exported network results has generators, storage_units, buses, loads, carriers, lines, transformers
/home/jonathan/virtualenvs/openego/etrago/lib/python3.5/site-packages/numpy/core/_methods.py:29: RuntimeWarning: invalid value encountered in reduce
  return umr_minimum(a, axis, None, out, keepdims)
/home/jonathan/virtualenvs/openego/etrago/lib/python3.5/site-packages/numpy/core/_methods.py:26: RuntimeWarning: invalid value encountered in reduce
  return umr_maximum(a, axis, None, out, keepdims)
/home/jonathan/virtualenvs/openego/etrago/lib/python3.5/site-packages/matplotlib/colors.py:581: RuntimeWarning: invalid value encountered in less
  cbook._putmask(xa, xa < 0.0, -1)
0.6030786673227946
...

Model was proven to be infeasible - something with the input data? :disappointed:. Same with other timesteps, but with 'scenario' = 'SH NEP 2035' the results are far more plausible.. So it's something with the status quo?

[ClaraBuettner]

To avoid feasibility problems you need to activate load shedding. Without clustering, the solver is unable to satisfy some loads and creates the error. This causes to small line loadings. When you want to compare it to the clustered network you need to use the same generator noise. When not using generator noise, there are a lot of generators with the same costs, so the solver sometimes has to choose randomly the feeding generators. So it's not comparable to other calculations. When activating generator noise, every generator gets a small noise, so the costs are varying a little bit. Then the solver always finds the same solution and saves the noise in 'noise_values.csv'. To compare another calculation, you can choose the csv file in the argument reproduce noise.

I tested two timesteps with and without clustering and got nearly the same results.

[nesnoj]

Now the loading looks similar. But some lines which are supposed to be equal (no superposed 110kV) with and without network clustering have different loadings. E.g. the 3 lines to NL. Ok I guess this goes too far here.. @IlkaCu mentioned that this is subject of a current analysis..

Thanks for debugging! :+1: