calculate_PTDF() results in ValueError: column index exceeds matrix dimensions

[djinnome]

Hi @kdheepak

When I try to run calculate_PTDF(),

from psst.case.utils import calculate_PTDF
matpower = {}
case = 'case1888rte'
matpower[case] =   read_matpower('matpower/data/{}.m'.format(case)) 
ptdf = calculate_PTDF(matpower[case])
ptdf

I run into the following error message. I'm hoping this is a simple index renaming issue that can be easily resolved without having to muck in the guts of the (no longer maintained) pypower.

makePTDF: buses must be numbered consecutively
makeBdc: buses must be numbered consecutively in bus matrix
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-49-0967a3f02167> in <module>
      1 from psst.case.utils import calculate_PTDF
      2 matpower[case] =  read_matpower('matpower/data/{}.m'.format(case)) 
----> 3 ptdf = calculate_PTDF(matpower[case])
      4 ptdf

~/Projects/Proposals/DataModelConvergence_2019/PSST/psst/psst/case/utils.py in calculate_PTDF(case, precision, tolerance)
     58     bus = np.array(bus.reset_index())
     59     branch = np.array(branch)
---> 60     ptdf = makePTDF(case.baseMVA, bus, branch, )
     61     if precision is not None:
     62         ptdf = ptdf.round(precision)

~/.pyenv/versions/anaconda3-2019.10/lib/python3.7/site-packages/pypower/makePTDF.py in makePTDF(baseMVA, bus, branch, slack)
     55 
     56     ## compute PTDF for single slack_bus
---> 57     Bbus, Bf, _, _ = makeBdc(baseMVA, bus, branch)
     58     Bbus, Bf = Bbus.todense(), Bf.todense()
     59     H = zeros((nbr, nb))

~/.pyenv/versions/anaconda3-2019.10/lib/python3.7/site-packages/pypower/makeBdc.py in makeBdc(baseMVA, bus, branch)
     61     i = r_[range(nl), range(nl)]                   ## double set of row indices
     62     ## connection matrix
---> 63     Cft = sparse((r_[ones(nl), -ones(nl)], (i, r_[f, t])), (nl, nb))
     64 
     65     ## build Bf such that Bf * Va is the vector of real branch powers injected

~/.pyenv/versions/anaconda3-2019.10/lib/python3.7/site-packages/scipy/sparse/compressed.py in __init__(self, arg1, shape, dtype, copy)
     55                     # (data, ij) format
     56                     from .coo import coo_matrix
---> 57                     other = self.__class__(coo_matrix(arg1, shape=shape))
     58                     self._set_self(other)
     59                 elif len(arg1) == 3:

~/.pyenv/versions/anaconda3-2019.10/lib/python3.7/site-packages/scipy/sparse/coo.py in __init__(self, arg1, shape, dtype, copy)
    196             self.data = self.data.astype(dtype, copy=False)
    197 
--> 198         self._check()
    199 
    200     def reshape(self, *args, **kwargs):

~/.pyenv/versions/anaconda3-2019.10/lib/python3.7/site-packages/scipy/sparse/coo.py in _check(self)
    285                 raise ValueError('row index exceeds matrix dimensions')
    286             if self.col.max() >= self.shape[1]:
--> 287                 raise ValueError('column index exceeds matrix dimensions')
    288             if self.row.min() < 0:
    289                 raise ValueError('negative row index found')

ValueError: column index exceeds matrix dimensions

[kdheepak]

The case you are loading is part of matpower? I will try in matpower to see if it works first, and then check that the differences are with the pypower implementation. I'll try to get to this tomorrow or on Thursday.

On Mon, Dec 9, 2019, 5:42 PM Jeremy Zucker notifications@github.com wrote:

Hi @kdheepak https://github.com/kdheepak

When I try to run calculate_PTDF(),

from psst.case.utils import calculate_PTDF matpower = {} case = 'case1888rte' matpower[case] = read_matpower('matpower/data/{}.m'.format(case)) ptdf = calculate_PTDF(matpower[case]) ptdf

I run into the following error message. I'm hoping this is a simple index renaming issue that can be easily resolved without having to muck in the guts of the (no longer maintained) pypower.

makePTDF: buses must be numbered consecutively makeBdc: buses must be numbered consecutively in bus matrix---------------------------------------------------------------------------ValueError Traceback (most recent call last) in 1 from psst.case.utils import calculate_PTDF 2 matpower[case] = set_pmin_to_zero( read_matpower('matpower/data/{}.m'.format(case)) )----> 3 ptdf = calculate_PTDF(matpower[case]) 4 ptdf ~/Projects/Proposals/DataModelConvergence_2019/PSST/psst/psst/case/utils.py in calculate_PTDF(case, precision, tolerance) 58 bus = np.array(bus.reset_index()) 59 branch = np.array(branch)---> 60 ptdf = makePTDF(case.baseMVA, bus, branch, ) 61 if precision is not None: 62 ptdf = ptdf.round(precision) ~/.pyenv/versions/anaconda3-2019.10/lib/python3.7/site-packages/pypower/makePTDF.py in makePTDF(baseMVA, bus, branch, slack) 55 56 ## compute PTDF for single slackbus---> 57 Bbus, Bf, , = makeBdc(baseMVA, bus, branch) 58 Bbus, Bf = Bbus.todense(), Bf.todense() 59 H = zeros((nbr, nb)) ~/.pyenv/versions/anaconda3-2019.10/lib/python3.7/site-packages/pypower/makeBdc.py in makeBdc(baseMVA, bus, branch) 61 i = r[range(nl), range(nl)] ## double set of row indices 62 ## connection matrix---> 63 Cft = sparse((r[ones(nl), -ones(nl)], (i, r[f, t])), (nl, nb)) 64 65 ## build Bf such that Bf Va is the vector of real branch powers injected ~/.pyenv/versions/anaconda3-2019.10/lib/python3.7/site-packages/scipy/sparse/compressed.py in init(self, arg1, shape, dtype, copy) 55 # (data, ij) format 56 from .coo import coo_matrix---> 57 other = self.class(coo_matrix(arg1, shape=shape)) 58 self._set_self(other) 59 elif len(arg1) == 3: ~/.pyenv/versions/anaconda3-2019.10/lib/python3.7/site-packages/scipy/sparse/coo.py in init(self, arg1, shape, dtype, copy) 196 self.data = self.data.astype(dtype, copy=False) 197 --> 198 self._check() 199 200 def reshape(self, args, **kwargs): ~/.pyenv/versions/anaconda3-2019.10/lib/python3.7/site-packages/scipy/sparse/coo.py in _check(self) 285 raise ValueError('row index exceeds matrix dimensions') 286 if self.col.max() >= self.shape[1]:--> 287 raise ValueError('column index exceeds matrix dimensions') 288 if self.row.min() < 0: 289 raise ValueError('negative row index found') ValueError: column index exceeds matrix dimensions

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/power-system-simulation-toolbox/psst/issues/10?email_source=notifications&email_token=AAN2VAL4C3AB4UYIFGEIAADQX3QXRA5CNFSM4JYVMVG2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4H7J4OSQ, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAN2VANUFCKWDYCO4XVIZBLQX3QXRANCNFSM4JYVMVGQ .

[djinnome]

Yes, I am just reading in this matpower case: https://github.com/MATPOWER/matpower/blob/master/data/case1888rte.m

[djinnome]

Hey @kdheepak Any luck reproducing the issue? I am available this week to chat if you have some time.

[kdheepak]

I was able to generate a PTDF matrix from that test case, meaning the problem isn't with the data but with the Python implementation. I am on travel at the moment though and will have to get to look at this until January.

[djinnome]

Hey @kdheepak were you able to reproduce the problem I described above in for calculate_PTDF(read_matpower('matpower/data/case1888rte.m))? I see you have upgraded from 0.1.3 to 0.1.5, but I am still getting the same error.

The error looks like it may just be a simple naming issue, since pypower warns that it expects buses to be numbered consecutively, perhaps that is causing the column indexing problem?

[djinnome]

Hey @kdheepak when you said you were able to generate a PTDF matrix from case1888rte, did you do it in pypower or psst or matpower or some other software? Do you have an hints as to what is causing the problem in python?

[djinnome]

Actually, the fix was simple: When preparing data for pypower, you don't need to use 1-based indexing. 0-based indexing works just fine.

I generated a pull request here: https://github.com/power-system-simulation-toolbox/psst/pull/11