LineGeometries and WireData not outputted correctly through class_to_dataframe

[NicolasGensollen]

Hi @kdheepak,

I was writing some tests for DiTTo and I noticed something weird when calling class_to_dataframe on LineGeometries and WireData. It seems like the values I'm getting do not make any sense.

Here is the master file I'm using for the test:

Clear

New Circuit.test_circuit

New Wiredata.ACSR336 GMR=0.0255000 DIAM=0.7410000 RAC=0.3060000 NormAmps=530.0000 Runits=mi radunits=in gmrunits=ft
New Wiredata.ACSR1/0 GMR=0.0044600 DIAM=0.3980000 RAC=1.120000 NormAmps=230.0000 Runits=mi radunits=in gmrunits=ft

New Linegeometry.HC2_336_1neut_0Mess nconds=4 nphases=3
~ cond=1 Wire=acsr336 x=-1.2909 h=13.716 units=m
~ cond=2 Wire=acsr336 x=-0.1530096 h=4.1806368 units=ft
~ cond=3 Wire=acsr336 x=0.5737 h=13.716 units=m
~ cond=4 Wire= ACSR1/0 x=0 h=14.648 ! units=m ! neutral

New Line.Line1 Bus1=bus1.1.2.3 Bus2=bus2.1.2.3
~ Geometry= HC2_336_1neut_0Mess
~ Length=300 units=ft 

Set Voltagebases=[4.8,34.5,115.0]
Calcvoltagebases
Solve

And this is what I get:

In [1]: import opendssdirect as dss

In [2]: dss.Basic.Version()
Out[2]: u'Version Version 8.0 release 1 build 0\n (64-bit build); License Status: Open '

In [3]: dss.run_command("redirect Master.dss")
Out[3]: u''

In [4]: dss.utils.is_pandas_installed=False

In [5]: dss.utils.class_to_dataframe("linegeometry")
Out[5]:
{'linegeometry.hc2_336_1neut_0mess': {u'B0': u'',
  u'B1': u'',
  u'C0': u'',
  u'C1': u'',
  u'EarthModel': u'',
  u'Rg': [u'acsr336 acsr336 acsr336 ACSR1/0'],
  u'Switch': [u'acsr336 acsr336 acsr336 ACSR1/0'],
  u'Xg': u'',
  u'basefreq': u'',
  u'bus1': u'4',
  u'bus2': u'3',
  u'cmatrix': u'ACSR1/0',
  u'cncables': u'',
  u'emergamps': u'',
  u'enabled': u'',
  u'faultrate': u'',
  u'geometry': u'',
  u'length': u'ACSR1/0',
  u'like': u'',
  u'linecode': u'4',
  u'normamps': u'',
  u'pctperm': u'',
  u'phases': u'0',
  u'r0': u'0',
  u'r1': u'14.648',
  u'repair': u'',
  u'rho': u'',
  u'rmatrix': [u'acsr336 acsr336 acsr336 ACSR1/0'],
  u'spacing': u'',
  u'tscables': u'',
  u'units': u'',
  u'wires': u'',
  u'x0': u'0',
  u'x1': u'm',
  u'xmatrix': u'ACSR1/0'}}

In [7]: dss.utils.class_to_dataframe("wiredata")
 'wiredata.acsr336': {u'B0': u'',
  u'B1': u'',
  u'C0': u'',
  u'C1': u'0.3980000',
  u'EarthModel': u'',
  u'Rg': u'',
  u'Switch': u'',
  u'Xg': u'',
  u'basefreq': u'',
  u'bus1': u'-1',
  u'bus2': u'1.120000',
  u'cmatrix': u'',
  u'cncables': u'',
  u'emergamps': u'',
  u'enabled': u'',
  u'faultrate': u'',
  u'geometry': u'',
  u'length': u'0.0044600',
  u'like': u'',
  u'linecode': u'mi',
  u'normamps': u'',
  u'pctperm': u'',
  u'phases': u'ft',
  u'r0': u'230.0000',
  u'r1': u'-1',
  u'repair': u'',
  u'rho': u'',
  u'rmatrix': u'',
  u'spacing': u'',
  u'tscables': u'',
  u'units': u'',
  u'wires': u'',
  u'x0': u'-1',
  u'x1': u'in',
  u'xmatrix': u''}}

[kdheepak]

This is what I get with the unreleased version.

In [1]: import opendssdirect as dss

In [2]: dss.Basic.Version()
Out[2]: 'Version Unknown. (64-bit build); License Status: Open '

In [3]: dss.run_command("redirect master.dss")
Out[3]: ''

In [4]: dss.utils.is_pandas_installed = False

In [5]: dss.utils.class_to_dataframe("linegeometry")
/Users/$USER/GitRepos/OpenDSSDirect.py/opendssdirect/utils.py:106: UserWarning: Pandas is not installed. Please see documentation for how to install extra dependencies.
  warnings.warn("Pandas is not installed. Please see documentation for how to install extra dependencies.")
Out[5]:
{'linegeometry.hc2_336_1neut_0mess': {'bus1': '4',
  'bus2': '3',
  'linecode': '4',
  'length': 'ACSR1/0',
  'phases': '0',
  'r1': '14.648',
  'x1': 'm',
  'r0': '0',
  'x0': '0',
  'C1': '',
  'C0': '',
  'rmatrix': ['acsr336 acsr336 acsr336 ACSR1/0'],
  'xmatrix': 'ACSR1/0',
  'cmatrix': 'ACSR1/0',
  'Switch': ['acsr336 acsr336 acsr336 ACSR1/0'],
  'Rg': ['acsr336 acsr336 acsr336 ACSR1/0'],
  'Xg': '',
  'rho': '',
  'geometry': '',
  'units': '',
  'spacing': '',
  'wires': '',
  'EarthModel': '',
  'cncables': '',
  'tscables': '',
  'B1': '',
  'B0': '',
  'normamps': '',
  'emergamps': '',
  'faultrate': '',
  'pctperm': '',
  'repair': '',
  'basefreq': '',
  'enabled': '',
  'like': ''}}

In [6]: dss.utils.class_to_dataframe("wiredata")
/Users/$USER/GitRepos/OpenDSSDirect.py/opendssdirect/utils.py:106: UserWarning: Pandas is not installed. Please see documentation for how to install extra dependencies.
  warnings.warn("Pandas is not installed. Please see documentation for how to install extra dependencies.")
Out[6]:
{'wiredata.acsr336': {'bus1': '-1',
  'bus2': '1.120000',
  'linecode': 'mi',
  'length': '0.0044600',
  'phases': 'ft',
  'r1': '-1',
  'x1': 'in',
  'r0': '230.0000',
  'x0': '-1',
  'C1': '0.3980000',
  'C0': '',
  'rmatrix': '',
  'xmatrix': '',
  'cmatrix': '',
  'Switch': '',
  'Rg': '',
  'Xg': '',
  'rho': '',
  'geometry': '',
  'units': '',
  'spacing': '',
  'wires': '',
  'EarthModel': '',
  'cncables': '',
  'tscables': '',
  'B1': '',
  'B0': '',
  'normamps': '',
  'emergamps': '',
  'faultrate': '',
  'pctperm': '',
  'repair': '',
  'basefreq': '',
  'enabled': '',
  'like': ''},
 'wiredata.acsr1/0': {'bus1': '-1',
  'bus2': '1.120000',
  'linecode': 'mi',
  'length': '0.0044600',
  'phases': 'ft',
  'r1': '-1',
  'x1': 'in',
  'r0': '230.0000',
  'x0': '-1',
  'C1': '0.3980000',
  'C0': '',
  'rmatrix': '',
  'xmatrix': '',
  'cmatrix': '',
  'Switch': '',
  'Rg': '',
  'Xg': '',
  'rho': '',
  'geometry': '',
  'units': '',
  'spacing': '',
  'wires': '',
  'EarthModel': '',
  'cncables': '',
  'tscables': '',
  'B1': '',
  'B0': '',
  'normamps': '',
  'emergamps': '',
  'faultrate': '',
  'pctperm': '',
  'repair': '',
  'basefreq': '',
  'enabled': '',
  'like': ''}}

[NicolasGensollen]

The output looks a little bit different but doesn't make more sense, right?

[kdheepak]

Yup. I think the property from opendss only returns strings instead of the actual values.

On Wed, Aug 1, 2018, 11:49 AM Gensollen notifications@github.com wrote:

The output looks a little bit different but doesn't make more sense, right?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NREL/OpenDSSDirect.py/issues/19#issuecomment-409662531, or mute the thread https://github.com/notifications/unsubscribe-auth/ABuqgew7kKkDJH1PjG1BmfYUCO8MSi_cks5uMeozgaJpZM4Vq4Cl .

[PMeira]

I believe the output should be the same as shown in the GUI. As @kdheepak mentioned, it's all strings:

I pushed a small change (to PR from the dss_capi_cffi branch) to use the general Element interface instead of CktElement -- LineGeometry and WireData are not circuit elements, so it explains part of the weird output.

Here's the updated output:

# LineGeometry
# ------------
{'LineGeometry.hc2_336_1neut_0mess': {'cncable': 'ACSR1/0',
                                      'cncables': ['acsr336 acsr336 acsr336 '
                                                   'ACSR1/0'],
                                      'cond': '4',
                                      'emergamps': '0',
                                      'h': '14.648',
                                      'like': '',
                                      'nconds': '4',
                                      'normamps': '0',
                                      'nphases': '3',
                                      'reduce': '',
                                      'spacing': '',
                                      'tscable': 'ACSR1/0',
                                      'tscables': ['acsr336 acsr336 acsr336 '
                                                   'ACSR1/0'],
                                      'units': 'm',
                                      'wire': 'ACSR1/0',
                                      'wires': ['acsr336 acsr336 acsr336 '
                                                'ACSR1/0'],
                                      'x': '0'}}

# WireData
# ------------
{'WireData.acsr1/0': {'GMRac': '0.0044600',
                      'GMRunits': 'ft',
                      'Rac': '1.120000',
                      'Rdc': '-1',
                      'Runits': 'mi',
                      'diam': '0.3980000',
                      'emergamps': '-1',
                      'like': '',
                      'normamps': '230.0000',
                      'radius': '-1',
                      'radunits': 'in'},
 'WireData.acsr336': {'GMRac': '0.0044600',
                      'GMRunits': 'ft',
                      'Rac': '1.120000',
                      'Rdc': '-1',
                      'Runits': 'mi',
                      'diam': '0.3980000',
                      'emergamps': '-1',
                      'like': '',
                      'normamps': '230.0000',
                      'radius': '-1',
                      'radunits': 'in'}}

We could expose it better in the future. I plan to add a more direct, low-level interface to all data someday, I'd say next couple of months. If you guys need something specific, just let me know and I'll see if it's easy to add to the Pascal code.

[NicolasGensollen]

Thanks @PMeira ! This is working well now! (The fact that everything is returned as a string isn't a huge concern for me.)

[NicolasGensollen]

Hmmm, it seems like I might have closed this too fast. I think the WireData output is fine but some properties of the geometry should be list of values not single values. For example:

• "x" should be "[-1.2909 -0.1530096 0.5737 0]" and not just "0"
• same for "h"
• same for "units" since this can be redefined between wires

The GUI is a little disturbing since it is also only showing the values of the last defined wire...

[PMeira]

@NicolasGensollen Yep, that's true. It seems to be like that by design, most likely to the DSS scripting language restrictions. We would need to loop through each conductor to get the values, e.g.:

run_command('redirect issue19.dss')
obj = 'Linegeometry.HC2_336_1neut_0Mess'
nconds = int(run_command(f'? {obj}.nconds'))
x = []
for cond in range(nconds):
    run_command(f'{obj}.cond={cond}')
    x.append(float(run_command(f'? {obj}.x')))

print(obj, x)    # Linegeometry.HC2_336_1neut_0Mess [0.0, -1.2909, -0.1530096, 0.5737]

For a quick solution, we could add the looping as a special case in class_to_dataframe, so it doesn't affect the end user. @kdheepak What do you think about that?

It doesn't seem wise to change how the properties work in Pascal since they're in the main OpenDSS code (LineGeometry.pas#l535. It only loops for the properties wires, cncables and tscables. We could add looping for all properties here but it can break something else and code that already does the manual loop.

I think the easiest and safest approach would be to try to avoid the property mechanism altogether by providing the missing pieces in the OpenDSS API in dss_capi. I have to take care of other tasks for a few days, but adding most of what's missing shouldn't take too long. The bonus is that there would be no string conversions anymore.

[kdheepak]

I agree. I'm fine with the slower loops in Python for this use case.

[kdheepak]

I'm on my mobile, I'll take a closer look when I'm home.

[kdheepak]

See #23 for a potential solution.

I'm somewhat conflicted about this since it defines a certain behavior that users will be come to expect. Ideally, we'd clean all the properties and ensure that they are correct. All the functions in utils were meant for just quick data gathering, and no accuracy was guaranteed. They simply returned what OpenDSS returned. The previous behavior wasn't correct and the new behavior is definitely better in terms of accurate results, and I now want to make sure we do this for all properties this is required for. That'll likely take some time though.

If there are no further comments about this, I'll merge and create a new issue to list all the changes we need. I think https://github.com/PMeira/dss_capi/issues/11 will go a long way to solving this when it is done.

[NicolasGensollen]

Thanks for the awesome work guys! I think this is really close to be working, but I'm still having a small issue with the wireData (I know I said they were OK before, but it looks like I wasn't careful enough...).

On the same little example, using the latest OpenDSSdirect.py (v.0.3.2), I get the following:

>>> dss.utils.class_to_dataframe("wiredata")
{'wiredata.acsr1/0': {'GMRac': '0.0044600',
  'GMRunits': 'ft',
  'Rac': '1.120000',
  'Rdc': '-1',
  'Runits': 'mi',
  'diam': '0.3980000',
  'emergamps': '-1',
  'like': '',
  'normamps': '230.0000',
  'radius': '-1',
  'radunits': 'in'},
 'wiredata.acsr336': {'GMRac': '0.0044600',
  'GMRunits': 'ft',
  'Rac': '1.120000',
  'Rdc': '-1',
  'Runits': 'mi',
  'diam': '0.3980000',
  'emergamps': '-1',
  'like': '',
  'normamps': '230.0000',
  'radius': '-1',
  'radunits': 'in'}}

So the values make sense but are the same for both wire (looks like the values of the last defined wire in the master file).

Do you have the same issue?

[PMeira]

@NicolasGensollen Yes, I can confirm the issue. I'm investigating right now.

[kdheepak]

I would guess that it is a problem with dss.Circuit.SetActiveElement(element)

[PMeira]

28 should address it. It's just that Circuit.SetActiveElement() only works for circuit elements, even though we need to use Circuit.SetActiveClass() to select a non-circuit class. It's a little misleading.

We could create something like ActiveClass.SetActiveClass() for consistency, or maybe leave that for a future reorganization of the methods if there are other issues like this.

[kdheepak]

@PMeira Thanks for quick replies and fixes!

[kdheepak]

I agree with creating ActiveClass.SetActiveClass. This can also be solved by better documentation here. I'm working towards better notebooks that describe how to use OpenDSSDirect.py. I'll probably add a notebook just to explain all the oddities that one might encounter.

[NicolasGensollen]

Works great! Thanks for addressing that so quickly! 👍