search

adriank / ObjectPath

The agile query language for semi-structured data
http://objectpath.org
MIT License
380 stars 93 forks source link

Potential hyphen issue? #46

Closed mirceaulinic closed 8 years ago

mirceaulinic commented 8 years ago

Hi,

I'm trying to parse the following dictionary:

>>> d
{'rpc-reply': {'data': {'groups': {'group': [{'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '172.17.17.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP1'}, {'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '192.168.0.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP2'}]}}}}

I am able to instantiate the Tree:

>>> from objectpath import Tree
>>> t = Tree(d)
>>> t
TreeObject()

But I notice that is unable to select:

>>> t.execute("$.rpc-reply")
>>> t.execute("$.data")

When selecting using the square brackets seems that no matter what is requested, returns the whole structure:

>>> t.execute("$['rpc-reply']")
{'rpc-reply': {'data': {'groups': {'group': [{'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '172.17.17.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP1'}, {'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '192.168.0.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP2'}]}}}}
>>> t.execute("$['anything']")
{'rpc-reply': {'data': {'groups': {'group': [{'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '172.17.17.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP1'}, {'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '192.168.0.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP2'}]}}}}

Loading the dict into a JSON file and executing form the console:

› objectpath -d groups.json
ObjectPath interactive shell
ctrl+c to exit, documentation at http://adriank.github.io/ObjectPath.

Loading JSON document from groups.json...INFO@35 All strings will be cut to 100 chatacters.
 done.
>>> $
START@43 Tree.execute

>>> $.rpc-reply
START@43 Tree.execute
PARSE STAGE
('-', ('.', ('(root)', 'rs'), ('name', 'rpc')), ('name', 'reply'))
START@56 executing node '('-', ('.', ('(root)', 'rs'), ('name', 'rpc')), ('name', 'reply'))'
START@56 executing node '('.', ('(root)', 'rs'), ('name', 'rpc'))'
START@56 executing node '('(root)', 'rs')'
DEBUG@260 . left is '{
    'rpc-reply': {u'data': {u'groups': {u'group': [{u'bgp': {u'instance': {u'instance-as': {u'four-byte-as': {u'as': ,
    ...
}'
START@56 executing node '('name', 'rpc')'
DEBUG@268 . right is 'rpc'
END@277 . returning 'None'
START@56 executing node '('name', 'reply')'
END@609 Tree.execute with: 'None'
null
>>> $['rpc-reply']
START@43 Tree.execute
PARSE STAGE
('(root)',)
START@56 executing node '('(root)',)'
END@609 Tree.execute with: '{
    'rpc-reply': {u'data': {u'groups': {u'group': [{u'bgp': {u'instance': {u'instance-as': {u'four-byte-as': {u'as': ,
    ...
}'
{"rpc-reply": {"data": {"groups": {"group": [
  {
    "bgp": {...},
    "group-name": "GROUP1"
  },
  {
    "bgp": {...},
    "group-name": "GROUP2"
  }
]}}}}
>>> $['anything']
START@43 Tree.execute
PARSE STAGE
('(root)',)
START@56 executing node '('(root)',)'
END@609 Tree.execute with: '{
    'rpc-reply': {u'data': {u'groups': {u'group': [{u'bgp': {u'instance': {u'instance-as': {u'four-byte-as': {u'as': ,
    ...
}'
{"rpc-reply": {"data": {"groups": {"group": [
  {
    "bgp": {...},
    "group-name": "GROUP2"
  },
  {
    "bgp": {...},
    "group-name": "GROUP2"
  }
]}}}}

Looks like $.rpc-reply returns None, while $['rpc-reply'] returns everything as above.

Is there anything I miss?

Thanks, Mircea

adriank commented 8 years ago

Hi,

Using ' should solve your problem:

t.execute("$.'rpc-reply'")

Also $['val'] is not proper expression. You always need to use $.['val']. The *.* is there because of some Python issues.

Greetings, Adrian Kalbarczyk

http://kalbarczyk.co

On Mon, Aug 15, 2016 at 1:09 AM, Mircea Ulinic notifications@github.com wrote:

Hi,

I'm trying to parse the following dictionary:

d {'rpc-reply': {'data': {'groups': {'group': [{'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '172.17.17.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP1'}, {'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '192.168.0.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP2'}]}}}}

I am able to instantiate the Tree:

from objectpath import Tree>>> t = Tree(d)>>> t TreeObject()

But I notice that is unable to select:

t.execute("$.rpc-reply")>>> t.execute("$.data")

When selecting using the square brackets seems that no matter what is requested, returns the whole structure:

t.execute("$['rpc-reply']") {'rpc-reply': {'data': {'groups': {'group': [{'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '172.17.17.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP1'}, {'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '192.168.0.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP2'}]}}}}>>> t.execute("$['anything']") {'rpc-reply': {'data': {'groups': {'group': [{'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '172.17.17.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP1'}, {'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '192.168.0.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP2'}]}}}}

Loading the dict into a JSON file and executing form the console:

› objectpath -d groups.json ObjectPath interactive shell ctrl+c to exit, documentation at http://adriank.github.io/ObjectPath.

Loading JSON document from groups.json...INFO@35 All strings will be cut to 100 chatacters. done.>>> $ START@43 Tree.execute

$.rpc-reply START@43 Tree.execute PARSE STAGE ('-', ('.', ('(root)', 'rs'), ('name', 'rpc')), ('name', 'reply')) START@56 executing node '('-', ('.', ('(root)', 'rs'), ('name', 'rpc')), ('name', 'reply'))' START@56 executing node '('.', ('(root)', 'rs'), ('name', 'rpc'))' START@56 executing node '('(root)', 'rs')' DEBUG@260 . left is '{ 'rpc-reply': {u'data': {u'groups': {u'group': [{u'bgp': {u'instance': {u'instance-as': {u'four-byte-as': {u'as': , ...}' START@56 executing node '('name', 'rpc')' DEBUG@268 . right is 'rpc' END@277 . returning 'None' START@56 executing node '('name', 'reply')' END@609 Tree.execute with: 'None' null>>> $['rpc-reply'] START@43 Tree.execute PARSE STAGE ('(root)',) START@56 executing node '('(root)',)' END@609 Tree.execute with: '{ 'rpc-reply': {u'data': {u'groups': {u'group': [{u'bgp': {u'instance': {u'instance-as': {u'four-byte-as': {u'as': , ...}' {"rpc-reply": {"data": {"groups": {"group": [ { "bgp": {...}, "group-name": "GROUP1" }, { "bgp": {...}, "group-name": "GROUP2" } ]}}}}>>> $['anything'] START@43 Tree.execute PARSE STAGE ('(root)',) START@56 executing node '('(root)',)' END@609 Tree.execute with: '{ 'rpc-reply': {u'data': {u'groups': {u'group': [{u'bgp': {u'instance': {u'instance-as': {u'four-byte-as': {u'as': , ...}' {"rpc-reply": {"data": {"groups": {"group": [ { "bgp": {...}, "group-name": "GROUP2" }, { "bgp": {...}, "group-name": "GROUP2" } ]}}}}

Looks like $.rpc-reply returns None, while $['rpc-reply'] returns everything as above.

Is there anything I miss? Otherwise I suspect the constructor might be sensible to keys containing hyphens.

Thanks, Mircea

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/adriank/ObjectPath/issues/46, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKycvimJE1tIrnPwIbt2hrLD996ZRWGks5qf6AzgaJpZM4JkAzd .

mirceaulinic commented 8 years ago

Thanks for replying @adriank!

I confirm that works:

>>> t.execute("$.'rpc-reply'")
{'data': {'groups': {'group': [{'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '172.17.17.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP1'}, {'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '192.168.0.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP2'}]}}}
>>> t.execute("$.'rpc-reply'.'data'")
{'groups': {'group': [{'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '172.17.17.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP1'}, {'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '192.168.0.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP2'}]}}

I've got one more question, sorry: is it possible to select certain keys using a regex? i.e.: something similar to the XPath matches (ref: https://www.w3.org/TR/xpath-functions/#func-matches).

Thanks, Mircea

adriank commented 8 years ago

Unfortunately no. It is on a list of possible enhancements, but you are the first to ask for it. Add issue and I'll see what can I do.

On Aug 15, 2016 11:20, "Mircea Ulinic" notifications@github.com wrote:

Thanks for replying @adriank https://github.com/adriank!

I confirm that works:

t.execute("$.'rpc-reply'") {'data': {'groups': {'group': [{'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '172.17.17.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP1'}, {'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '192.168.0.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP2'}]}}}>>> t.execute("$.'rpc-reply'.'data'") {'groups': {'group': [{'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '172.17.17.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP1'}, {'bgp': {'instance': {'instance-as': {'four-byte-as': {'as': '13335', 'bgp-running': '', 'default-vrf': {'global': {'global-afs': {'global-af': {'af-name': 'ipv4-unicast', 'enable': '', 'aggregate-addresses': {'aggregate-address': {'aggregate-prefix': '20', 'route-policy-name': 'anycast-attributes', 'aggregate-addr': '192.168.0.0'}}}}}}}, 'as': '0'}, 'instance-name': 'default'}}, 'group-name': 'GROUP2'}]}}

I've got one more question, sorry: is it possible to select certain keys using a regex? i.e.: something similar to the XPath matches (ref: https://www.w3.org/TR/xpath-functions/#func-matches).

Thanks, Mircea

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/adriank/ObjectPath/issues/46#issuecomment-239761395, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKycqwnyVhx0bCTiQIRipaVJNjts4p8ks5qgC9PgaJpZM4JkAzd .

mirceaulinic commented 8 years ago

Thanks @adriank!