search

juju / python-libjuju

Python library for the Juju API
Apache License 2.0
59 stars 99 forks source link

KeyError when querying relations when CMRs are present #366

Closed gnuoy closed 4 years ago

gnuoy commented 4 years ago

When querying the relations of a unit when CMRs are present libjuju throws a KeyError exception:

#!/usr/bin/env python3

from juju import loop
from juju.model import Model

async def deploy():
    model = Model()
    await model.connect_current()
    model.applications['swift-storage-region2-zone1'].relations

if __name__ == '__main__':
    loop.run(deploy())
$ ./lib-juju-repre.py 
Traceback (most recent call last):
  File "./lib-juju-repre.py", line 12, in <module>
    loop.run(deploy())
  File "/home/ubuntu/work/bugs/1815879/.tox/func-smoke/lib/python3.6/site-packages/juju/loop.py", line 38, in run
    raise task.exception()
  File "./lib-juju-repre.py", line 9, in deploy
    model.applications['swift-storage-region2-zone1'].relations
  File "/home/ubuntu/work/bugs/1815879/.tox/func-smoke/lib/python3.6/site-packages/juju/application.py", line 58, in relations
    return [rel for rel in self.model.relations if rel.matches(self.name)]
  File "/home/ubuntu/work/bugs/1815879/.tox/func-smoke/lib/python3.6/site-packages/juju/application.py", line 58, in <listcomp>
    return [rel for rel in self.model.relations if rel.matches(self.name)]
  File "/home/ubuntu/work/bugs/1815879/.tox/func-smoke/lib/python3.6/site-packages/juju/relation.py", line 123, in matches
    if app_name == endpoint.application.name and \
  File "/home/ubuntu/work/bugs/1815879/.tox/func-smoke/lib/python3.6/site-packages/juju/relation.py", line 35, in application
    raise KeyError(app_name)
KeyError: 'remote-93461129d917440087d3a507516188c7'
$ juju status
Model              Controller         Cloud/Region             Version  SLA          Timestamp
zaza-7366e869882d  gnuoy-serverstack  serverstack/serverstack  2.7-rc1  unsupported  15:17:48Z

SAAS                         Status  Store              URL
keystone                     active  gnuoy-serverstack  admin/zaza-2e24097355ab.keystone-offer
swift-proxy-region1          active  gnuoy-serverstack  admin/zaza-2e24097355ab.swift-proxy-region1-offer
swift-storage-region1-zone1  active  gnuoy-serverstack  admin/zaza-2e24097355ab.swift-storage-region1-zone1-offer
swift-storage-region1-zone2  active  gnuoy-serverstack  admin/zaza-2e24097355ab.swift-storage-region1-zone2-offer
swift-storage-region1-zone3  active  gnuoy-serverstack  admin/zaza-2e24097355ab.swift-storage-region1-zone3-offer

App                          Version  Status  Scale  Charm          Store       Rev  OS      Notes
swift-proxy-region2          2.23.0   active      1  swift-proxy    jujucharms    7  ubuntu  
swift-storage-region2-zone1  2.23.0   active      1  swift-storage  jujucharms    3  ubuntu  
swift-storage-region2-zone2  2.23.0   active      1  swift-storage  jujucharms    3  ubuntu  
swift-storage-region2-zone3  2.23.0   active      1  swift-storage  jujucharms    3  ubuntu  

Unit                            Workload  Agent  Machine  Public address  Ports     Message
swift-proxy-region2/0*          active    idle   0        172.20.0.35     8080/tcp  Unit is ready
swift-storage-region2-zone1/0*  active    idle   1        172.20.0.13               Unit is ready
swift-storage-region2-zone2/0*  active    idle   2        172.20.0.34               Unit is ready
swift-storage-region2-zone3/0*  active    idle   3        172.20.0.26               Unit is ready

Machine  State    DNS          Inst id                               Series  AZ    Message
0        started  172.20.0.35  e20f26cc-70d1-4a68-a613-f13b4ce3a92b  bionic  nova  ACTIVE
1        started  172.20.0.13  d9ccde47-dc93-4e49-ac85-9506b3382644  bionic  nova  ACTIVE
2        started  172.20.0.34  3c1893ba-ecf9-4390-bff7-a5e34acb41fe  bionic  nova  ACTIVE
3        started  172.20.0.26  b3499aba-1e79-40ff-ba58-c471050b1b40  bionic  nova  ACTIVE

Offer                              Application                  Charm          Rev  Connected  Endpoint       Interface  Role
swift-storage-region2-zone1-offer  swift-storage-region2-zone1  swift-storage  3    1/1        swift-storage  swift      provider
swift-storage-region2-zone2-offer  swift-storage-region2-zone2  swift-storage  3    1/1        swift-storage  swift      provider
swift-storage-region2-zone3-offer  swift-storage-region2-zone3  swift-storage  3    1/1        swift-storage  swift      provider
gnuoy commented 4 years ago

Steps to reproduce.

The bundles used in the example are:

http://paste.ubuntu.com/p/3BY9YwFYXC/ (bionic-train-gr-r1.yaml) http://paste.ubuntu.com/p/QwK3XFwSwT/ (bionic-train-gr-r2.yaml)

juju add-model swift-gr-region1
juju add-model swift-gr-region2       
juju deploy -m swift-gr-region1 ./bionic-train-gr-r1.yaml
juju deploy -m swift-gr-region2 ./bionic-train-gr-r2.yaml

juju offer swift-gr-region1.swift-storage-region1-zone1:swift-storage
Application "swift-storage-region1-zone1" endpoints [swift-storage] available at "admin/swift-gr-region1.swift-storage-region1-zone1"

juju offer swift-gr-region1.swift-storage-region1-zone2:swift-storage
Application "swift-storage-region1-zone2" endpoints [swift-storage] available at "admin/swift-gr-region1.swift-storage-region1-zone2"

juju offer swift-gr-region1.swift-storage-region1-zone3:swift-storage
Application "swift-storage-region1-zone3" endpoints [swift-storage] available at "admin/swift-gr-region1.swift-storage-region1-zone3"

juju add-relation -m swift-gr-region2 swift-proxy-region2 admin/swift-gr-region1.swift-storage-region1-zone1
juju add-relation -m swift-gr-region2 swift-proxy-region2 admin/swift-gr-region1.swift-storage-region1-zone2
juju add-relation -m swift-gr-region2 swift-proxy-region2 admin/swift-gr-region1.swift-storage-region1-zone3

Once the CMRs have been added the bug appears:

./lib-juju-repre.py
Traceback (most recent call last):
  File "./lib-juju-repre.py", line 12, in <module>
    loop.run(deploy())
  File "/home/ubuntu/work/bugs/libjuju-issues-366/venv3/lib/python3.6/site-packages/juju/loop.py", line 38, in run
    raise task.exception()
  File "./lib-juju-repre.py", line 9, in deploy
    print(model.applications['swift-storage-region1-zone1'].relations)
  File "/home/ubuntu/work/bugs/libjuju-issues-366/venv3/lib/python3.6/site-packages/juju/application.py", line 58, in relations
    return [rel for rel in self.model.relations if rel.matches(self.name)]
  File "/home/ubuntu/work/bugs/libjuju-issues-366/venv3/lib/python3.6/site-packages/juju/application.py", line 58, in <listcomp>
    return [rel for rel in self.model.relations if rel.matches(self.name)]
  File "/home/ubuntu/work/bugs/libjuju-issues-366/venv3/lib/python3.6/site-packages/juju/relation.py", line 110, in matches
    if app_name == endpoint.application.name and \
  File "/home/ubuntu/work/bugs/libjuju-issues-366/venv3/lib/python3.6/site-packages/juju/relation.py", line 23, in application
    raise KeyError(app_name)
KeyError: 'remote-a24d164799004bc889a3ec925e44bee6'
SimonRichardson commented 4 years ago

@gnuoy is there any chance you can make available the bigger bundle, so I can try and replicate where the gap is, with my patch in #367?

gnuoy commented 4 years ago

Hi @SimonRichardson I'm using the deployment I mentioned earlier *1. This is what I see when I use your branch:

$ virtualenv -ppython3 venv                                             
Already using interpreter /usr/bin/python3                                                                  
Using base prefix '/usr'                                                                                    
New python executable in /home/ubuntu/cmr-test/venv/bin/python3                                             
Also creating executable in /home/ubuntu/cmr-test/venv/bin/python                                              
Installing setuptools, pkg_resources, pip, wheel...done.

$ source venv/bin/activate     

$ cat sr.txt
git+https://github.com/SimonRichardson/python-libjuju.git@key-error#egg=juju

$ pip install -r sr.txt       
...
$ cat ./lib-juju-repre.py
#!/usr/bin/env python3

from juju import loop
from juju.model import Model

async def deploy():
    model = Model()
    await model.connect('swift-gr-region2')
    print("Model2 swift-storage-region2-zone1 relations:")
    print(model.applications['swift-storage-region2-zone1'].relations)
    await model.disconnect()

    print()

    model = Model()
    await model.connect('swift-gr-region1')
    print("Model1 swift-storage-region1-zone1 relations:")
    print(model.applications['swift-storage-region1-zone1'].relations)
    await model.disconnect()

if __name__ == '__main__':
    loop.run(deploy())

$ ./lib-juju-repre.py
Model2 swift-storage-region2-zone1 relations:
[<Relation id=1 swift-proxy-region2:swift-storage swift-storage-region2-zone1:swift-storage>]

Model1 swift-storage-region1-zone1 relations:
Traceback (most recent call last):
  File "./lib-juju-repre.py", line 22, in <module>
    loop.run(deploy())
  File "/home/ubuntu/cmr-test/venv/lib/python3.6/site-packages/juju/loop.py", line 38, in run
    raise task.exception()
  File "./lib-juju-repre.py", line 18, in deploy
    print(model.applications['swift-storage-region1-zone1'].relations)
  File "/home/ubuntu/cmr-test/venv/lib/python3.6/site-packages/juju/application.py", line 58, in relations
    return [rel for rel in self.model.relations if rel.matches(self.name)]
  File "/home/ubuntu/cmr-test/venv/lib/python3.6/site-packages/juju/application.py", line 58, in <listcomp>
    return [rel for rel in self.model.relations if rel.matches(self.name)]
  File "/home/ubuntu/cmr-test/venv/lib/python3.6/site-packages/juju/relation.py", line 115, in matches
    if endpoint.application is None:
  File "/home/ubuntu/cmr-test/venv/lib/python3.6/site-packages/juju/relation.py", line 26, in application
    raise JujuEntityNotFoundError(app_name, ["application", "remoteApplication"])
juju.errors.JujuEntityNotFoundError: Entity not found: remote-412f14470cba4f2a8624e3f2ce164af6

*1 https://github.com/juju/python-libjuju/issues/366#issuecomment-555967400

SimonRichardson commented 4 years ago

@gnuoy this is amazing, thank-you, I'll try and replicate that this afternoon.