--model seems broken on an LXD environment

[juditnovak]

When using the --model switch 1 on k8s, all goes fine. However on an LXD environment I'm getting the following error (both locally and on pipelines ):

Traceback (most recent call last):
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/_pytest/runner.py", line 341, in from_call
    result: Optional[TResult] = func()
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/_pytest/runner.py", line 262, in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/pluggy/_hooks.py", line [43](https://github.com/canonical/opensearch-operator/actions/runs/5745348380/job/15573293066#step:6:44)3, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/pluggy/_manager.py", line 112, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/pluggy/_callers.py", line 155, in _multicall
    return outcome.get_result()
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/pluggy/_result.py", line 108, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/pluggy/_callers.py", line 80, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/_pytest/runner.py", line 157, in pytest_runtest_setup
    item.session._setupstate.setup(item)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/_pytest/runner.py", line [49](https://github.com/canonical/opensearch-operator/actions/runs/5745348380/job/15573293066#step:6:50)7, in setup
    raise exc
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/_pytest/runner.py", line 494, in setup
    col.setup()
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/_pytest/python.py", line 1791, in setup
    self._request._fillfixtures()
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/_pytest/fixtures.py", line [56](https://github.com/canonical/opensearch-operator/actions/runs/5745348380/job/15573293066#step:6:57)6, in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/_pytest/fixtures.py", line [58](https://github.com/canonical/opensearch-operator/actions/runs/5745348380/job/15573293066#step:6:59)5, in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/_pytest/fixtures.py", line 607, in _get_active_fixturedef
    self._compute_fixture_value(fixturedef)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/_pytest/fixtures.py", line 693, in _compute_fixture_value
    fixturedef.execute(request=subrequest)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/_pytest/fixtures.py", line 1045, in execute
    fixturedef = request._get_active_fixturedef(argname)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/_pytest/fixtures.py", line 607, in _get_active_fixturedef
    self._compute_fixture_value(fixturedef)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/_pytest/fixtures.py", line 693, in _compute_fixture_value
    fixturedef.execute(request=subrequest)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/_pytest/fixtures.py", line 1069, in execute
    result = ihook.pytest_fixture_setup(fixturedef=self, request=request)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/pluggy/_hooks.py", line 433, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/pluggy/_manager.py", line 112, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/pluggy/_callers.py", line 155, in _multicall
    return outcome.get_result()
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/pluggy/_result.py", line 108, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/pluggy/_callers.py", line 80, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/_pytest/fixtures.py", line 1123, in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/_pytest/fixtures.py", line 902, in call_fixture_func
    fixture_result = fixturefunc(**kwargs)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/pytest_asyncio/plugin.py", line 304, in _asyncgen_fixture_wrapper
    result = event_loop.run_until_complete(setup())
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/pytest_asyncio/plugin.py", line 286, in setup
    res = await gen_obj.__anext__()
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/pytest_operator/plugin.py", line 231, in ops_test
    await ops_test._setup_model()
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/pytest_operator/plugin.py", line [71](https://github.com/canonical/opensearch-operator/actions/runs/5745348380/job/15573293066#step:6:72)9, in _setup_model
    model_state = await self._connect_to_model(
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/pytest_operator/plugin.py", line 678, in _connect_to_model
    await model.connect(state.full_name)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/juju/model.py", line 6[72](https://github.com/canonical/opensearch-operator/actions/runs/5745348380/job/15573293066#step:6:73), in connect
    _, model_uuid = await self._connector.connect_model(model_name, **kwargs)
  File "/home/runner/work/opensearch-operator/opensearch-operator/.tox/charm-integration/lib/python3.10/site-packages/juju/client/connector.py", line 161, in connect_model
    raise JujuConnectionError('Model not found: {}'.format(_model_name))
juju.errors.JujuConnectionError: Model not found: admin/testing
------------------------------ Captured log setup ------------------------------
INFO     pytest_operator.plugin:plugin.py:6[75](https://github.com/canonical/opensearch-operator/actions/runs/5745348380/job/15573293066#step:6:76) Connecting to existing model github-pr-86a89-lxd:testing on unspecified cloud

Enviroment:

• Juju: 3.1/stable
• ops==2.3.0
• pytest-operator==0.28.0

Are there any hitns or known workarounds available perhaps? Thank you.

[addyess]

github-pr-86a89-lxd:testing on unspecified cloud seems to hint you need to also specify the --cloud switch

[juditnovak]

Dear @addyess , thanks much for your response and the advice.

Unfortunately it's impossible to use both the --model and the --cloud switch together :-( And specifying the cloud still leaves me with the same problem -- I can't specify the model name :-(

[addyess]

perhaps the issue with using --cloud and --model simultaneously are that this block ignores the --cloud argument when connecting to an existing model. Note the None there. I bet this is something we could patch and try

[juditnovak]

I see your point, gave it a quick try locally. However no change, the error is the same :-(

[addyess]

@juditnovak do you have an example of this "all going fine" on k8s? My understanding of the switch --model is to use an existing model. I looked in your pipeline and i don't see that model already existing -- only that you've bootstrapped a lxd controller. Is it really important to use an existing model or is it ok for ops_test to create the model for you by omitting the --model switch entirely?

[juditnovak]

Yes, this pipeline: https://github.com/canonical/mongodb-k8s-operator/actions/runs/5623068946/job/15240177776?pr=178#step:8:1

I run the tests using the --model=testing parameter, which allowed me to switch to the model in the next step, and replay the debug-logs

[addyess]

nice, on this linked pipeline -- i see that after bootstrapping the controller, a model named testing is created

I've never seen this before -- but that's because the actions-operator github action randomly makes this model and that's why it works on k8s. Perhaps this issue should be linked to actions-operator rather than pytest-operator

https://github.com/charmed-kubernetes/actions-operator/blob/main/src/bootstrap/index.ts#L272-L280

[juditnovak]

I don't think so... As I can use the --model operator locally (precisely as demonstrated on the pipeline) on a k8s multipass instance without any issues. No Github action involved.

However, I can't do the same on an LXD multipass isntace... Failure is exactly as on the corresponding "failure" pipeline.

In fact, I don't think it's the github action that's suppsoed to create the model name, but the test framework (I could dig in for a more precise response)... As model name is generated exactly the same way locally as on the pipeline.

[addyess]

So, maybe this issue could be titled:

"Feature: reuse or create a model name if provided via the --model switch"

Unless there's some kind of magic hidden with pylibjuju's kubernetes client to create a model if it doesn't already exist during the model.connect phase, there's nothing in ops_test to currently do this. One should first create a juju model before trying to connect to it as an existing model.

The ops_test docs fail to mention that the model must first exist -- we could also improve those doc to further clarify the juju model must first exist otherwise a connection failure to that model will occur

[addyess]

https://github.com/charmed-kubernetes/pytest-operator/pull/113

[juditnovak]

Thank you very much :-) :-) :-)

In k8s the model is sure created if it didn't exist. The issue only appears when using LXD.

[addyess]

to say it another way -- in kubernetes something special is going on -- but in lxd or any machine based model (openstack, aws, vsphere, gce, azure....) or metal (maas, equinox) -- the model must first be there before we can connect to it.

[juditnovak]

All right! Thanks very much, understood :-)

[addyess]

Resolved with 0.29.0 release