Only one role allowed

[camille-rodriguez]

Hi,

I need to deploy more than one role for an application. The manifest I am trying to implement in the charm is this one https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/metallb.yaml, where you can find the role "config-watcher", "pod-lister", in addition to the ClusterRoles for the controller.

I attempted at doing so in the pod_spec with this :

        self.framework.model.pod.set_spec(
            {
                'version': 3,
                'serviceAccount': {
                    'roles' :  [{
                        'global': True,
                        'rules': [
                            {
                                'apiGroups': [''],
                                'resources': ['services'],
                                'verbs': ['get', 'list', 'watch', 'update'],
                            },
                            {
                                'apiGroups': [''],
                                'resources': ['services/status'],
                                'verbs': ['update'],
                            },
                            {
                                'apiGroups': [''],
                                'resources': ['events'],
                                'verbs': ['create', 'patch'],
                            },
                            {
                                'apiGroups': ['policy'],
                                'resourceNames': ['controller'],
                                'resources': ['podsecuritypolicies'],
                                'verbs': ['use'],
                            },
                        ],
                    },
                    {
                        'rules': [
                            {
                                'apiGroups': [''],
                                'resources': ['configmaps'],
                                'verbs': ['get', 'list', 'watch'],
                            },
                        ]
                    }
                  ],
                },
                'containers': [{
                    'name': 'metallb',
                    'image': 'metallb/controller:v0.9.3',
                    'ports': [{
                        'containerPort': advertised_port,
                        'protocol': 'TCP'
                    }],
                    'kubernetes': {
                        'readinessProbe': {
                            'httpGet': {
                                'path': '/api/health',
                                'port': advertised_port
                            },
                            'initialDelaySeconds': 10,
                            'timeoutSeconds': 30
                        }
                    }   
                }]
            },
        )

But turns out that it is impossible to set more than one role in a pod_spec. See traceback below.

application-metallb-controller: 12:53:38 ERROR unit.metallb-controller/9.juju-log Uncaught exception while in charm code:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-metallb-controller-9/charm/venv/ops/model.py", line 977, in _run
    result = run(args, check=True, **kwargs)
  File "/usr/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('pod-spec-set', '--file', '/tmp/tmpf_2c8xjv-pod-spec-set/spec.json')' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./src/charm.py", line 207, in <module>
    main(MetallbCharm)
  File "/var/lib/juju/agents/unit-metallb-controller-9/charm/venv/ops/main.py", line 347, in main
    _emit_charm_event(charm, dispatcher.event_name)
  File "/var/lib/juju/agents/unit-metallb-controller-9/charm/venv/ops/main.py", line 123, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-metallb-controller-9/charm/venv/ops/framework.py", line 212, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-metallb-controller-9/charm/venv/ops/framework.py", line 624, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-metallb-controller-9/charm/venv/ops/framework.py", line 667, in _reemit
    custom_handler(event)
  File "./src/charm.py", line 65, in on_start
    self.framework.model.pod.set_spec(
  File "/var/lib/juju/agents/unit-metallb-controller-9/charm/venv/ops/model.py", line 856, in set_spec
    self._backend.pod_spec_set(spec, k8s_resources)
  File "/var/lib/juju/agents/unit-metallb-controller-9/charm/venv/ops/model.py", line 1080, in pod_spec_set
    self._run('pod-spec-set', *args)
  File "/var/lib/juju/agents/unit-metallb-controller-9/charm/venv/ops/model.py", line 979, in _run
    raise ModelError(e.stderr)
ops.model.ModelError: b'ERROR the prime service can only have one role or cluster role\n'
application-metallb-controller: 12:53:38 ERROR juju.worker.uniter.operation hook "start" (via hook dispatching script: dispatch) failed: exit status 1

Is there a strategy to be able to do this with the framework? Or should I bypass the pod_spec and use the k8s API instead?

[johnsca]

This is a Juju issue rather than a framework one (the message from the ModelError is coming from the pod-spec-set hook tool) and should probably be moved there.

[camille-rodriguez]

True, I've posted it here https://bugs.launchpad.net/juju/+bug/1896076 and will close this bug.