charm tracing fails over tls

[michaeldmitry]

Bug Description

When trying to integrate grafana-agent with tempo to send its charm traces over tls, it fails to do with an exception "Exception while exporting Span batch."

To Reproduce

juju deploy grafana-agent-k8s grafana-agent --channel=latest/edge juju deploy tempo-k8s tempo --channel latest/edge juju deploy self-signed-certificates juju integrate self-signed-certificates:certificates grafana-agent:certificates juju integrate self-signed-certificates:certificates tempo:certificates juju integrate tempo:tracing grafana-agent:tracing wait for an event on grafana-agent (e.g: update-status) look for error in juju logs

Environment

juju version: 3.4.2-genericlinux-amd64 microk8s version: MicroK8s v1.28.7 revision 6532 grafana-agent rev: 76

Relevant log output

unit-grafana-agent-0: 13:15:18 ERROR unit.grafana-agent/0.juju-log tracing:54: Exception while exporting Span batch.
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-grafana-agent-0/charm/venv/opentelemetry/sdk/trace/export/__init__.py", line 367, in _export_batch
    self.span_exporter.export(self.spans_list[:idx])  # type: ignore
  File "/var/lib/juju/agents/unit-grafana-agent-0/charm/venv/opentelemetry/exporter/otlp/proto/http/trace_exporter/__init__.py", line 145, in export
    resp = self._export(serialized_data)
  File "/var/lib/juju/agents/unit-grafana-agent-0/charm/venv/opentelemetry/exporter/otlp/proto/http/trace_exporter/__init__.py", line 114, in _export
    return self._session.post(
  File "/var/lib/juju/agents/unit-grafana-agent-0/charm/venv/requests/sessions.py", line 637, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
  File "/var/lib/juju/agents/unit-grafana-agent-0/charm/venv/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/var/lib/juju/agents/unit-grafana-agent-0/charm/venv/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/var/lib/juju/agents/unit-grafana-agent-0/charm/venv/requests/adapters.py", line 458, in send
    self.cert_verify(conn, request.url, verify, cert)
  File "/var/lib/juju/agents/unit-grafana-agent-0/charm/venv/requests/adapters.py", line 261, in cert_verify
    raise OSError(
OSError: Could not find a suitable TLS CA certificate bundle, invalid path: /tmp/agent/grafana-agent.pem

Additional context

No response