Allow configurable "instance" label to allow node_name instead of FQDN

[MichaelThamm]

Enhancement Proposal

Allow the "instance" label to be configurable via juju config. Currently, we return the FQDN as the instance label for machine charms since:

While in grafana-agent-k8s we want the instance name to made up of the topology, for machine charms we simply want the fqdn, as for VMs it's a vital part of the fingerprint.

Likely related to: Inject a hostname label Note: This issue is more generic than the following reproduction steps.

Reproduce with:

Deploy cos-lite

juju add-model -c microk8s cos
juju switch microk8s:cos
juju deploy cos-lite --trust
juju offer prometheus:receive-remote-write

Deploy kafka

juju add-model -c lxd test-labels
juju switch lxd:test-labels
juju deploy kafka --channel 3/edge --trust
juju deploy grafana-agent --channel latest/edge --trust
juju deploy zookeeper --channel 3/edge --trust
juju relate grafana-agent kafka
juju relate kafka zookeeper
juju relate grafana-agent prometheus
juju relate grafana-agent:send-remote-write prometheus
juju consume microk8s:admin/cos.prometheus
juju run grafana/leader get-admin-password --model cos-model

juju ssh grafana-agent cat/etc/grafana-agent.yaml` Results in:

    scrape_configs:
    - job_name: kafka_0_default
      metrics_path: /metrics
      static_configs:
      - labels:
          juju_application: kafka
        targets:
        - localhost:9101

[sed-i]

Not sure if we need to make this customizable, or do something about the string literals sprinkled all over cos-agent. Example:

https://github.com/canonical/grafana-agent-operator/blob/09e390c558e2133651f21a6dd1b7978cc74e0f61/lib/charms/grafana_agent/v0/cos_agent.py#L262-L263

Also we should document better the difference between grafana-agent's own labels (subordinate) and the principal's labels.