Figure out how debug Kibana on the local stack created by elastic-package

[zmoog]

During Elastic Agent integration development workflow developers start a local stack using elastic-agent:

elastic-package build && elastic-package stack up -d -v --version 8.12.0-SNAPSHOT

I would love to replace the Kibana version created by the stack and replace it with a version started from source and use the debugger.

Overview

• Start the local stack
• Stop the Kibana container
• Start Kibana from sources
• Enjoy

[zmoog]

Start the local stack

• Clone the integration repo
• Walk into a package
• Create and start the local stack

Clone the integration repo

Walk into a package

Create and start the local stack

elastic-package build && elastic-package stack up -d -v --version 8.12.0-SNAPSHOT

[zmoog]

Stop the Kibana container

All we need to do is stop the Kibana container part of the local stack:

docker stop elastic-package-stack-kibana-1

[zmoog]

Start Kibana from sources

Here I'm building on top of the experience made with https://github.com/zmoog/public-notes/issues/59.

Certs

elastic-package creates a local stack that uses HTTPS using generated CA, certificates, and keys.

All the certificates are stored locally in the $HOME/.elastic-package/profiles/default/certs/ sub folders.

config/kibana.yml

server.name: kibana
server.host: "0.0.0.0"
server.ssl.enabled: true
server.ssl.certificate: "/Users/zmoog/.elastic-package/profiles/default/certs/kibana/cert.pem"
server.ssl.key: "/Users/zmoog/.elastic-package/profiles/default/certs/kibana/key.pem"
server.ssl.certificateAuthorities: ["/Users/zmoog/.elastic-package/profiles/default/certs/kibana/ca-cert.pem"]

elasticsearch.hosts: [ "https://127.0.0.1:9200" ]
elasticsearch.ssl.certificateAuthorities: "/Users/zmoog/.elastic-package/profiles/default/certs/kibana/ca-cert.pem"

elasticsearch.serviceAccountToken: "<TOKEN>"

monitoring.ui.container.elasticsearch.enabled: true

# xpack.fleet.agents.elasticsearch.hosts: ["https://127.0.0.1:9200"]

xpack.fleet.registryUrl: "https://127.0.0.1:8080"
xpack.fleet.agents.enabled: true
xpack.fleet.agents.fleet_server.hosts: ["https://127.0.0.1:8220"]

xpack.encryptedSavedObjects.encryptionKey: "12345678901234567890123456789012"

xpack.cloudSecurityPosture.enabled: true

xpack.fleet.packages:
  - name: system
    version: latest
  - name: elastic_agent
    version: latest
  - name: fleet_server
    version: latest
xpack.fleet.agentPolicies:
  - name: Elastic-Agent (elastic-package)
    id: elastic-agent-managed-ep
    is_default: true
    is_managed: false
    namespace: default
    monitoring_enabled:
      - logs
      - metrics
    package_policies:
      - name: system-1
        id: default-system
        package:
          name: system
  - name: Fleet Server (elastic-package)
    id: fleet-server-policy
    is_default_fleet_server: true
    is_managed: false
    namespace: default
    package_policies:
      - name: fleet_server-1
        id: default-fleet-server
        package:
          name: fleet_server
xpack.fleet.outputs:
  - id: fleet-default-output
    name: default
    type: elasticsearch
    hosts: [ https://127.0.0.1:9200 ]
    # ca_trusted_fingerprint: "${ELASTIC_PACKAGE_CA_TRUSTED_FINGERPRINT}"
    is_default: true
    is_default_monitoring: true

Set up extra CA certificates for nodejs

Setting the NODE_EXTRA_CA_CERTS environment variable allows Kibana to trust HTTPS certificates from package-registry and other services.

export NODE_EXTRA_CA_CERTS="$HOME/.elastic-package/profiles/default/certs/kibana/ca-cert.pem"

Prepare Kibana sources

Follow the step-by-step instruction at https://github.com/zmoog/public-notes/issues/59#issuecomment-1779026823

Start Kibana in debug mode

yarn debug

Open a browser

Open https://localhost:5601 with your favourite browser.

Note: on debug mode, Kibana uses a context like https://localhost:5601/wyh/ (wyh is an example, it seems change at every run.

[mgiota]

@zmoog Thanks for this really helpful!

Start the local stack

• Clone the integration repo
• Walk into a package
• Create and start the local stack

Clone the integration repo

Walk into a package

Create and start the local stack

elastic-package build && elastic-package stack up -d -v --version 8.12.0-SNAPSHOT

Not sure if the Clone the integration repo and Walk into a package sections were intended to have some contents or not, since it is pretty straightforward what needs to be done there. I did't want to directly edit your content, so if you agree for consistency reasons I suggest following:

Start the local stack

• Clone the forked integration repo
• Walk into a package

• Create and start the local stack

  Clone the forked integration repo

  git clone git@github.com:{YOUR_FORK}/integrations.git

  Walk into a package

  cd integrations/{PACKAGE}

  Create and start the local stack

  elastic-package build && elastic-package stack up -d -v --version 8.12.0-SNAPSHOT

I also made a change regarding cloning the fork and not directly the kibana integrations repo. I guess your team uses the Pull/Fork model. What do you think? Feel free to update your initial content.