Figure out how to debug functional tests in Kibana

[zmoog]

Kibana has a suite of functional tests that the CI runs with all PRs, and we can run them locally as well.

If a test fails due to a change, I would love to attach a debugger to the local Kibana instance running the functional tests to troubleshoot and solve the problem.

[zmoog]

Requirements

• Clone the repo and bootstrap Kibana for development

You need to clone the Kibana repo and bootstrap it for development. You can follow the steps at https://github.com/zmoog/public-notes/issues/59#issuecomment-1779026823. You may need to use a different nodejs version, depending on the Kibana version you plan to run

[zmoog]

Functional Tests Overview

Kibana offers several options to run functional tests.

I will use the "best for development" option 2 for this research.

Option 2: functional test server and runner

With this option, we will:

1. Start a functional test server and leave it running for the whole test session.
2. Start a functional test runner that will execute the tests multiple times against the test server

The functional test server spawns an Elasticsearch and Kibana instance.

We aim to start the Kibana instance in the test server in debug mode so we can attach a debugger.

[zmoog]

Start a functional test server in debug mode

I have the feeling we can simplify this step, but as of now, we can start the Kibana instance in debug mode, making this small change:

Enable inspect

diff --git a/packages/kbn-test/src/functional_tests/start_servers/start_servers.ts b/packages/kbn-test/src/functional_tests/start_servers/start_servers.ts
index f16b24f59bc..859c29ab5ef 100644
--- a/packages/kbn-test/src/functional_tests/start_servers/start_servers.ts
+++ b/packages/kbn-test/src/functional_tests/start_servers/start_servers.ts
@@ -50,6 +50,7 @@ export async function startServers(log: ToolingLog, options: StartServerOptions)
               ? '--server.versioned.versionResolution=newest'
               : '--server.versioned.versionResolution=oldest',
           ],
+      inspect: true
     });

     reportTime(runStartTime, 'ready', {

Run the functional test server

node \
  scripts/functional_tests_server \
  --config x-pack/test/fleet_api_integration/config.package_policy.ts 

After a while, the functional test server will be ready:

Functional test server troubleshooting

Sometimes, the server fails to start complaining about problems starting the ML in Elasticsearch. I still don't get why, but here is how you can disable ML in ES by setting xpack.ml.enabled=false, if required:

diff --git a/x-pack/test/api_integration/config.ts b/x-pack/test/api_integration/config.ts
index e43c76d42ad..34d83c08ab5 100644
--- a/x-pack/test/api_integration/config.ts
+++ b/x-pack/test/api_integration/config.ts
@@ -37,6 +37,7 @@ export async function getApiIntegrationConfig({ readConfigFile }: FtrConfigProvi
       ...xPackFunctionalTestsConfig.get('esTestCluster'),
       serverArgs: [
         ...xPackFunctionalTestsConfig.get('esTestCluster.serverArgs'),
+        'xpack.ml.enabled=false',
         'node.attr.name=apiIntegrationTestNode',
         'path.repo=/tmp/repo,/tmp/repo_1,/tmp/repo_2,/tmp/cloud-snapshots/',
       ],

[zmoog]

Run the functional test runner

node \
  scripts/functional_test_runner \
  --config=x-pack/test/fleet_api_integration/config.package_policy.ts

[zmoog]

Attach the debugger

Launch configurations

You can create a launch configurations:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Kibana Server (9230)",
            "port": 9230,
            "request": "attach",
            "restart": true,
            "skipFiles": [
                "<node_internals>/**"
            ],
            "type": "node"
        }
    ]
}

Attach!

https://github.com/zmoog/public-notes/assets/25941/2c534eaf-fc03-4205-9ac1-581cf7190603

[zmoog]

Run the functional test runner

Set a breakpoint

Pick the piece of Kibana code you want to troubleshoot and set a breakpoint

Run!

On a different shell, run the functional test runner.

node \
  scripts/functional_test_runner \
  --config=x-pack/test/fleet_api_integration/config.package_policy.ts

When the test will cause the invocation of the code with the breakpoint: