[ML] An HTTP line is larger than 4096 bytes error on ML Overview page when there are too many anomaly jobs

[sw-jung]

Kibana version: 8.14.3

Elasticsearch version: 8.14.3

Server OS version: Elastic Cloud

Browser version: Microsoft Edge 127.0.2651.98

Browser OS version: Windows 11 Pro 23H2 Build 22631.4037

Original install method (e.g. download page, yum, from source, etc.): Deployed on Elastic Cloud

Describe the bug: In case too many anomaly jobs in the group, the overall score column is not displayed and the user gets "An HTTP line is larger than 4096 bytes" error.

Steps to reproduce:

1. Add anomaly jobs with the same group. Depending on the length of job IDs, but I could reproduce this with more than 80+ jobs.
2. Go to the ML Overview page

Expected behavior: ML UI will work normally regardless of the number of jobs.

Screenshots (if relevant):

Any additional context: The culprit was the below request (extracted from HAR), which is for fetching "Overall score" column.

https://<my_kibana_uri>/internal/ml/anomaly_detectors/clone__clone__long_long_very_long_job_id_to_reproduce_issue___1,clone__clone__long_long_very_long_job_id_to_reproduce_issue___2,...(total 4046 chars)/results/overall_buckets

# Response
{
  "statusCode": 400,
  "error": "Bad Request",
  "message": "[too_long_http_line_exception\n\tRoot causes:\n\t\ttoo_long_http_line_exception: An HTTP line is larger than 4096 bytes.]: An HTTP line is larger than 4096 bytes.",
  "attributes": {
    "body": {
      "error": {
        "root_cause": [
          {
            "type": "too_long_http_line_exception",
            "reason": "An HTTP line is larger than 4096 bytes."
          }
        ],
        "type": "too_long_http_line_exception",
        "reason": "An HTTP line is larger than 4096 bytes."
      },
      "status": 400
    }
  }
}

I think the possible update is modifying this API to envelope the job IDs in the request body, or split the request into multiples by the total length.

[elasticmachine]

Pinging @elastic/ml-ui (:ml)