Beat enrollment error 400 on 6.7

[matthenning]

Kibana version: 6.7

Elasticsearch version: 6.7

Server OS version: Debian GNU/Linux 9.8 (stretch)

Browser version: Firefox 60.5.1esr (32-Bit)

Browser OS version: Windows 10 1803

Original install method (e.g. download page, yum, from source, etc.): Office Debian repositories

Describe the bug: When enrolling a filebeat when trying to finish step two nothing happens. Checking the debug console revealed the following message:

[strict_dynamic_mapping_exception] mapping set to strict, dynamic introduction of [name] within [tag] is not allowed

The step looks like this:

The "Filebeat Input" configuration block:

Steps to reproduce:

1. Enroll a filebeat
2. Add the necessary configuration block
3. click Save & Continue
4. Check your browser's debug console

Expected behavior: The beat enrollment works

Provide logs and/or server output (if relevant):

2019-03-27T14:21:09.609215+01:00 redacted kibana[1116]: {"type":"response","@timestamp":"2019-03-27T13:21:09Z","tags":[],"pid":1116,"method":"put","statusCode":400,"req":{"url":"/api/beats/tag/427f0e5f-50d5-4a35-90f8-085e5cb72a6f","method":"put","headers":{"host":"redacted:5601","user-agent":"Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"application/json","accept-language":"de,en-US;q=0.7,en;q=0.3","accept-encoding":"gzip, deflate, br","referer":"https://redacted:5601/app/kibana","content-type":"application/json","credentials":"same-origin","kbn-version":"6.7.0","kbn-xsrf":"6.7.0","content-length":"34","connection":"keep-alive","pragma":"no-cache","cache-control":"no-cache"},"remoteAddress":"redacted","userAgent":"10.23.51.19","referer":"https://redacted:5601/app/kibana"},"res":{"statusCode":400,"responseTime":84,"contentLength":9},"message":"PUT /api/beats/tag/427f0e5f-50d5-4a35-90f8-085e5cb72a6f 400 84ms - 9.0B"}

Any additional context: This is a dev environment which was set up with version 6.6 last week and upgraded to 6.7 today.

[elasticmachine]

Pinging @elastic/beats

[jguay]

detailed reproduction steps using docker (ports 5601 and 9200 need to be free) testbeatenroll.zip 1- execute ./runAll.sh - this will start ES+kibana(+filebeat idle container) in 6.6.2 - ES data folder will be persisted 2- in kibana (http://localhost:5601), go to Management / Beats / Beats central management and just click "enroll" (no need to enroll any beat at this point) 3- Go to kibana dev tool and check with "GET .management-beats" that index was created, output contains the fields and:

    "mappings" : {
      "_doc" : {
        "dynamic" : "strict",
        "properties" : {
          "beat" : {
            "properties" : { # (...)
        "tags" : {
                "type" : "keyword"
              }

Note there is no property called "name" within property "tag" for mapping "_doc" 4- Edit .env file and replace all instances of "6.6.2" with "6.7.0" 5- Execute ".runAll.sh" (this will recreate all containers in 6.7.0) 6- Go back to kibana and and start enrolling a beat copy and modify the URL of kibana (inside the docker network hostname is kibana0 and not localhost)

docker exec -i -t --privileged -u root filebeat0 /bin/bash
filebeat enroll http://kibana0:5601 eyJhbG...

7- From kibana go to next step and type test for tag called "test" and configuration block with "Paths" with any text 8- When you try to save the response for the PUT request /api/beats/tag/03c9dd55-7289-4518-acd9-931fffcc7a7a is:

{"statusCode":400,"error":"Bad Request","message":"[strict_dynamic_mapping_exception] mapping set to strict, dynamic introduction of [name] within [tag] is not allowed"}

Workaround solution seem to be using https://github.com/elastic/migrate-management-beats

[matthenning]

Can confirm this workaround. Thank you.

[kvch]

This specific error is fixed in the latest release of the migration tool. However, the mapping still needs adjustments as a property of beat is missing from the mapping.