Dashboards and Visualizations don't work in a fresh install of 211.0.1

[mymasse]

Did a fresh deployment of 211.0.1 and most (if not all) dashboards and visualizations don't work and return errors:

[esaggs] > "field" is a required parameter

and

Saved "field" parameter is now invalid. Please select a new field.

Digging a little bit it looks like the removal of the index mappings caused this. When checking the mappings I have as an example these two:

• @cf.app of type string with searchable true
• @cf.app.keywork of type string with searchable true and aggregatable true

This is like that for all the string fields. Prior to 211.0.1 the included mappings didn't include any of the .keyword fields. Since the visualizations still reference @cf.app for example they don't work, but do if @cf.app.keyword is used.

[mymasse]

I think at a minimum we need to have the index-mapping that was specified in logsearch-boshrelease/jobs/elasticsearch_config/templates/index-templates/index-mappings.json.erb, more specifically the dynamic_templates

[mymasse]

Could also maybe added in the index-settings.json instead?

[mymasse]

Temporarly added and index template in Kibana with just this dynamic mapping and that solved most issues:

{
  "string_fields": {
    "match": "*",
    "match_mapping_type": "string",
    "mapping": {
      "type": "keyword",
      "index": true,
      "omit_norms": true
     }
   }
}

I still get a problem with the geoip.location field not being present

[mymasse]

And now adding this mapping fixes the location.

"geoip": {
  "type": "object",
  "dynamic": true,
  "properties": {
    "location": {
      "type": "geo_point"
    }
  }
}

So basicaly what was in the dopped index-mapping

[axelaris]

Nice job! Would you submit a PR?

[mymasse]

Do you want me to put the whole index-mapping back? or just with these settings? I would be tempted to put everything back but I have no way in testing if this works. My deployment is on a private network so I can't easily test this.

[TheMoves]

Ah, was also affected by this! Glad i stumbled across this