This updates Kibana to 7.9.3, and coordinates with cloudfoundry-community/logsearch-boshrelease#186

breaking changes/upgrade complications

this change requires manual deployment steps I'm not sure where to add this, but updating from 6.x.x to 7.x.x requires exporting and re-importing any objects (e.g. visualizations) you have.

This change also introduces new, required properties:

 elasticsearch_config.base_index_component_name:
    description: Name component-index-mappings.json is uploaded as.
  elasticsearch_config.app_index_component_name:
    description: Name component-index-mappings-app.json is uploaded as.
  elasticsearch_config.platform_index_component_name:
    description: Name component-index-mappings-platform.json is uploaded as.
  elasticsearch_config.index_mappings_component_name:
    description: Name index-mappings component from logsearch-boshrelease is uploaded as
  elasticsearch_config.index_settings_component_name:
    description: Name index-settings component from logsearch-boshrelease is uploaded as
  elasticsearch_config.shards_and_replicas_component_name:
    description: Name shards-and-replicas component from logsearch-boshrelease is uploaded as

overview

There are two big complications here:

The upgrade to 7.x.x introduced changes to index patterns. Kibana added data streams and data stream templates, including defaults that conflicted with the existing templates. We had to switch from index templates to component templates. Because of this, kibana now needs to know the name of the templates used in the logsearch-boshrelease, causing the new properties.
Kibana changed endpoints. This required new routes in the auth plugin.

I think it's worth noting that this set of changes reflects a fairly minimal changeset to get our system working on the new versions. I'm not at all familiar with how others in the community are using this release, so I wouldn't be surprised if there's some back-and-forth to get this to a place that works for the whole community :)

@bengerman13 Thanks for this PR :) I was working on upgrade logsearch to the newest ELK stack version but I got stuck on 7.9.3 because of #365. Could you please provide add-blobs.sh as a part of this repo? I want to test this PR but I have no access to the cloud-gov-release-blobstore:

Getting blob from inner blobstore:
  AccessDenied: Access Denied
  status code: 403, request id: 6Q3HJSDVKVVYXAT4, host id: nkAXNMAfio7VFlz9ybDkRIqNI5fi7Ts4cY64gKVpd7ohmE2ecVMxKjJ4ZlV5iFqD83vJo1ozE2U=

Something like:

#!/bin/sh

DIR=`pwd`
mkdir -p .downloads
cd .downloads

KI_VERSION=7.9.3-linux-x86_64
if [ ! -f ${DIR}/blobs/cf-kibana/kibana-${KI_VERSION}.tar.gz ];then
    curl -L -J -o kibana-${KI_VERSION}.tar.gz https://artifacts.elastic.co/downloads/kibana/kibana-${KI_VERSION}.tar.gz
    bosh add-blob --dir=${DIR} kibana-${KI_VERSION}.tar.gz cf-kibana/kibana-${KI_VERSION}.tar.gz
fi

#....

cd -

cloudfoundry-community / logsearch-for-cloudfoundry

Update to 7.9.3 #363

breaking changes/upgrade complications

overview