Functional test fixtures should be unzipped

[cjcenizal]

It's impossible to grep the contents of the fixtures because they're all gzipped. Ideally, these files would all be unzipped so we can grep their contents more easily.

Here's a Python script which will unzip them all:

import os
import sys
import gzip

# Copy this file into the root directory of all the files you want to rename and execute it:
# `python unzip_files.py`

directory = os.path.dirname(os.path.realpath(sys.argv[0]))

for subdir, dirs, files in os.walk(directory):
  for filename in files:
    if filename in files:
      if filename.find(".gz") > 0:
        subdirectoryPath = os.path.relpath(subdir, directory)
        filePath = os.path.join(subdirectoryPath, filename)
        newFilePath = filePath.replace(".gz", "")
        print filePath

        inF = gzip.GzipFile(filePath, 'rb')
        s = inF.read()
        inF.close()

        outF = file(newFilePath, 'wb')
        outF.write(s)
        outF.close()

[cjcenizal]

CC @nreese

[LeeDr]

I think the "test fixtures" @cjcenizal is referring to are the esArchives which can be either .kibana index docs or data docs like logstash.

I could see a case for wanting to search one of the .kibana archives for the name of a visualization or a dashboard. But it doesn't seem likely you'd want to grep for some logstash data?

the es_archiver CLI has an edit option which unzips and untars an archive, then waits for you to read or edit it, and then tars and zips it back up. Works pretty well. I don't think it currently has a way to do all of them in a folder. Do we need that? Or can we close this?

[nreese]

I agree with @cjcenizal that all esArchives files containing Kibana saved objects should be stored uncompressed. This provides 2 really important benefits: 1) They can be searched - supper useful when a test not in your area is failing your PR 2) They provide meaningful changes when updated in Pull Requests- again super useful to investigate changes to these files over time.

I can volunteer to close this and uncompress all data.json files for .kibana index.

[LeeDr]

We're just starting the process to switch from esArchiver to kbnArchiver for importing saved objects for tests. kbnArchiver uses the saved object API. We should put our efforts into that change. @wayneseymour is working on it if you would like to coordinate with him.

[wayneseymour]

Just a place holder for now: Use kbn-archiver instead of es-archiver https://github.com/elastic/kibana/issues/93797