search

elastic / elasticsearch

Free and Open, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.51k stars 24.6k forks source link

Simplifying cleanup after code examples #51576

Open lcawl opened 4 years ago

lcawl commented 4 years ago

Currently the examples in the Elasticsearch documentation are tested using the functionality described here: https://github.com/elastic/elasticsearch/tree/master/docs#snippet-testing

When the X-Pack code was merged into the OSS code, it seems that cleanup for the documentation tests was not carried over from the x-pack/docs folder to the docs folder. There seem to be structures created during testing (system indices, etc) that sometimes don’t get cleaned up and then cause problems with subsequent tests. As a result, cleanup has been added on an as-needed basis. For example:

https://github.com/elastic/x-pack-elasticsearch/issues/2818 https://github.com/elastic/elasticsearch/pull/31450 https://github.com/elastic/elasticsearch/issues/43271 https://github.com/elastic/elasticsearch/pull/44123

Ideally the steps for cleaning up after documentation tests can be clarified or simplified so that it's easier to accomplish for new features and/or existing features (security, watcher) that are still lingering in the x-pack/docs due to testing issues.

elasticmachine commented 4 years ago

Pinging @elastic/es-docs (:Docs)

elasticmachine commented 4 years ago

Pinging @elastic/es-core-infra (:Core/Infra/Build)

mark-vieira commented 4 years ago

@lcawl this is labeled with :Core/Infra/Build but it's not clear to me if any build infrastructure changes really need to be made here. Rather this just describes some improvement to the docs tests themselves. I ask because this label is generally meant to indicate some issue or effort that would fall under the scope of responsibility of the build engineering team (i.e. me 😉).

Does this issue need intervention from our end, or is this just some backlog cleanup work that will primarily be handled by the docs team?

lcawl commented 4 years ago

@mark-vieira I think @nik9000 @polyfractal or @davidkyle are better able to answer that question than I am, since they were involved in migrating the cleanup for rollup and ML features. Anything we can do to make it simpler or faster for other features to get the cleanup in place would be great in my opinion. I have no idea what cleanup steps would be required for the security and watcher features, for example, if they finally moved from the x-pack/docs to the docs folders.

nik9000 commented 4 years ago

fall under the scope of responsibility of the build engineering team

This is part of the test framework and we decided a while back to use core/infra/build for that. If that isn't a thing we want to do any more maybe we should add another label?

mark-vieira commented 4 years ago

@nik9000 we can keep the label, just wanted to know if I need to keep this on my radar or not.

davidkyle commented 4 years ago

It was @colings86 who added the :core/infra/build label after this issue had already been tagged with :Docs. @colings86 is this something you've automated and regarding https://github.com/elastic/elasticsearch/issues/51576#issuecomment-580933042 is it necessary

colings86 commented 4 years ago

Fixing this will require test infrastructure that actually cleans up the cluster including the x-pack features after each test. This is going to need to be a combination of Gradle and Java infrastructure so it comes under the definition of the :Core/Infra/Build label

elasticsearchmachine commented 2 years ago

Pinging @elastic/es-delivery (Team:Delivery)