search

Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.4k stars 1.06k forks source link

Delete index via API not scaling with huge amount of index sets #18563

Open HenryTheSir opened 7 months ago

HenryTheSir commented 7 months ago

Expected Behavior

Fast deleting of indices via Graylog REST API independent of index set amount

Current Behavior

Deleting of indices gets slower as more index sets the cluster has. The reason can be found here, as graylog iterates over all index sets if an index gets deleted by api: https://github.com/Graylog2/graylog2-server/blob/5.2/graylog2-server/src/main/java/org/graylog2/indexer/MongoIndexSetRegistry.java#L234

This Method iterates over all index Sets which are currently in the cluster to check if the current write index equals the index which should be deleted. In our env this are over 500 Calls to Opensearch (/_alias/_deflector) As this calls are made one after another the response time for the delete calls stack ups to ~5-10 Minutes.

Possible Solution

A much nicer approach would be to test //_alias if this response contains any alias ending with _deflector
(All methods in this Class seam not to scale very well with rising index sets, we observed problems with deleting sofar. )

Steps to Reproduce (for bugs)

  1. Create high amound index sets
  2. try to delete an index in graylog via rest api
  3. wait very long

Context

There are various reasons why one would like to delete an index via graylog webui or by rest api.

Your Environment

dennisoelkers commented 7 months ago

Hey @HenryTheSir!

Thank you very much for reporting this and providing very helpful details!