Plugins PoC - Kotlin

[AlexRuiz7]

Description

To better understand how plugins work, their lifecylce, how they interact with core functionalities and how to develop them in general, we are going to develop a simple plugin for wazuh-indexer that exposes an API to manage ToDos, as a proof of concept. We'll use Kotlin as the programming language.

As a definition of done, we expect to have an API from which it's possible to:

• Create ToDos
• Read ToDos
• Update ToDos
• Delete ToDos

ToDo model

The ToDos will be stored using an index, for example: wazuh-todos. Each of the documents in the index could have these properties:

• id: uuid
• title: string
• description: text
• due_date: date

The index name and its fields must follow the naming convention.

• https://opensearch.org/docs/latest/im-plugin/#naming-restrictions-for-indexes
• https://www.elastic.co/guide/en/ecs/current/ecs-guidelines.html#_guidelines_for_field_names

Tasks

• [x] Create a plugin using the existing template.
• [x] Rename and set up the plugin accordingly.
• [ ] Create tests.
• [x] Create routes.
• [ ] Create route handles: request, action, response.
• [x] Create ToDo model.
• [ ] Create Index manager.

[AlexRuiz7]

Licenses

Gradle task :licenseHeaders requires every source code file to have a header, such as this one:

/*
 * SPDX-License-Identifier: Apache-2.0
 *
 * The OpenSearch Contributors require contributions made to
 * this file be licensed under the Apache-2.0 license or a
 * compatible open source license.
 */

Plugin development roadmap

Create routes

1. Create the resthandler package.
2. Create the PluginNameRestHandler class. Inherits from BaseRestHandler.
3. Override getName(), prepareRequest() and routes() methods.

[f-galland]

I'm currently trying to index data with the following method:

    public static void populateData(Client client) {
        client.index(
            new IndexRequest(HELLO_WORLD_INDEX_NAME).id("test_id")
                .setRefreshPolicy(WriteRequest.RefreshPolicy.IMMEDIATE)
                .source("{ \"field\": value }", XContentType.JSON)
        ).actionGet();
    }

But this seems to be blocking the whole of opensearch's engine:

Caused by: java.lang.AssertionError: Expected current thread [Thread[#63,opensearch[integTest-0][transport_worker][T#3],5,main]] to not be a transport thread. Reason: [Blocking operation]

[f-galland]

It seems there is an action class that already handles indexing data from a REST API call:

• https://github.com/opensearch-project/OpenSearch/blob/84ad3a7d2dc4410954b30d1c22d908047cfdd73b/server/src/main/java/org/opensearch/rest/action/document/RestIndexAction.java#L139

[AlexRuiz7]

Having this issue on client.index(indexRequest)

java.lang.AssertionError: Returned a 201 CREATED but not set up to support a Location header

[AlexRuiz7]

I finally managed to get the POST endpoint working without errors. The endpoint receives the Task data in JSON format and stores it in the task index.

curl -XPOST http://localhost:9200/_plugins/tasks -H "Content-Type:application/json" -d '{"document_id":112, "title": "A task", "description": "A simple task", "is_completed": false}'

[2024-06-11T19:50:39,584][INFO ][o.o.p.PluginsService     ] [integTest-0] PluginService:onIndexModule index:[tasks/rgf-_e9ARNCtoBSKg3gKFg]
[2024-06-11T19:50:39,601][INFO ][o.o.c.m.MetadataMappingService] [integTest-0] [tasks/rgf-_e9ARNCtoBSKg3gKFg] update_mapping [_doc]
[2024-06-11T19:50:39,605][DEBUG][o.o.c.c.PublicationTransportHandler] [integTest-0] received diff cluster state version [6] with uuid [7V7jT36DTN6nKAi5DF7zfA], diff size [485]
[2024-06-11T19:50:39,626][DEBUG][o.o.c.c.C.CoordinatorPublication] [integTest-0] publication ended successfully: Publication{term=1, version=6}
[2024-06-11T19:50:39,632][INFO ][o.o.t.i.TasksIndex       ] [integTest-0] Successfully persisted task status

As a result, http://localhost:9200/tasks/_search returns:

{
  "took": 211,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "max_score": 1,
    "hits": [
      {
        "_index": "tasks",
        "_id": "qLloCJABfI4fIpO-1tC7",
        "_score": 1,
        "_source": {
          "document_id": null,
          "title": "",
          "description": "",
          "is_completed": false
        }
      },
      {
        "_index": "tasks",
        "_id": "qblsCJABfI4fIpO-Y9DY",
        "_score": 1,
        "_source": {
          "document_id": 112,
          "title": "",
          "description": "",
          "is_completed": false
        }
      }
    ]
  }
}

I'll be implementing the rest of the endpoints, tests and redacting a documentation about how to properly implement a REST plugin.