[Discussion] Can this extension load as a object in Opensearch Plugins

[conggguan]

After register a python extension, can we use loadExtensions interface or other interface to load extension as a object in a plugin? Like this code. So that we can use any extension developed by python in our plugin.

    @Override
    public void loadExtensions(ExtensionLoader loader) {
        externalToolFactories = new HashMap<>();
         for (MLCommonsExtension extension : loader.loadExtensions(MLCommonsExtension.class)) {
            List<Tool.Factory<? extends Tool>> toolFactories = extension.getToolFactories();
            for (Tool.Factory<? extends Tool> toolFactory : toolFactories) {
                ToolAnnotation toolAnnotation = toolFactory.getClass().getDeclaringClass().getAnnotation(ToolAnnotation.class);
                if (toolAnnotation == null) {
                    throw new IllegalArgumentException(
                        "Missing ToolAnnotation for Tool " + toolFactory.getClass().getDeclaringClass().getSimpleName()
                    );
                }
                String annotationValue = toolAnnotation.value();
                externalToolFactories.put(annotationValue, toolFactory);
            }
        }
    }

[dblock]

Are you just trying to call Python code? What's the complete use-case?

[conggguan]

Are you just trying to call Python code? What's the complete use-case?

I am attempting to register Python SDK extensions within a Java plugin to invoke them. For instance, if I have 10 extensions, can the plugin recognize that there are 10 extensions, along with their names and descriptions, and subsequently invoke them?

[dblock]

Users install extensions today by making a REST call to _extensions/initialize, which lands in https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/java/org/opensearch/extensions/ExtensionsManager.java. That establishes a connection to the extension and exchanges some messages, then you can invoke it. The only difference with what you're asking is that you want the plugin to know where your extensions are? Have you tried invoking ExtensionsManager from your plugin? Does this help?

To invoke the extension it depends what the extension currently exposes for services within OpenSearch. In https://code.dblock.org/2023/09/29/writing-opensearch-plugins-and-extensions.html I have an example that adds a REST interface. You can make transport calls, too, like this.

[dbwiddis]

register Python SDK extensions within a Java plugin to invoke them

The current model is that extensions register themselves with OpenSearch itself. Each extension is identified by a unique ID which is used as part of its rest path.

You can use a REST client to directly call the extension's API with a direct call to the node the extension connected to. There's also a means to directly trigger transport actions, but it's slightly more complex.

Note that extensions are experimental and currently implemented only connected to a single node.