We need a way to develop, test, demo, version and release features when they are in a preview or pre-release state independent from the stable core sdk. New features should only enter the sdk core when rapid refinement and iteration has tapered off and we're ready to stabilize the feature. This will help us innovate and ship faster.
Solution
We have implemented a plugin interface that allows us to extend the Pinecone class with experimental features defined elsewhere. For now, this plugin interface is only intended for internal use at Pinecone.
There are three pieces involved here: the changes in this diff, a small separate package called pinecone_plugin_interface, and the plugin implementations themself. Currently, there are no publicly available plugins.
The Pinecone class here imports a load_and_install function from the pinecone_plugin_interface package. When invoked, this function scans the user's environment looking for packages that have been implemented within a namespace package called pinecone_plugins. If it finds any, it will use the client_builder function supplied by the Pinecone class to pass user configuration into the plugin for consistent handling of configurations such as api keys, proxy settings, etc.
Although the amount of code involved in pinecone_plugin_interface is quite meager, externalizing it should give some flexibility to share code between the sdk and plugin implementations without creating a circular dependency should we ever wish to include a plugin as a dependency of the sdk (after functionality has stabilized).
I have been defensive here by wrapping the plugin installation in a try/catch. I want to be confident that the mere presence of an invalid or broken plugin cannot break the rest of the sdk.
Type of Change
[x] New feature (non-breaking change which adds functionality)
Test Plan
Tests should be green. I added some unit tests covering the new builder function stuff. As far as integration testing, I can't really add those here until we have publicly available plugins. But I've done manual testing to gain confidence.
Problem
We need a way to develop, test, demo, version and release features when they are in a preview or pre-release state independent from the stable core sdk. New features should only enter the sdk core when rapid refinement and iteration has tapered off and we're ready to stabilize the feature. This will help us innovate and ship faster.
Solution
We have implemented a plugin interface that allows us to extend the
Pineconeclass with experimental features defined elsewhere. For now, this plugin interface is only intended for internal use at Pinecone.There are three pieces involved here: the changes in this diff, a small separate package called
pinecone_plugin_interface, and the plugin implementations themself. Currently, there are no publicly available plugins.The
Pineconeclass here imports aload_and_installfunction from thepinecone_plugin_interfacepackage. When invoked, this function scans the user's environment looking for packages that have been implemented within a namespace package calledpinecone_plugins. If it finds any, it will use theclient_builderfunction supplied by thePineconeclass to pass user configuration into the plugin for consistent handling of configurations such as api keys, proxy settings, etc.Although the amount of code involved in
pinecone_plugin_interfaceis quite meager, externalizing it should give some flexibility to share code between the sdk and plugin implementations without creating a circular dependency should we ever wish to include a plugin as a dependency of the sdk (after functionality has stabilized).I have been defensive here by wrapping the plugin installation in a try/catch. I want to be confident that the mere presence of an invalid or broken plugin cannot break the rest of the sdk.
Type of Change
Test Plan
Tests should be green. I added some unit tests covering the new builder function stuff. As far as integration testing, I can't really add those here until we have publicly available plugins. But I've done manual testing to gain confidence.