Server-client Architectural flaw

[cyb3rdog]

Issue description

Unfortunatelly, escape pod itself, and this extensions for escape pod, has been designed incorrectly with an architectural flaw. Instead of escapepod being client to to the escapepod-extension server, it should be vice-versa, to allow multiple extensions to be used with the escapepod. Currently, it is possible to have only one extension, to which the escapepod binary connects for passing the extent intents for further processing.

Expected functionality

To have more the possibility to use more than one extension with the escape pod. Example

• first extension from an author A will process intents 1, 2 and 3.
• second extension from an author B will process intents 4. 5 and 6. In other words, when i release my extension to the community, it wont be possible for them to use their own extensions anymore.

Resolution proposal

1) Change the design and architecture of this to perhaps include the server part into the escapepod binary itself, and for the extension connectivity, expose the server with a bi-directional event streaming grpc service, so multiple clients can connect and subsribe to the events raised from the escape pod server, and also respond with a respose of whether they have processed/will be processing the given intent or not. Conceptual-wise this would be similar principle to current event streaming implemented in original anki API, which as well publishes various events for multiple clients to subsribe for, never knowing which client will actually perform any further processing of such event and which don't. 2) to re-implement this 'escape-pod-extension' as a one static 'extension server' to the 'escapepod' binary (so the rpi will always have configured this one as a static extension endpoint), but instead of direct processing the intents itself, add the new server into this to actually change this to be kind of double-server 'proxy hub' awaiting the intents from ep, and forwarding them into the bi-directional stream for various clients.

[cyb3rdog]

After some more research and implementation of custom made extension proxy some time ago, which does serve the purpose of publishing the event stream of the escapepod extent intents to the clients, and works as an actual workarround for this issue, I have eventually ended up with merging all three sources of intents (user intents+wake word end+escapepod extension) into one target endpoint anyways.

Here is an overview I have drawn to better undestand the current extensibility implementation and its limits (please let me know if there's something which is not correct...)

This all leads me to conviction, that the whole thing should have been perhaps realized slightly differently, so I am posting this as resolution proposal no. 3.

[bussardrobbie]

Hey @cyb3rdog, thanks for this. I know it's been a while since this was posted but I wanted to give you an update.

Currently we are refactoring a large portion of the Escape Pod to increase the flexibility of the platform. Once we have completed that, we are going to work on a couple of problems with the current implementation. Along with the architectural issue you mention, we also understand that the "extension" doesn't add in the intent / behavior to the service that manages those. We would like to implement a Pub-Sub grpc endpoint in chipper that would allow extensions to listen in for certain "intents" and have a way to auto register those intents into the intent manager.

Stay tuned for these adjustments and thanks for your patience.

[moribundant]

@bussardrobbie,

This needs to have some prioritization! High prioritization! The problem @cyb3rdog discovered turns into a colossal aggravation if you are using the EP as a development platform; i.e., V's 'brain'. This, of course, makes the most sense because 1) V's processor architecture cannot support ASR or any other type of recognition other than minimal face and cube detection, and 2) the EP has the fastest interface so that S-R delays are minimal.

[moribundant]

Let me add that I appreciate the fact that DDL has recognized this.