Closed AdamSobieski closed 5 years ago
benbrown
commented
5 years ago Hey Adam -- These are interesting ideas! You should see what you can achieve with the current SDK and let us know! The web chat component in particular would allow you to send and receive events between the bot and external pieces of software (at least on the web)
You might consider contributing anything you build to the community repo which houses a collection of plugins and extensions from third party devs!
AdamSobieski
commented
5 years ago Thank you. I'm looking at the source code of the current version of the SDK presently. I think that I can prototype the aforementioned concepts utilizing sockets for platform-independent interprocess communication between dialogue systems and other software applications.
The prototype would include two software applications. The first software application would be a dialogue system or bot which would be able to function as a natural-language user interface for the second software application. The second software application might include some colored shapes to be moved around and stacked atop one another as well as a popup window with a form on it for entering data. The second software application would send diagrammatic content to the dialogue system for users to be able to move and stack the colored shapes or for users to be able to interact with the popup window using natural language.
The prototype would include a means of transmitting diagrammatic content between the applications, a means of serializing and deserializing diagrams or conversation flows (e.g. JSON). The SDK's diagram model (Microsoft.Bot.Builder.Dialogs.Adaptive) might be extended to include action(s) for interacting with connected-to software applications.
On these topics, the declarative adaptive dialog samples are useful.
Providing Software Applications with Natural-language User Interfaces via Interoperation with Dialogue Systems
There are a number of interesting scenarios to consider with respect to a general-purpose interoperation between dialogue systems and other software applications and services.
Scenario 1: We can consider interoperation between dialogue systems and CAD/CAE software. As users interact with dialogue systems, the dialogue systems could interoperate with CAD/CAE software to assist users by performing tasks.
What are the best practices with respect to developing software applications and services towards their interoperation with dialogue systems and intelligent virtual assistants?
What are the best practices with respect to developing dialogue systems towards their interoperation with other software applications and services?
Dialogue Systems Controlling Software Applications
Scenario 2: We can consider interoperation between dialogue systems and Web browsers. Dialogue systems could open Web browsers to addresses, scroll to specific content, and highlight specific content. Web browsers are also digital textbook reader software. Dialogue systems could open digital textbooks to specific content and highlight specific content. Such interoperation could be useful for QnA systems and intelligent tutoring systems.
Dialogue Systems Receiving Events and Notifications from Software Applications
Considering general-purpose interoperation and automation, dialogue systems should be able to receive asynchronous events and notifications from the software applications and services which they are connected to.
Brainstorming
Perhaps we might consider an
applicationscripting object from the dialogue system development and diagramming perspective. The matter could, then, be phrased as interacting with anapplicationscripting object (representing an interface to a software application or service) which might implement something like anINotifyPropertyChangedinterface with respect to its object graph.Perhaps designing or developing interactions between dialogue systems and software applications and services could make use of new diagramming elements.
Perhaps general-purpose interoperation and automation could expand upon adaptive dialogue concepts. Connected-to software applications and services could provide diagrammatic content for dialogue systems. This would be useful in the CAD/CAE scenario. One can envision using natural language to initiate actions on selected objects or regions. After so doing, multiple parameters may need to be specified. As a popup window appears in the CAD/CAE application for entering scalars, the CAD/CAE application could provide diagrammatic content to the connected-to dialogue system for entering the values using natural language.