Investigate Adding a management UI to CassIO

[karlunho-datastax]

Anant.us has developed a management UI (Iris) that can be added to CassIO. Leverages multiple opensource projects.

[karlunho-datastax]

@xingh Please add the slides you showed to this ticket.

[karlunho-datastax]

Per our conversations, CassIO focuses on persistence, so we should focus on persistence related management UIs.

[xingh]

Potential starting points

• Officially got ownership of the langchain-ui toolkit.
• Langchain-UI has a starting point for login etc. and object listing / editing. Works well. Built in Chakra UI - React ( open source UI framework)
• Langchain-UI-API uses Next . It uses Supabase for SSO and as a Backend as a service - Needs to be refactored for SSO or built from scratch. Next code can run on Vercel. (This is just for running the UI)
• Refine - React

UI

• SSO - user registers and logs in
• Add connection details for Astra
• Add data source
• See indexing status
• Explore results

UI API

• There are some back end as a service that can bootstrap on sqlite that dont require anything
• Could bootstrap with sqlite until user connects astra
• The objects that are being managed would be defined in yaml - will make it easier to react to changes in indexing / retreival framework like llamaindex
• Objects are initialized in astra connection on startup and hydrated with basic data

[karlunho-datastax]

Here is my thoughts:

• I think we need several UI components for Vector search. Here is UIs that I think we need to build:

1. Simple Prompt -> Vector UI. This UI would look very similar to the prompt UI that you see in tools like Google Vertex, but instead of returning a prompt from an LLM, it would return a vector search result. The reason why I'm prioritizing this item as the #1 thing to implement is because when a user first gets started, they want to get an intuition of what vector search is returning. Once they get the vector search results, they can paste the results into a separate tool (e.g. the Vertex AI prompting tool) to see if LLM gets a good result. This UI is very simple to implement compared to a table explorer or clustering visualization tool.

2. Simple table explorer - Take in a CQL statement, and dump out a table

   

3. Visualization clustering tool - similar to clustering analysis that you see in tensorboard or python

https://www.kaggle.com/code/minc33/visualizing-high-dimensional-clusters

Google Colab Link on outlier detection: https://colab.research.google.com/drive/1iiV6VAMGf28TwAlBuLpwfxzS0DSxz7be

5. Prompt -> Vector -> LLM . This UI will extend the Prompt -> Vector UI to do the last mile of generating a prompt that can call an LLM. This is more sophisticated because we will need the user's credentials for calling an LLM.

[xingh]

@karlunho-datastax acknowledged. Add my notes here.

[xingh]

Introduction

The goal of this proposal is to create a user interface for a data exploration suite that focuses on querying vector databases. The proposed user interface will leverage existing open source tools and apps that are traditionally built for web access to SQL databases.

Objectives

The objectives of this project are as follows:

• To create a user-friendly interface that allows users to query vector databases with ease.
• To leverage existing open source tools and apps that are currently used for web access to SQL databases to create a user interface that is familiar to users.
• Future: To provide a suite of tools that enable users to explore and analyze data in a variety of ways, including testing agents.

Scope

The scope of this project includes the following:

• Evaluation
  • Evaluate and Identify the right UI frameworks that can build upon existing code for Langchain UI which uses React, Chakra-UI for the front end, Next JS / Supabase for state management and has an existing Python based API
  • Evaluate and Identify the right UI components from existing web SQL applications that can integrate into React, Chakra-UI
  • Evaluate and Identify any vector search visualizations or appropriate visualizations beyond record listings with distances.
• Creation of a web-based user interface that is optimized for querying vector databases.
  • Integrate web SQL components initially as a “CQL” search explorer that goes against Cassandra CQL syntax wrapped in a python powered API
  • Integrate a natural language search which uses a chosen embedding to embed the query, find similar documents, distances, and displays records and distances.
• Future: Implementation of a suite of tools for data exploration and analysis, including data visualization and statistical analysis tools.
  • This could be to expose data in standardized ways for use in external tools via Jupyter, Zeppelin, Gephi (graph exploration)

Prior Art

Administrative

• Refine
• Chakra
• Superagent/Langchain-UI/Chakra

SQL

• Hue
• React SQL Editor

Table Components

• tanstack/table
• refine/table

Chat

• Several available applications that have a UI to upload documents and search. Some have multiple sessions, etc.

Prompt Tester

• Superagent/Langchain-UI/Chakra

[xingh]

Cool natural language to results UI. I like the data profiling, we can potentially do something with distances, thresholds contextually at a column level or at the row level

[xingh]

Here are my thoughts on a Cassio UI focused on vector search exploration w/ optimizations around making agents, though the interface looks similar, the use case is different. I do think that some of the parameters that a user can tune, could spit out some python code to just copy and paste - Copy Langchain Code , Copy LlamaIndex Code or Copy Python Code or even Copy Chainlit Code which has a full fledged UI already built.

It addresses Patricks' point about developers. It can lay the groundwork for doing IDE functionality.

Here would be the idea flow

• Sign up
  • Login using Github (can be just turned off if its just a mode that people start for exploration)
• Onboarding
  • Add astra connection info
  • If no Astra account, shows how to create account and download the creds / tokens to put it in there.
  • Right now vector search isn't GA in Cassandra so later this would allow them to add a Cassandra cluster credential.
  • If they don't have shit in the DB, give them a snippet to copy and paste anywhere and run.
• Choose a db, table
  • If you don't have data in there, show them the same options to copy code and ingest data.
  • If the data is coming from a URL and not a file, it could be run for them, but there's too many variables to account for. Makes sense to just keep it simple.
  • Likely that is not going to be the case since if they are using Cassio in Langchain, Cassio sets up those tables for them.
• Ask a question
  • if you like results, copy code and run it, or test it right there with a simple Langchain or LlamaIndex runner.
  • We don't have to reinvent the wheel just configure Chainlit. It takes care of running langchain / llamaindex and providing a chat UI.

Case in point. See LllamaHub's site and how it show cases each plugin.

Except in our case, we have a playground like thing to try things, and then it gives you code to copy code and run it.