search

kuzudb / kuzu

Embeddable property graph database management system built for query speed and scalability. Implements Cypher.
https://kuzudb.com/
MIT License
1.4k stars 99 forks source link

Proposal of Kùzu GUI / Visualization #1680

Closed mewim closed 1 year ago

mewim commented 1 year ago

This issue proposes a frontend user application for Kùzu that aims to provide end user: interactive querying, graph visualization, and schema design functionalities.

Main Features

Schema Design

The schema design feature should provide user with a graph visualization of the current schema and allow user to add/remove new node and rel tables, as well as modifying the existing tables (add / drop properties). The UI design of this feature should be the standard node-link diagram that's provided by many existing systems. Where the nodes corresponds to the node tables and the edges correspond to the rel tables. How to visualize RDFGraphs when they are integrated is to be decided. When the user clicks on a node/edge, the properties of the table should be displayed in a side panel with the options to modify it. There should also be a button to add a new node table. Connecting the node tables results in a new rel table to be created.

Interactive Query and Visualization

This feature is for users to run a Cypher query interactively on the UI and get the query results back in both tabular and graph format. This is a standard feature on every graph DBMS application that I'm aware of (e.g., Neo4J, Nebula, Memgraph). I propose to support the following features:

Graph Explore

This allows a user to pick up a starting point and interactively navigate the graph. By double clicking on a node, the nodes which has a connection to the node via a relationship should show up in the visualization. This is similar to Neptune graph explorer.

Loading Data

We should provide users with a collection of bundled dataset for quickly exploring Kùzu. The bundled dataset can be loaded to a database with one click.

Tech Stack

We will use the web stack to build this application. The frontend will be a Vue.js-based single-page application. The backend will be in Node.js. For graph visualization, we are considering the following libraries:

If none of the libraries meets all the design requirements, we can also use D3.js to render SVG manually.

Deployment

We have several options here:

Docker

This is what we should start with because a Docker deployment would be suitable for the users whose databases are located on a remote server (instead of their local machines). Since Kùzu is an embeddable database, there is not an option to connect to a remote Kùzu server. But with the Docker image, the user can simply start a Kùzu web server on the remote machine via:

docker run -it -p 8080:8080 -v /path/to/database:/db kuzudb/kuzu-gui

Then, the application can be accessed in the browser.

The downside of this approach is that for users who are solely working locally, they would have to install Docker on their machines.

Electron App

For users who are solely working locally, with the Electron wrapper, the app could be downloaded to local machine and started immediately without any additional setup. The user simply needs to download the app for their operating system and launch it, then open the local database file.

Sandbox

We should also install a sandbox deployment Kùzu's homepage to allow users to try Kùzu without downloading/installing anything.

Since all three deployment share the same frontend codebase, they should provide a similar user experience and it should be relatively easy to port a web application into different platforms.

Accessing the Same Database Through Multiple Processes

As stated in our documentation, Kùzu currently does not provide any safety guarantees if multiple processes access the same database:

Each database you create is identified by its directory. If you are going to concurrently interact with the same database through more than one Database instances, you should ensure that you only issue read queries to the system. Your writes from one Database instance will not be visible to your other program and you can easily corrupt your database. Kùzu currently does not ensure that you are connecting to the same database directory through a single Database instance.

If the GUI application writes to the database while the user is running a script in Python or using the CLI writing to the same database, users might corrupt their databases. In fact it is not even safe for there to be 1 write process and one or more read-only processes. That's because we currently do not have a mechanisms for different processes to coordinate with each other. For example, currently if there are multiple threads in the same process that are using the same Database object, we have a way of coordinating them, e.g., if one of them is committing a write transaction, we can ensure that all read transactions first complete and leave the system and no new transaction starts. When multiple processes access the same database directory using different Database instances, the process that is writing could not block other processes starting transactions during checkpointing. Therefore if we will allow multiple processes to access the same database directory, we need a mechanism to ensure that either: (i) there is only one process accessing the database and it can read and write freely; or (ii) there are multiple read-only processes accessing the same database (so none can write to the database). To enforce these, we first need to introduce the notion of creating a read-only Database instance and then keeping track of the processes accessing a directory.

mewim commented 1 year ago

TODOs:

prrao87 commented 1 year ago

Thanks for the update! This sounds great overall. If possible, an interim solution that allows some form of basic visualization (something like the approach related to Docker) would be a great start, as the development for the full-fledged Electron-based editor happens over the longer term. It would really help new users onboard to Kùzu and experiment with it faster. Thanks for the great work!

prrao87 commented 1 year ago

Just wanted to leave this idea here: in case there's an appetite for integrating more Rust code into Kùzu, Tauri, an Electron alternative written in Rust, offers a more light-weight solution for a future GUI or web-based application. I've been seeing more and more talk about it because of its relatively small binaries, fast performance, and it seems easy enough to learn, allowing developers to leverage the full power of the Rust ecosystem too 👍

mewim commented 1 year ago

Closing this issue now as the major features for the UI app are implemented. Remaining feature requests are now moved to https://github.com/kuzudb/kuzu-ui/issues.