Outline best practices for sharing Kuzu databases (hashtag2885)

[vikramsubramanian]

I'm wondering whether and how Kuzu databases may be shared to others. For example, after creating a Kuzu database, one ends up with a directory of files associated with the database. May this set of files be safely shared to others with the intention of enabling reproducible graph data science work? Mostly here I wonder: would it work at all from system to system, and separately, is it secure to do this openly/publicly (no "invisible" sensitive data included which wasn't a part of data ingest or creation)?

If Kuzu databases themselves shouldn't be shared, what is the best way of "exporting/converting" the entire database so it may be reused elsewhere?

Apologies if this is already outlined somewhere, I tried to find this and may not have been using the correct keywords. Thanks in advance for any thoughts you have on this! )

[mayil-ai[bot]]

• Determine if Kùzu databases are designed to be portable:

  • Check the documentation or contact the developers to understand if the database files are self-contained and if there are any system-specific dependencies.
  • Ensure that file paths and configurations within the database do not rely on absolute paths that would not be valid on another system.

• Assess security and privacy concerns:

  • Review the database files for any embedded sensitive information that may not be immediately apparent, such as metadata or logs.
  • If sensitive data might be included, provide a mechanism to scrub or anonymize this data before sharing.

• Sharing Kùzu databases:

  • If the database files are portable and free of sensitive information, they can be compressed and shared as an archive (e.g., zip, tar.gz).
  • Use secure transfer methods to share the database files (e.g., encrypted email, secure file transfer services).

• Exporting/converting the database:

  • If direct sharing of database files is not recommended, implement an export feature that can dump the database contents into a standard format (e.g., CSV, JSON, GraphML).
  • Provide an import feature that can reconstruct the database from the exported data on another system.

• Documentation and best practices:

  • Document the process for safely and securely sharing Kùzu databases.
  • Include instructions for exporting and importing data if direct sharing is not advised.
  • Update the documentation regularly as the software evolves.

• Example pseudo code for an export feature:

  void exportDatabase(KuzuDB& db, const std::string& exportPath) {
  // Iterate over all nodes and edges in the database
  for (auto& node : db.getNodes()) {
      // Convert node to a standard format (e.g., JSON)
      std::string nodeData = convertNodeToJson(node);
      // Write to export file
      writeFile(exportPath, nodeData);
  }
  for (auto& edge : db.getEdges()) {
      // Convert edge to a standard format (e.g., JSON)
      std::string edgeData = convertEdgeToJson(edge);
      // Write to export file
      writeFile(exportPath, edgeData);
  }
  }

• Example pseudo code for an import feature:

  void importDatabase(KuzuDB& db, const std::string& importPath) {
  // Read the exported data
  std::string data = readFile(importPath);
  // Parse the data and reconstruct the database
  for (auto& nodeData : parseNodes(data)) {
      Node node = createNodeFromJson(nodeData);
      db.addNode(node);
  }
  for (auto& edgeData : parseEdges(data)) {
      Edge edge = createEdgeFromJson(edgeData);
      db.addEdge(edge);
  }
  }

• Ensure that the sharing and exporting/importing mechanisms are well-tested and include error handling for compatibility and data integrity issues.