Backend - Add/Delete/Edit Node and Edges in Database

[ChTiSh]

Objective: Develop the backend logic and algorithms necessary to store and update the tree in the testing MongoDB database. Assumption: Zustand store can hold info for all tree nodes and edges, which will be sent when user click "save" button to backend API Assumption: The data structure of Gemini generated JSON file (Initial tree) would be different from what backend received from the Zustand store (User edited tree)

Project Scope: Create Sample Data: - 2 hours Create Sample Node.ts and Edge.ts with min. 30 nodes and corresponding edges, assuming received from frontend then stored in database Reference: Sample node.ts and edge.ts on: https://reactflow.dev/learn/advanced-use/state-management

Develop one endpoint that receives node data from the frontend, Including necessary node and edge attributes - 3 hours Explore Feasibility: Investigate the limits and drawbacks for this approach, if this works at a reasonable speed, cost, space, time - then AC2 considered satisfied, CRUD would become stretch) - 4 hours

Acceptance Criteria: AC1: Implement API endpoints to receive updated tree data (nodes and edges). The endpoints should be capable of handling large payloads if the tree is complex. AC2: Implement Logic to handle the updated tree data and save it to the database

(Optional if replacing/adding whole tree doc approach is sufficiently functional for MVP): Explore CRUD Operations Endpoints and workflow: Implement separate logic to handle inserting, deleting, and updating each individual node to database. - 4 hours

Stretch Goal - Definitely should not be included, but it would be helpful: Explore if backup version can be stored if inserting a new tree as document makes sense Explore if can only send changed nodes instead of the whole tree

Reference:

Using Zustand to manage the entire state of a React Flow diagram and then sending the complete state to a single backend endpoint upon user completion of editing offers several advantages and some potential drawbacks compared to handling add, edit, and delete operations for each node and edge individually. Here’s a breakdown of the two approaches, considering a scenario with no more than 100 nodes for an MVP (Minimum Viable Product).

Bulk Update Approach (Single Endpoint) Advantages:

Simplicity: Managing the state and syncing with the backend is straightforward because it involves a single operation at a defined point in time (i.e., when the user finishes editing). This reduces the complexity of handling numerous API calls and state synchronization issues. (Improvement can be done with proper implementation of React Query) Consistency: By updating the entire tree at once, you ensure that the state on the backend is always consistent with the client. There's no risk of operations being applied out of order. (Improvement can be done with React Query) Performance on Frontend: Fewer API calls can lead to better performance on the frontend, as the system makes a single request rather than multiple requests over the editing session. Disadvantages:

Data Overhead: Sending the entire tree, even with 100 nodes, can become a large payload, especially if nodes contain substantial metadata. Error Handling: Errors in saving can result in losing all changes made during a session, as opposed to incremental saves where only the last action might be lost or retried. Backend Complexity: The backend must be capable of efficiently processing bulk data and managing potential conflicts in a full replacement scenario.

Incremental Update Approach (Separate Endpoints for Add, Edit, Delete) Advantages:

Data Efficiency: Only changed data is sent to the backend, which can minimize network usage and improve responsiveness. Recovery from Errors: Easier to handle errors and rollback changes on a per-action basis without affecting the entire tree. Real-Time Updates: Better suited for applications requiring real-time updates across multiple clients, as changes can be broadcasted and applied incrementally. Disadvantages:

Complexity: Requires more sophisticated state management on both the client and server to handle individual changes and ensure consistency. Frequent API Calls: Each action triggers a separate API call, which can increase the load on the backend and require more comprehensive rate limiting and performance optimization. Partial Update Issues: Risk of partial updates or conflicts, particularly in a collaborative environment where multiple users might be editing the same tree concurrently. Conclusion For an MVP with up to 100 nodes, the choice between these approaches depends on your specific requirements:

If the application's use case involves infrequent, session-based edits where users are expected to make many changes that only need saving once they're complete, the bulk update approach might be more suitable. This method simplifies both the development and the usage patterns. If the application might evolve into needing more real-time, collaborative features, or if network efficiency and error resilience are priorities, the incremental update approach could provide more flexibility and robustness.

[relandboyle]

This has been discussed with Chen, Kinski, Jordan, and Colin, and I feel we're in agreement to commit to the below strategy until further notice.

The short-term solution will be Bulk Update Approach (Single Endpoint).

In addition to allowing for a simpler API, this approach allows for simpler edit logic and state management in the client.

A good way to avoid session data loss will be to implement AutoSave, e.g.: If there have been no edits for five seconds, Save.

It is unclear just how many nodes and how much metadata will be transmitted, or at what point data volume becomes problematic. Let's start simple and adapt if necessary.