An RPC method exists to apply code

[dustinbyrne]

Problem

The CLI needs a new RPC method that allows for code changes to be applied to a specific file. This method should take in a file path and new code content, and then use the language model (LLM) provided by packages/navie to determine the best way to apply these changes to the file's existing content.

Analysis

To implement this feature, the following steps need to be taken:

1. RPC Method Definition: Define a new RPC method in the CLI that accepts the path of the file and the code to be applied.
2. File Reading: The method must read the existing content of the specified file.
3. Navie LLM Integration: Integrate the use of the Navie LLM to intelligently determine where and how to insert the new code into the existing file content.
4. Apply Changes: Modify the file content as determined by the Navie LLM.
5. File Writing: Save the modified content back to the file.

Proposed Changes

Files to be Modified:

1. Add RPC Method: Define a new method to handle the RPC request for applying code changes.
2. Implement File Operations: Implement logic to read from and write to files in the CLI.
3. Integrate Navie LLM for Code Insertion: Add logic to use the Navie LLM to analyze and determine the optimal place to insert the new code.

Specific changes per file:

1. rpcHandler.ts:

   • Location: packages/cli/src/rpcHandler.ts
   • Changes:
     • Add a new function to handle the applyCodeChanges RPC method.
     • This function should accept two parameters: filePath and newCode.

2. fileOperations.ts:

   • Location: packages/cli/src/fileOperations.ts
   • Changes:
     • Implement a function to read content from the specified filePath.
     • Implement a function to write updated content back to the filePath.

3. llmIntegration.ts:

   • Location: packages/cli/src/llmIntegration.ts
   • Changes:
     • Implement a function that uses the Navie LLM to analyze the existing file content and determine the optimal place to insert the newCode.
     • This function should take the existingContent and newCode as inputs and return the modifiedContent.

Detailed Implementation Plan

1. Add RPC Handler:

   • Location: packages/cli/src/rpcHandler.ts

   • Method Signature:

     async function applyCodeChanges(filePath: string, newCode: string): Promise<void> {
     // Step 1: Read existing content
     const existingContent = await readFileContent(filePath);
     
     // Step 2: Use Navie LLM to determine changes
     const modifiedContent = await useNavieLLMToInsertCode(existingContent, newCode);
     
     // Step 3: Write new content back to the file
     await writeFileContent(filePath, modifiedContent);
     }

2. Implement File Reading and Writing Operations:

   • Location: packages/cli/src/fileOperations.ts

   • Functions to Implement:

     • readFileContent:

       
       import { readFile, writeFile } from 'fs/promises';

     async function readFileContent(filePath: string): Promise { return await readFile(filePath, 'utf-8'); }

     - **writeFileContent**:
     ```typescript
     async function writeFileContent(filePath: string, content: string): Promise<void> {
      await writeFile(filePath, content, 'utf-8');
     }

3. Integrate Navie LLM to Identify Code Insertion Points:

   • Location: packages/cli/src/llmIntegration.ts

   • Function to Implement:

     • useNavieLLMToInsertCode:

       
       import { analyzeAndInsert } from 'packages/navie/llmService';

     async function useNavieLLMToInsertCode(existingContent: string, newCode: string): Promise { // Interface with the Navie LLM service const response = await analyzeAndInsert(existingContent, newCode);

     if (response.success) { return response.modifiedContent; } else { throw new Error('Navie LLM failed to determine code insertion point'); } }

sequenceDiagram
    participant CLI as CLI
    participant RPCHandler as rpcHandler
    participant FileOps as fileOperations
    participant LLMIntegration as llmIntegration
    participant NavieLLM as packages/navie

    CLI->>RPCHandler: applyCodeChanges(filePath, newCode)
    activate RPCHandler

    RPCHandler->>FileOps: readFileContent(filePath)
    activate FileOps
    FileOps-->>RPCHandler: existingContent
    deactivate FileOps

    RPCHandler->>LLMIntegration: useLLMToInsertCode(existingContent, newCode)
    activate LLMIntegration

    LLMIntegration->>NavieLLM: analyzeAndInsert(existingContent, newCode)
    activate NavieLLM
    NavieLLM-->>LLMIntegration: modifiedContent
    deactivate NavieLLM

    LLMIntegration-->>RPCHandler: modifiedContent
    deactivate LLMIntegration

    RPCHandler->>FileOps: writeFileContent(filePath, modifiedContent)
    activate FileOps
    FileOps-->>RPCHandler: write success
    deactivate FileOps

    RPCHandler-->>CLI: applyCodeChanges success 
    deactivate RPCHandler

Summary

By adding these changes, the CLI will be able to apply new code to a given file path intelligently using the Navie LLM. This includes reading the current file content, utilizing the Navie LLM to determine how and where to insert the new code, and writing the updated content back to the file. This new RPC method will allow developers to programmatically modify files with the intelligence of Navie LLM to ensure changes are applied correctly and efficiently.

Related to #1894

[kgilpin]

Actually, I get it - Navie can't call RPC. So maybe we can reformulate as UI -> RPC "apply" -> calls Navie to get the FileUpdate -> applies file update. I'll provide an updated proposal.

[kgilpin]

Title

Implement RPC Command for File Update

Problem

Users need to apply file changes proposed by Navie in their code editor, but since Navie might not be running locally, all file operations must be handled by a separate JSON RPC service. This service must be able to either create new files or update existing files based on the proposed changes received from Navie.

Analysis

To solve this problem, the JSON RPC service (file.update command) needs to handle:

1. File Existence Check: Determine if the file exists on the filesystem.
2. File Creation: Create the file with the provided content if it does not exist.
3. File Update: If the file exists, read its content, validate the proposed changes, and apply the changes accordingly.

The file.update RPC command needs to perform these steps using a defined structure for the incoming request and the response from Navie (formatted with XML tags).

Proposed Changes

1. New Directory for File Commands:

   • Create a new directory packages/cli/src/rpc/file.

2. New File for Update Command:

   • Create a new file packages/cli/src/rpc/file/update.ts to implement the file.update RPC command.

3. Implementing File Update Logic:

   • Implement logic to check for the existence of the file.
   • If the file does not exist, create it with the provided content.
   • If the file exists, read its content, and update it using the proposed changes from Navie.

4. Leverage File-Update Logic:

   • Use the logic from file-update-service.ts to apply the proposed changes to the file.

File: packages/cli/src/rpc/file/update.ts

• Import necessary modules like fs for filesystem operations.
• Define the RPC command file.update which accepts a path and content as input arguments.
• Check if the file exists:
  • If not, create the file with the provided content.
  • If it exists, read the existing content of the file.
• Use the Navie AI @update command to get the proposed changes, passing the existing content as codeSelection and proposed change as question.
• Parse the XML response to get the original and modified content sections.
• Verify that the original content is in the file.
• Apply the proposed changes by replacing the original content with the modified content.
• Write the updated content back to the file.

This approach makes sure that the file updates are handled correctly even when Navie is running remotely, and all filesystem operations are managed by the JSON RPC service.

flowchart TD
    subgraph "Code Editor"
        A["User requests file update"]
    end

    subgraph "RPC Service"
        A --> B["RPC file.update"]
        B --> C(["Check if file exists"])
        C -->|No| D["Create file with provided content"]
        C -->|Yes| E["Read existing file content"]
        E --> F["Send content to Navie @update command"]
        F --> G["Validate original content in file"]
        G -->|Valid| H(["Replace original content with modified content"])
        G --->|Invalid| I["Error: Original content not found"]
    end 

    subgraph "Navie Service"
        F --> J["Process @update command"]
        J --> K[/"FileUpdate: { original, modified }"/]
        K --> G
    end

I generated the plan above using the following prompt (which I edited as an unnamed file in VSCode, then selected all the text and simply typed @plan)

# Title

AppMap can apply code changes to files.

# Summary

In the course of using Navie AI UI, Navie suggests code changes to files.

The user can choose to apply the change to the actual file on the filesystem.

The file change might be a new file or an update to an existing file.

The JSON RPC service provides a new commands to modify a file: file.update.

RPC file.update accepts the following type as the argument: export type File = {
  path: string;
  content: string;
};

RPC file.update first checks if the file exists, or not.

If the file does not exist, it creates the file with the provided content.

If the file exists, RPC file.update reads the content of the existing file. It then passes the
proposed change and the existing content to the Navie AI command @update. 

THe existing file content is passed as the codeSelection parameter. The proposed change is passed as the
"question".

The Navie AI @update command returns the following type, formatted with XML tags.

export type FileUpdate = {
  // Lines from the original content that should be replaced.
  original: string;
  // Updated content to be applied in place of the original content.
  modified: string;
};

An example:

\`\`\`xml
<change>
  <original>
    // This is the original content that should be replaced.
    // It can be multiple lines.
  </original>
  <modified>
    // This is the updated content that should replace the original content.
    // It can be multiple lines.
  </modified>
</change>
\`\`\`

RPC file.update checks that the `original` content exists in the file. Then in replaces that matching
region with the `modified` content.

# Architecture

The UI runs as an extension in the user's code editor.

The JSON RPC service runs as a separate process on the same machine as the code editor. 

The Navie service can either be running in-process with the JSON RPC service, or it can be running
on a remote server.

# Considerations

The Navie project cannot update files directly, because in some deployments it is not running on the 
same machine as the user's filesystem. Therefore, all file operations must be performed by the JSON RPC layer.

The RPC layer is implemented in packages/cli/src/rpc. File commands should be located in packages/cli/src/rpc/file.

Each RPC command should be implemented in a separate file, such as packages/cli/src/rpc/file/update.ts.

The file.update method can use logic from the Navie code in file-update-service.ts

Don't try and reuse or refactor the logic in file-update-service.ts.