Navie emits references to file paths along with generated code

[dustinbyrne]

Problem

Navie must be modified to automatically prepend a comment directive specifying the file path before emitting code fences to the frontend. We need to ensure that the buffer correctly identifies code blocks and inserts the comment directive before them.

Analysis

To accomplish this, we need to modify Navie's code generation process to detect potential code fences and withhold emitting tokens until it is certain a code block is being emitted. At that point, a comment directive must be prepended, and emitting tokens should continue as normal. This ensures that all generated code snippets include the necessary metadata for further frontend operations.

Here’s a conceptual breakdown:

1. Detection: Identify when a code block is about to be emitted.
2. Withholding Tokens: Withhold tokens until it is certain a code block is to be emitted.
3. Prepending: Prepend the comment directive indicating the file path.
4. Emitting Tokens: Continue emitting tokens as normal after the directive is added.

Proposed Changes

1. Update GenerateAgent to handle code fences:
   • Modify the logic to detect potential code fences.
   • Withhold tokens and confirm the presence of a code block.
   • Prepend the comment directive once a code block is confirmed.
   • Continue emitting tokens as normal.

File-specific changes:

1. generate-agent.ts:
   • Detection of Code Fences: Implement logic to detect potential code fences and withhold tokens as needed.
   • Prepending Directives: Add logic to prepend the comment directive before emitting confirmed code blocks.
   • Continue Emitting Tokens: Ensure tokens continue being emitted normally after the directive is added.

Detailed Implementation Plan

1. Update generate-agent.ts:
   • Location: packages/navie/src/agents/generate-agent.ts
   • Changes:
     • Add logic to intercept and analyze the token stream to detect potential code fences.
     • Implement a buffer to withhold tokens until enough context confirms a code block.
     • Prepend a comment directive (e.g., <!-- file: path/to/file.ext -->) before the start of the code block.
     • Continue emitting the tokens for the rest of the code block and subsequent content.

Specific Logic to Implement:

1. Buffer Tokens: Create a buffer to hold tokens temporarily.
2. Detect Code Fence Start: Monitor tokens to detect the start of a code fence.
3. Confirm Code Block:
   • Analyze the buffered tokens to confirm the presence of an actual code block.
   • Check if the necessary comment directive is missing.
4. Prepend Directive: Inject the comment directive indicating the file path once a code block is confirmed.
5. Emit Tokens: Release the buffered tokens and continue emitting new tokens.

Updating generate-agent.ts will involve the following areas in the code:

• Token processing loop: Modify this part to integrate the buffering logic.
• Directive Injection: Implement the logic to prepend the comment directive once the presence of a code block is confirmed.

Example Change in generate-agent.ts

• Detection and Buffering:

  • Intercept tokens as they are processed.
  • Buffer tokens until a potential code fence is identified.

• Prepending Directive:

  • Once a code block is identified, insert a comment directive with the file path.

By following this plan, Navie will be capable of automatically prepending comment directives to code snippets, providing the necessary metadata for further processing by the frontend.

Related to #1893

[kgilpin]

How do you know what the file name is in order to prepend it?

Maybe we can do this with prompting alone. If it’s wrong sometimes, that is ok. The user will have to fall back to copy-paste, or the apply button can prompt the user for the file name.

[appland-release]

:tada: This issue has been resolved in version @appland/components-v4.30.0 :tada:

The release is available on:

• @appland/components-v4.30.0
• GitHub release
• @appland/components-v4.30.0

Your semantic-release bot :package::rocket: