kleneway / jacob

Just Another Coding Bot
https://jacb.ai
Apache License 2.0
0 stars 0 forks source link

Implement Structured Research Storage and Retrieval #60

Open kleneway opened 1 week ago

kleneway commented 1 week ago

Implement Structured Research Storage and Retrieval

Description

Currently, research data is stored as a string in the todo description. We need to update this to a more structured format using a new TypeScript interface called Research and create a new research table in the database. Various parts of the codebase need to be updated to use this new structure.

Requirements

  1. Create a new TypeScript interface Research with the following fields:

    • type: ResearchAgentActionType (Type of research conducted)
    • question: string (The query or question asked)
    • answer: string (The response or answer obtained)
    • issueId: string (Reference to the GitHub issue this research is related to)
  2. Create a new research table in the database with the following fields:

    • id: identity().primaryKey()
    • todoId: integer().foreignKey("todos", "id").onDelete("CASCADE")
    • issueId: integer()
    • type: enum("research_agent_action_type", Object.values(ResearchAgentActionType))
    • question: text()
    • answer: text()
    • Timestamps
  3. Update the researchIssue function to capture data using the new Research format and return an array of Research objects instead of a string.

  4. Update the Todos to:

    • Check if any research has already been done for the issue.
    • If research exists, skip the research step.
    • If not, do the research, create the Todo first, and link the research to the todos using the todoId.
  5. Add a function to research.ts to pass in an issue ID (payload.issue.number) and get back an array of research items.

  6. Update agentEditFiles and agentFixError to:

    • Retrieve research from the database using the new function in research.ts.
    • Convert the research into a string of question/answers.

Files to Update:

New Files:

Acceptance Criteria:

Research:

ResearchCodebase

Question: How is the current 'research' string created and stored for a new Todo item in the codebase? This information is needed to understand the existing implementation before updating it to use a structured 'Research' interface.

  1. Detailed Specifications:

    • Title: Research on Current 'Research' String Implementation for New Todo Items
    • Description: This research aims to understand how the current 'research' string is created and stored for new Todo items in the codebase. This information is needed to update the implementation to use a structured 'Research' interface and database table.
  2. File Structure and File Paths:

    • Files to Modify:
      • src/server/api/routers/events.ts
      • src/server/agent/research.ts
      • src/server/code/agentEditFiles.ts
      • src/server/code/agentFixError.ts
      • src/server/db/db.ts
      • src/server/db/dbScript.ts
    • New Files to Create:
      • src/server/db/migrations/[timestamp]_createResearchTable.ts
      • src/server/db/tables/research.table.ts
    • Directory Structure Overview:
      • src/server/api: Contains API routers and related utilities.
      • src/server/agent: Contains agent logic for different tasks.
      • src/server/code: Contains code generation and modification logic.
      • src/server/db: Contains database configuration, tables, and migrations.
  3. Code Snippets:

    • src/server/api/routers/events.ts:

      // ... other imports
      import { researchIssue } from "~/server/agent/research";
      
      // ... inside createTodo function
      const sourceMap = await cloneAndGetSourceMap(repo, accessToken);
      const research = await researchIssue(issueText, sourceMap, rootPath);
      
      await db.todos.create({
      // ... other fields
      description: `${issue.title}\n\n${issueBody}\n### Research\n${research}`,
      // ... other fields
      });
    • src/server/agent/research.ts:

      // ... other imports
      
      export const researchIssue = async function (
      githubIssue: string,
      sourceMap: string,
      rootDir: string,
      // ... other params
      ): Promise<string> {
      // ... implementation details
      return `## Research: ${gatheredInformation.join("\n")} \n\n## Questions for Project Owner: \n\n [ ] ${questionsForProjectOwner.join("\n [ ] ")}`;
      };
    • src/server/code/agentEditFiles.ts and src/server/code/agentFixError.ts:

      // ... other imports
      
      // ... inside agentEditFiles or agentFixError function
      const research = ""; // TODO: currently this is part of the GitHub issue, need to separate it out
    • src/server/db/db.ts:

      // ... other imports
      
      export const db = orchidORM(config.database, {
      // ... other tables
      });
    • src/server/db/dbScript.ts:

      // ... other imports
      
      export const change = rakeDb(config.allDatabases, {
      // ... other options
      commands: {
       // ... other commands
      },
      import: (path) => import(path),
      });
    • Example of table creation migration:

      import { change } from "../dbScript";
      
      change(async (db) => {
      await db.createTable("projects", (t) => ({
       id: t.identity().primaryKey(),
       // ... other fields
      }));
      });
    • Example of table definition:

      import { BaseTable } from "../baseTable";
      
      export class ProjectsTable extends BaseTable {
      readonly table = "projects";
      columns = this.setColumns((t) => ({
       id: t.identity().primaryKey(),
       // ... other fields
      }));
      }

      ResearchCodebase

Question: Is there an existing database schema or migration file that includes the 'todos' table definition? This is required to understand the current database structure and to plan the creation of a new 'research' table with a foreign key to 'todos'.

  1. Detailed Specifications:

    • Title: Database Schema for 'todos' Table
    • Description: The information is needed to understand the existing database schema of the 'todos' table, which is necessary for creating a new 'research' table with a foreign key relationship to the 'todos' table. This understanding will ensure the new table integrates seamlessly with the existing database structure.
  2. File Structure and File Paths:

    • Files to Update:
      • src/server/db/migrations/20240604211756_add_todos_table.ts: This file contains the migration script for creating the 'todos' table. It will be updated to include the new 'research' table and establish the foreign key relationship.
      • src/server/db/tables/todos.table.ts: This file defines the schema for the 'todos' table using the Orchid ORM. It will be updated to reflect the relationship with the new 'research' table.
    • New Files to Create:
      • src/server/db/tables/research.table.ts: This file will define the schema for the new 'research' table using the Orchid ORM.
      • src/server/db/migrations/[timestamp]_create_research_table.ts: This file will contain the migration script for creating the 'research' table.
    • Directory Structure Overview:
      • src/server/db: This directory contains all database-related files, including migrations, table definitions, and the database connection setup.
      • migrations: This subdirectory stores the migration scripts for managing database schema changes.
      • tables: This subdirectory contains the schema definitions for each database table using the Orchid ORM.
  3. Code Snippets:

    • Existing 'todos' Table Migration (src/server/db/migrations/20240604211756_add_todos_table.ts):

      import { change } from "../dbScript";
      
      const todoStatusValues = ["todo", "in_progress", "done", "error"] as [
      string,
      ...string[],
      ];
      
      change(async (db, up) => {
      if (up) {
       await db.createEnum("todo_status", todoStatusValues);
      }
      
      await db.createTable("todos", (t) => ({
       id: t.identity().primaryKey(),
       projectId: t.integer().foreignKey("projects", "id"),
       description: t.text(),
       name: t.text(),
       status: t.enum("todo_status"),
       issueId: t.integer().nullable(),
       position: t.integer().default(0),
       branch: t.text().nullable(),
       isArchived: t.boolean().default(false),
       ...t.timestamps(),
      }));
      
      if (!up) {
       // Drop the enum AFTER we drop the table when we are rolling back
       await db.dropEnum("todo_status", todoStatusValues);
      }
      });
    • Existing 'todos' Table Schema (src/server/db/tables/todos.table.ts):

      import type { Selectable, Insertable, Updateable, Queryable } from "orchid-orm";
      import { BaseTable } from "../baseTable";
      import { TodoStatus } from "../enums";
      
      const TODO_STATUS_VALUES = Object.values(TodoStatus) as [
      TodoStatus,
      ...TodoStatus[],
      ];
      
      export type Todo = Selectable<TodosTable>;
      export type NewTodo = Insertable<TodosTable>;
      export type TodoUpdate = Updateable<TodosTable>;
      export type TodoQueryable = Queryable<TodosTable>;
      
      export class TodosTable extends BaseTable {
      readonly table = "todos";
      columns = this.setColumns((t) => ({
       id: t.identity().primaryKey(),
       projectId: t.integer().foreignKey("projects", "id"),
       description: t.text(),
       name: t.text(),
       status: t.enum("todo_status", TODO_STATUS_VALUES),
       position: t.integer().default(0),
       issueId: t.integer().nullable(),
       branch: t.text().nullable(),
       isArchived: t.boolean().default(false),
       ...t.timestamps(),
      }));
      }
    • Example of Enum Creation in Migration (src/server/db/migrations/20240604211756_add_todos_table.ts):

      if (up) {
      await db.createEnum("todo_status", todoStatusValues);
      }
    • Example of Foreign Key Relationship in Migration (src/server/db/migrations/20240604211756_add_todos_table.ts):

      projectId: t.integer().foreignKey("projects", "id"),
    • Example of Table Creation Migration (src/server/db/migrations/20240604211756_add_todos_table.ts):

      await db.createTable("todos", (t) => ({
      // ... column definitions
      }));
    • Example of Orchid ORM Table Schema Definition (src/server/db/tables/todos.table.ts):

      export class TodosTable extends BaseTable {
      readonly table = "todos";
      columns = this.setColumns((t) => ({
       // ... column definitions
      }));
      }
    • Coding Conventions:

      • Follow the existing coding conventions and style used in the provided code snippets.
      • Use TypeScript for type safety.
      • Adhere to the Orchid ORM syntax for defining table schemas and relationships.
      • Use clear and descriptive variable and function names.
      • Add appropriate comments to explain the code.

ResearchCodebase

Question: Where is the 'researchIssue' function located within the codebase, and how does it currently operate? This information is necessary to update the function to capture data using the new 'Research' interface format.

  1. Detailed Specifications:

    • Title: Location and Functionality of the 'researchIssue' Function
    • Description: This information is needed to locate the 'researchIssue' function in the codebase and understand its current implementation. This understanding is crucial for updating the function to capture research data using the new 'Research' interface format.
  2. File Structure and File Paths:

    • Files to Update:
      • src/server/agent/research.ts: This file contains the 'researchIssue' function that needs to be modified.
    • Directory Structure Overview:
      • The 'src/server/agent' directory contains files related to different agent functionalities, including research.
  3. Code Snippets:

    • Relevant Code Snippet (src/server/agent/research.ts):

      export const researchIssue = async function (
      githubIssue: string,
      sourceMap: string,
      rootDir: string,
      maxLoops = 3,
      model: Model = "gpt-4-0125-preview",
      ): Promise<string> {
      console.log("Researching issue...");
      //  - ResearchInternet: Search the internet for specific details if there are unique aspects that need further understanding.
      const systemPrompt = dedent`You are an advanced AI coding assistant designed to efficiently gather information and address GitHub issues. Your role is to analyze the provided GitHub issue, codebase source map, and previously gathered information to determine the necessary steps for resolving the issue.
      
       Key Responsibilities:
           1. Thoroughly understand the GitHub issue and its requirements.
           2. Assess the available information, including the codebase, external resources, and previous clarifications.
           3. Identify any missing information that is crucial for addressing the issue.
           4. Decide on the most effective actions to obtain the missing information:
               - ResearchCodebase: Analyze the existing codebase for relevant information.
               - ResearchInternet: Search the internet for specific details if unique aspects require further understanding
               - AskProjectOwner: Only if absolutely necessary, ask the project owner for additional details or clarifications.
           5. Formulate clear and concise questions or queries for each action to gather the missing information.
           6. If all necessary information is available, proceed with providing a solution to the GitHub issue.
      
       Guidelines:
           - Maintain a professional and objective tone throughout the information gathering process.
           - Break down complex issues into smaller, manageable tasks.
           - Prioritize the most critical missing information first.
           - Never respond with text questions, use the provided tools to gather the required information.
           - Provide clear reasoning for each identified missing piece of information and how it relates to the issue.
           - Ensure that the planned actions and queries are detailed, specific, relevant, and likely to yield the required information.
           - If no further information is needed, confidently state that you have sufficient details to proceed with resolving the issue.
      
       Remember, your goal is to efficiently gather all necessary information to provide a comprehensive solution to the GitHub issue. Approach the task methodically to ensure that no details are overlooked.`;
      
      //2. **ResearchInternet:** Search the internet for specific details if there are unique aspects that need further understanding.
      const userPrompt = dedent`You are an AI coding assistant tasked with addressing a specific GitHub issue for a codebase. Your first step is to gather all necessary information to fully understand the issue and its context. Your goal is to identify all missing information and determine the best way to obtain it using the following tools:
      
       - **ResearchCodebase:** Analyze the existing codebase for relevant information. This is the preferred tool for most queries.
       - **ResearchInternet:** Search the internet for specific details if there are unique aspects that need further understanding.
       - **AskProjectOwner:** Ask the project owner for any additional details or clarifications. This tool should rarely be used, if ever. Only use this if there is no other way to obtain the information.
       - **ResearchComplete:** Confirm that the most important information has been gathered to address the issue.
      
       ### GitHub Issue Details:
           - **Issue:** ${githubIssue}
           - **Repo Source Map:** ${sourceMap}
      
       ### Task:
           1. **Initial Understanding:**
               - Understand the GitHub issue and its requirements.
               - Identify the goal and what needs to be achieved.
      
           2. **Assess Available Information:**
               - Review the provided codebase information, external information, and previous clarifications.
               - Determine what information is already available.
      
           3. **Identify Missing Information:**
               - Reflect on the provided details and identify all missing pieces of information needed to fully address the issue.
               - Specify clearly what information is missing and why it is needed.
      
           4. **Plan Information Gathering:**
               - Decide on the best action to obtain each missing piece of information (Research Codebase, Research Internet).
               - Formulate the specific questions or queries for each action.
      
       Choose the correct tools and formulate very specific, detailed queries to gather all of the missing information effectively. If you need to ask questions, only ask a maximum of 5 questions. Each question should be a full sentence and clearly state what information you are looking for.
       ### Important:
           If you have all the necessary information to proceed with the task, return a single tool call to confirm that the research is complete. If you need more information, use up to 5 of the appropriate tools to gather it.`;
      
      const messages: OpenAI.ChatCompletionMessageParam[] = [
       { role: "system", content: systemPrompt },
       { role: "user", content: userPrompt },
      ];
      
      let allInfoGathered = false;
      const gatheredInformation: string[] = [];
      const questionsForProjectOwner: string[] = [];
      let loops = 0;
      
      while (!allInfoGathered && loops < maxLoops) {
       loops++;
       const response = await sendGptToolRequest(
         messages,
         researchTools,
         0.3,
         undefined,
         2,
         60000,
         model,
         "required",
         true,
       );
      
       const toolCalls = response.choices[0]?.message.tool_calls;
      
       if (toolCalls && toolCalls.length > 0) {
         allInfoGathered = true;
      
         for (const toolCall of toolCalls) {
           const functionName = toolCall.function.name as ResearchAgentActionType;
           if (!Object.values(ResearchAgentActionType).includes(functionName)) {
             console.error(`Invalid function name: ${functionName}`);
             continue;
           }
           if (functionName === ResearchAgentActionType.ResearchComplete) {
             allInfoGathered = true;
             break;
           }
           const args = JSON.parse(toolCall.function.arguments) as {
             query: string;
           };
           console.log(`Calling function: ${functionName} with arguments:`, args);
           const functionResponse = await callFunction(
             functionName,
             args,
             githubIssue,
             sourceMap,
             rootDir,
           );
           // messages.push({
           //   role: "function",
           //   name: functionName,
           //   content: functionResponse,
           // });
           if (functionName === ResearchAgentActionType.AskProjectOwner) {
             questionsForProjectOwner.push(args.query);
           } else {
             gatheredInformation.push(
               `### ${functionName} \n\n#### Question: ${args.query} \n\n${functionResponse}`,
             );
           }
           allInfoGathered = false;
         }
      
         if (!allInfoGathered) {
           const updatedPrompt = dedent`
               ### Gathered Information:
               ${gatheredInformation.join("\n")}
               ### Questions for Project Owner:
               ${questionsForProjectOwner.join("\n")}
               ### Missing Information:
               Reflect on the gathered information and specify what is still needed to fully address the issue and why it is needed.
               ### Plan Information Gathering:
               Decide on the best action to obtain each missing piece of information (ResearchCodebase, ResearchInternet, AskProjectOwner, ResearchComplete).
               Choose the correct tools and formulate very specific, detailed queries to gather all of the missing information effectively. If you need to ask questions, only ask a maximum of 5 questions. Each question should be a full sentence and clearly state what information you are looking for.
               ### Important:
                   If you have all the necessary information to proceed with the task, return a single tool call to confirm that the research is complete. If you need more information, use up to 5 of the appropriate tools to gather it.
           `;
           messages.push({ role: "user", content: updatedPrompt });
         }
       } else {
         allInfoGathered = true;
       }
      }
      
      if (loops >= maxLoops) {
       console.log("Max loops reached, exiting loop.");
      }
      return `## Research: ${gatheredInformation.join("\n")} \n\n## Questions for Project Owner: \n\n [ ] ${questionsForProjectOwner.join("\n [ ] ")}`;
      };
    • Explanation:

      • The 'researchIssue' function is located in the 'src/server/agent/research.ts' file.
      • It takes the GitHub issue, source map, root directory, maximum loops, and model as input.
      • It uses a loop to iteratively gather information using the 'ResearchCodebase', 'ResearchInternet', and 'AskProjectOwner' tools.
      • It returns a string containing the gathered research information and questions for the project owner.
    • Coding Conventions:

      • The code follows TypeScript conventions.
      • It uses the 'dedent' function for multi-line string formatting.
      • It uses async/await for asynchronous operations.
      • It uses descriptive variable names and comments.
    • Similar Functions:

      • The 'researchCodebase' and 'researchInternet' functions in the same file are similar in that they gather information from different sources.
    • Areas Missing Information:

      • The specific implementation details of the 'ResearchCodebase', 'ResearchInternet', and 'AskProjectOwner' tools are not provided in the codebase.
      • The logic for determining when to use each tool is also not explicitly defined.
      • These details may need to be obtained from external documentation or by further analyzing the codebase.

ResearchCodebase

Question: Is there a file named 'research.ts' in the codebase, and what functions does it currently contain? This is needed to add a new function for retrieving research items by issue id.

  1. Detailed Specifications:

    • Title: Research on Existing Research Agent Functions
    • Description: This information is needed to determine the existing functions within the research.ts file and how they are currently used. This will guide the implementation of a new function to retrieve research items by issue ID.
  2. File Structure and File Paths:

    • Files to Modify:
      • src/server/agent/research.ts
    • New Files to Create:
      • src/server/db/migrations/[timestamp]_create_research_table.ts
      • src/server/db/tables/research.table.ts
  3. Code Snippets:

    • Existing research.ts File:

      export enum ResearchAgentActionType {
      ResearchCodebase = "ResearchCodebase",
      ResearchInternet = "ResearchInternet",
      AskProjectOwner = "AskProjectOwner",
      ResearchComplete = "ResearchComplete",
      }
      
      // ... other code ...
      
      export const researchIssue = async function (
      githubIssue: string,
      sourceMap: string,
      rootDir: string,
      maxLoops = 3,
      model: Model = "gpt-4-0125-preview",
      ): Promise<string> {
      // ... existing implementation ...
      };
      
      // ... other functions ...
    • Example Migration File (src/server/db/migrations/[timestamp]_create_research_table.ts):

      import { change } from "../dbScript";
      
      change(async (db) => {
      await db.createTable("research", (t) => ({
       id: t.identity().primaryKey(),
       todoId: t.integer().foreignKey("todos", "id").onDelete("CASCADE"),
       issueId: t.integer(),
       type: t.enum("research_agent_action_type", [
         "ResearchCodebase",
         "ResearchInternet",
         "AskProjectOwner",
         "ResearchComplete",
       ]),
       question: t.text(),
       answer: t.text(),
       ...t.timestamps(),
      }));
      });
    • Example Table File (src/server/db/tables/research.table.ts):

      import { BaseTable } from "../baseTable";
      import { ResearchAgentActionType } from "../enums";
      
      const RESEARCH_AGENT_ACTION_TYPE_VALUES = Object.values(
      ResearchAgentActionType,
      ) as [ResearchAgentActionType, ...ResearchAgentActionType[]];
      
      export class ResearchTable extends BaseTable {
      readonly table = "research";
      columns = this.setColumns((t) => ({
       id: t.identity().primaryKey(),
       todoId: t.integer().foreignKey("todos", "id").onDelete("CASCADE"),
       issueId: t.integer(),
       type: t
         .enum("research_agent_action_type", RESEARCH_AGENT_ACTION_TYPE_VALUES)
         .default(ResearchAgentActionType.ResearchCodebase),
       question: t.text(),
       answer: t.text(),
       ...t.timestamps(),
      }));
      }
    • New Function in research.ts:
      export const getResearchByIssueId = async (issueId: number) => {
      return db.research.where({ issueId }).all();
      };
    • Example Usage in agentEditFiles.ts and agentFixError.ts:

      // ... other code ...
      
      const researchItems = await getResearchByIssueId(issue.number);
      const research = researchItems
      .map((item) => `Question: ${item.question}\nAnswer: ${item.answer}`)
      .join("\n\n");
      
      // ... use the 'research' string in the prompts ...
    • Example of Enum Usage in Migration and Table Files:
      • The migration file uses db.createEnum to define the enum type.
      • The table file uses t.enum to reference the enum type in the column definition.
    • Example of Foreign Key and Delete Cascade:
      • The todoId column in the research table is a foreign key referencing the todos table.
      • The onDelete("CASCADE") option ensures that if a todo is deleted, all associated research items are also deleted.
  4. Component Breakdown:

    • Frontend: No frontend changes are required for this task.
    • Backend:
      • research.ts: This file will be modified to include the new function getResearchByIssueId.
      • agentEditFiles.ts and agentFixError.ts: These files will be updated to use the new getResearchByIssueId function and format the research data for prompts.
      • Migration File: A new migration file will be created to add the research table to the database.
      • Table File: A new table file will be created to define the Research interface and the structure of the research table.
  5. API Contracts:

    • No API changes are required for this task.
  6. Styles and Themes:

    • No style or theme changes are required for this task.

ResearchCodebase

Question: How are the 'agentEditFile' and 'agentFixError' functions implemented, and where do they currently use the research string? This is crucial for updating these functions to fetch and use research data from the database.

  1. Detailed Specifications:

    • Title: Research on agentEditFiles and agentFixError Functions and Research String Usage
    • Description: This research aims to understand the implementation of the agentEditFiles and agentFixError functions and how they currently utilize the research string. This information is crucial for updating these functions to fetch and use research data from the database.
  2. File Structure and File Paths:

    • Files to Modify:
      • src/server/code/agentEditFiles.ts
      • src/server/code/agentFixError.ts
      • src/server/agent/research.ts
      • src/server/db/migrations/[timestamp]_createResearchTable.ts (new file)
      • src/server/db/tables/research.table.ts (new file)
      • src/server/db/db.ts
    • Directory Structure:
      • src/server/code: Contains code related to code generation and modification.
      • src/server/agent: Contains code for different agents, including research and bug fixing.
      • src/server/db: Contains database-related code, including migrations and table definitions.
  3. Code Snippets:

    • src/server/code/agentEditFiles.ts:
      export async function agentEditFiles(params: EditFilesParams) {
      // ...
      const research = ""; // TODO: currently this is part of the GitHub issue, need to separate it out
      // ...
      const codeTemplateParams = {
       // ...
       research,
       // ...
      };
      // ...
      }
    • src/server/code/agentFixError.ts:
      export async function agentFixError(params: AgentFixErrorParams) {
      // ...
      // TODO: this is a temporary fix, need to figure out the right way to do research for a bugfix
      const research = ""; // TODO: currently this is part of the GitHub issue, need to separate it out
      // ...
      const projectContext: ProjectContext = {
       // ...
       research,
       // ...
      };
      // ...
      }
    • src/server/agent/research.ts:
      export const researchIssue = async function (
      // ...
      ): Promise<string> {
      // ...
      return `## Research: ${gatheredInformation.join("\n")} \n\n## Questions for Project Owner: \n\n [ ] ${questionsForProjectOwner.join("\n [ ] ")}`;
      };
    • Example migration file (src/server/db/migrations/[timestamp]_createResearchTable.ts):

      import { change } from "../dbScript";
      
      change(async (db) => {
      await db.createTable("research", (t) => ({
       id: t.identity().primaryKey(),
       todoId: t.integer().foreignKey("todos", "id"),
       issueId: t.integer(),
       type: t.enum("research_agent_action_type", ["ResearchCodebase", "ResearchInternet", "AskProjectOwner"]),
       question: t.text(),
       answer: t.text(),
       ...t.timestamps(),
      }));
      });
    • Example table definition file (src/server/db/tables/research.table.ts):

      import { BaseTable } from "../baseTable";
      import { ResearchAgentActionType } from "../enums";
      
      export class ResearchTable extends BaseTable {
      readonly table = "research";
      columns = this.setColumns((t) => ({
       id: t.identity().primaryKey(),
       todoId: t.integer().foreignKey("todos", "id"),
       issueId: t.integer(),
       type: t.enum("research_agent_action_type", Object.values(ResearchAgentActionType)),
       question: t.text(),
       answer: t.text(),
       ...t.timestamps(),
      }));
      }
    • src/server/db/db.ts:
      // ...
      export const db = orchidORM(config.database, {
      // ...
      research: ResearchTable,
      // ...
      });
    • Example function to get research by issue ID (src/server/agent/research.ts):
      export async function getResearchByIssueId(issueId: number): Promise<Research[]> {
      return db.research.where({ issueId }).all();
      }

The codebase analysis indicates that the agentEditFiles and agentFixError functions currently have a placeholder for the research variable, set to an empty string. The researchIssue function in src/server/agent/research.ts returns a string containing research information.

To implement the requested changes, you will need to:

  1. Create a new TypeScript interface named Research with the specified fields.
  2. Create a new table in the database named research with the appropriate columns and foreign keys.
  3. Update the researchIssue function to return an array of Research objects instead of a string.
  4. Modify the Todos logic to check for existing research and skip the research step if it's already done.
  5. Add a new function to src/server/agent/research.ts to fetch research items by issue ID.
  6. Update the agentEditFiles and agentFixError functions to fetch research data from the database using the new function and convert it into a string of question/answers.

@jacob-ai-bot This information should provide a good starting point for implementing the requested changes. If you encounter any further questions or need additional clarification, please let me know.

jacob-local-kevin[bot] commented 1 week ago

JACoB here...

You mentioned me on this issue and I am busy taking a look at it.

I'll continue to comment on this issue with status as I make progress.

jacob-local-kevin[bot] commented 1 week ago

I was not able to write any code that would address this issue.

Please include more specific details in issues when you mention me.

jacob-local-kevin[bot] commented 1 week ago

JACoB here...

You mentioned me on this issue and I am busy taking a look at it.

I'll continue to comment on this issue with status as I make progress.

jacob-local-kevin[bot] commented 1 week ago

Unfortunately, I ran into trouble working on this.

Here is some error information:

ENOENT: no such file or directory, open '/tmp/tmp-33781-D29prju8r8AO/src/server/db/migrations/20240314000000_createResearchTable.ts'
jacob-local-kevin[bot] commented 1 week ago

JACoB here...

You mentioned me on this issue and I am busy taking a look at it.

I'll continue to comment on this issue with status as I make progress.

jacob-local-kevin[bot] commented 1 week ago

JACoB here...

You mentioned me on this issue and I am busy taking a look at it.

I'll continue to comment on this issue with status as I make progress.

jacob-local-kevin[bot] commented 1 week ago

I was not able to write any code that would address this issue.

Please include more specific details in issues when you mention me.

jacob-local-kevin[bot] commented 1 week ago

JACoB here...

You mentioned me on this issue and I am busy taking a look at it.

I'll continue to comment on this issue with status as I make progress.

jacob-local-kevin[bot] commented 1 week ago

I was not able to write any code that would address this issue.

Please include more specific details in issues when you mention me.

jacob-local-kevin[bot] commented 1 week ago

JACoB here...

You mentioned me on this issue and I am busy taking a look at it.

I'll continue to comment on this issue with status as I make progress.

jacob-local-kevin[bot] commented 1 week ago

I was not able to write any code that would address this issue.

Please include more specific details in issues when you mention me.

jacob-local-kevin[bot] commented 1 week ago

JACoB here...

You mentioned me on this issue and I am busy taking a look at it.

I'll continue to comment on this issue with status as I make progress.

jacob-local-kevin[bot] commented 1 week ago

Update

I've completed my initial work on this issue and have created a pull request: JACoB PR for Issue Implement Structured Research Storage and Retrieval.

The changes currently result in an error, so I'll be making some additional changes before it is ready to merge.