gpt-plugin

[Keyrxng]

Resolves #16

This issue is for QA purposes to demonstrate what the GPT plugin is capable of.

[Keyrxng]

/gpt test

[ubqbot[bot]]

To test the GPT plugin, you can use the /gpt command followed by your question. For example:

/gpt What is the capital of France?

This command will trigger the plugin to process the question and provide an answer based on the context of the issue or pull request. Make sure the plugin is enabled and configured correctly with the necessary API keys.

[Keyrxng]

/gpt can you create a table and summarize all of the context you aware of, categorize them into issues and PRs and give a summary of what is each is for and what's been going on

[ubqbot[bot]]

Here is a summary of the context categorized into issues and pull requests:

Type	Number	Title/Description	Summary
Issue	#49	GPT Plugin QA Demonstration	This issue is for QA purposes to demonstrate the capabilities of the GPT plugin.
Issue	#16	QA for #15	This issue involves setting up and integrating a database using Supabase to improve chatbot memory.
Issue	#15	DAO, UbiquityDAO	This issue is related to UbiquityDAO.
PR	#1	Resolves https://github.com/ubiquibot/plugins-wishlist/issues/29	This PR involves creating a plugin to assist in research and discussion on GitHub issues and PRs.

Summary of Conversations:

• Issue #49: Demonstrates the GPT plugin's capabilities.
• Issue #16: Focuses on setting up a database with Supabase for chatbot memory.
• Issue #15: Related to UbiquityDAO.
• PR #1: Discusses the implementation details and considerations for the GPT plugin, including handling comments, context depth, and plugin configuration.

[Keyrxng]

/gpt create a detailed analysis of the conversation of each issue/PR

[ubqbot[bot]]

Here is a summary of the context categorized into issues and pull requests:

Type	ID	Title/Description	Summary
Issue	#49	Demonstrate GPT Plugin Capabilities	This issue is for QA purposes to demonstrate the capabilities of the GPT plugin.
Issue	#16	QA for #15	This issue involves setting up and integrating a database using Supabase to improve the chatbot's memory. It includes defining schemas, writing migration scripts, and implementing CRUD functionality.
Issue	#15	DAO, UbiquityDAO	This issue involves setting up the UbiquityDAO.
PR	#1	Resolves https://github.com/ubiquibot/plugins-wishlist/issues/29	This pull request involves creating a plugin that can assist in research and discussion on GitHub issues and pull requests. It includes various improvements and discussions on the plugin's functionality and integration.

Detailed Analysis of Each Issue/PR Conversation:

Issue #49 Conversation:

• Commands Executed:
  • /gpt test
  • /gpt can you create a table and summarize all of the context you aware of, categorize them into issues and PRs and give a summary of what is each is for and what's been going on
  • /gpt create a detailed analysis of the conversation of each issue/PR

Issue #16 Conversation:

• Commands Executed:
  • /agent calculate the labels needed (stick to the predefined options) and once they are set, assign the issue to me
  • /agent I reset everything for testing. Can you recalculate the labels needed (stick to the predefined options) and once they are set, assign the issue to me

Issue #15 Conversation:

• Commands Executed:
  • /help
  • /agent are you operational? (multiple times)
  • /agent set my wallet address to 0x959d25B75324fBE0ADc75a454Df286eaBc7B45a7 (multiple times)
  • /start
  • /agent can you assign me this issue?
  • /agent calculate and set the time and priority labels (multiple times)
• Link:
  • https://www.github.com/ubiquibot/ask-plugin/pull/1

Pull Request #1 Conversation:

• Discussion Points:
  • Suggestions for plugin name and functionality.
  • Whether the plugin should read its own comments.
  • Handling of context length and recursion.
  • Various code suggestions and improvements.
• Related PR for Fix:
  • ubiquity/ubiquibot-kernel/issues/75

This analysis provides a comprehensive overview of the issues and pull requests, including the commands executed and the discussions that took place.

[Keyrxng]

/gpt I want you to review the pull request code and give me your thoughts on things as it stands. What can be improved, what's been done well etc

[ubqbot[bot]]

Analysis of the Pull Request

Summary

This pull request aims to resolve the issue of integrating a database to improve the chatbot's memory by using Supabase. It includes the following key changes:

• Removal of Supabase-related code and dependencies.
• Addition of OpenAI integration for handling GPT-4o model queries.
• Refactoring of the plugin to handle GitHub issue comments and generate responses using OpenAI.

Key Changes

1. Dependencies and Configuration:

   • Removed Supabase dependencies from package.json.
   • Added OpenAI dependencies.
   • Updated .cspell.json and .github/workflows/compute.yml to reflect the new plugin name and configurations.

2. Code Refactoring:

   • Removed Supabase adapter and helper files.
   • Added new handlers for adding comments and asking GPT questions.
   • Refactored the main plugin logic to handle the new functionality.
   • Updated the context type to include the new logger and removed Supabase-related properties.

3. New Functionality:

   • Implemented ask-gpt.ts to handle GPT-4o queries.
   • Added utility functions for formatting chat history and fetching issue details.
   • Created mock data and handlers for testing the new functionality.

4. Testing:

   • Added comprehensive tests to cover the new functionality.
   • Mocked OpenAI and GitHub API responses for testing.

Improvements

• Code Quality:

  • The code is well-structured and modular, making it easy to understand and maintain.
  • The use of TypeScript ensures type safety and reduces potential runtime errors.

• Functionality:

  • The new functionality to handle GPT-4o queries is a significant improvement, providing more intelligent responses.
  • The refactoring has removed unnecessary dependencies, making the codebase cleaner and more efficient.

Recommendations

• Documentation:

  • Update the documentation to reflect the new changes, especially the removal of Supabase and the addition of OpenAI.
  • Provide clear instructions on how to configure the OpenAI API key.

• Error Handling:

  • Ensure comprehensive error handling, especially for API calls to OpenAI, to handle potential failures gracefully.

• Performance:

  • Monitor the performance of the new functionality, especially the response times for GPT-4o queries, and optimize if necessary.

Conclusion

This pull request significantly improves the chatbot's capabilities by integrating OpenAI's GPT-4o model and removing the dependency on Supabase. The changes are well-implemented, and the new functionality is thoroughly tested. With some minor improvements in documentation and error handling, this update will provide a robust and intelligent chatbot experience.

[Keyrxng]

/gpt I want you to review the pull request code for PR | #1 | Resolves ubiquibot/plugins-wishlist#29 https://github.com/ubq-testing/bot-ai/issues/1

[ubqbot[bot]]

Pull Request Review for ubiquibot/ask-plugin#1

Summary

This pull request aims to resolve the issue of creating a Next.js 13 chatbot interface that uses OpenAI's GPT-3.5-turbo. The main changes include:

• Adding a new command command-gpt.
• Integrating OpenAI's API for generating responses.
• Refactoring the code to remove Supabase dependencies.
• Adding new handlers and utilities for managing comments and issues.

Key Changes

1. Configuration Changes:

   • Updated .cspell.json to include new words.
   • Updated GitHub workflow to use command-gpt.
   • Updated package.json to include new dependencies and scripts.

2. Code Refactoring:

   • Removed Supabase-related code and dependencies.
   • Added new handlers for adding comments and asking GPT questions.
   • Added utilities for formatting chat history and fetching issues.

3. New Features:

   • Implemented ask-gpt handler to interact with OpenAI's API.
   • Added functionality to fetch and handle linked issues and comments.
   • Created mock data and handlers for testing.

Code Quality

• Modularity: The code is well-organized into different modules and handlers, making it easy to understand and maintain.
• Error Handling: There is adequate error handling, especially when dealing with API responses and missing configurations.
• Logging: The use of a logging library (@ubiquity-dao/ubiquibot-logger) ensures that all significant actions are logged, which is useful for debugging.

Improvements

1. Code Duplication:

   • There are some repetitive patterns in the code, especially in the handlers. Consider abstracting common functionalities into utility functions.

2. Type Safety:

   • Ensure that all types are correctly defined and used, especially when dealing with API responses and context objects.

3. Documentation:

   • Add more comments and documentation to explain the purpose of each function and module. This will help new contributors understand the codebase more quickly.

Testing

• The pull request includes comprehensive tests using jest and msw for mocking API responses.
• Ensure that all edge cases are covered, especially for error scenarios and API failures.

Conclusion

Overall, this pull request significantly improves the functionality and maintainability of the chatbot interface. With a few minor improvements in code duplication and documentation, it will be a robust addition to the project.

Recommendations

• Merge: The pull request is well-structured and addresses the issue effectively. It is recommended to merge after addressing the minor improvements mentioned above.

[Keyrxng]

/gpt take your time to review the pull request code and be sure to avoid mixing context together which does not belong, isolate the relevant context related to the PR (comments, diff etc) and give your review

[ubqbot[bot]]

Review of Pull Request #1

Summary

This pull request aims to resolve the issue of integrating a chatbot interface using Next.js 13 and OpenAI's GPT-3.5-turbo. It includes several changes and additions to the codebase to support this functionality.

Key Changes

1. Configuration and Setup:

   • Updated .cspell.json to include new words.
   • Modified GitHub Actions workflow in .github/workflows/compute.yml.
   • Updated eslint.config.mjs to ignore additional paths.
   • Updated package.json to include new dependencies and scripts.

2. Code Changes:

   • Removed Supabase-related code and adapters.
   • Added new handlers for adding comments and asking GPT questions.
   • Refactored the main plugin logic to handle the /gpt command.
   • Added utility functions for formatting chat history and fetching issues.

3. Testing:

   • Updated mock data and handlers for testing.
   • Added new tests to cover the functionality of the GPT plugin.

Improvements

• Code Organization:

  • The code is well-organized with clear separation of concerns. Handlers, utilities, and types are defined in separate files, making the codebase easier to navigate and maintain.

• Logging:

  • The use of the @ubiquity-dao/ubiquibot-logger for logging is a good practice, providing structured and consistent logging throughout the plugin.

• Testing:

  • Comprehensive tests have been added to cover various scenarios, ensuring the robustness of the plugin.

Suggestions

• Error Handling:

  • Ensure that all potential error points, especially in asynchronous operations, are properly handled and logged. This will help in debugging and maintaining the code.

• Documentation:

  • Add comments and documentation to the new functions and handlers. This will help other developers understand the purpose and functionality of the code.

• Performance:

  • Consider optimizing the recursive fetching of linked issues to avoid potential performance bottlenecks, especially for repositories with a large number of linked issues.

Conclusion

Overall, this pull request is well-structured and addresses the requirements effectively. With a few improvements in error handling and documentation, it will be a solid addition to the codebase.

[Keyrxng]

LOGS

I did not paste the entire diff but it does include everything even the yarn.lock. Token usage below from the latest call made ```json "tokenUsage": { "output": 473, "input": 29965, "total": 30438 } ``` ``` === Current Issue #49 Specification === ubq-testing/bot-ai/49 ===\n\nResolves #16\r\n\r\nThis issue is for QA purposes to demonstrate what the GPT plugin is capable of.\n=== End Current Issue #49 Specification ===\n\n=== Current Issue #49 Conversation === ubq-testing/bot-ai #49 ===\n\n2247719803 Keyrxng: /gpt test\n2247724843 Keyrxng: /gpt can you create a table and summarize all of the context you aware of, categorize them into issues and PRs and give a summary of what is each is for and what's been going on \n2247728345 Keyrxng: /gpt create a detailed analysis of the conversation of each issue/PR\n2247730679 Keyrxng: /gpt I want you to review the pull request code and give me your thoughts on things as it stands. What can be improved, what's been done well etc\n2247736440 Keyrxng: /gpt I want you to review the pull request code for\r\n `PR | #1 | Resolves ubiquibot/plugins-wishlist#29\r\nhttps://github.com/ubq-testing/bot-ai/issues/1`\n2247744130 Keyrxng: /gpt take your time to review the pull request code and be sure to avoid mixing context together which does not belong, isolate the relevant context related to the PR (comments, diff etc) and give your review\n=== End Current Issue #49 Conversation ===\n\n=== Linked Issue #1 Specification === ubq-testing/bot-ai/1 ===\n\nWe need a nextjs 13 chatbot interface, it should just be a very simple text input at the bottom of the screen with a white background. The text responses should be centred. We will be using OpenAI's GPT-3.5-turbo.\n=== End Linked Issue #1 Specification ===\n\n=== Linked Issue #1 Conversation === ubq-testing/bot-ai #1 ===\n\n1763357909 Keyrxng: /ask test\n=== End Linked Issue #1 Conversation ===\n\n=== Linked Issue #16 Specification === ubq-testing/bot-ai/16 ===\n\nRelated to #15 \r\n\r\nWe need to setup and integrate a database to improve our chatbot 'memory'.\r\n\r\nWe'll use Supabase.\r\n\r\nWe'll need the schema's defined, there are about eight off the top of my head with no less than 4 columns a piece. \r\n\r\nWe'll need the migrations scripts written.\r\n\r\nWe'll need functionality to work with the database such as updating, creating, deleting, fetching, …etc.\r\n\r\nThis is foundational for the next stages of development.\r\n\n=== End Linked Issue #16 Specification ===\n\n=== Linked Issue #16 Conversation === ubq-testing/bot-ai #16 ===\n\n1774226378 Keyrxng: /agent calculate the labels needed (stick to the predefined options) and once they are set, assign the issue to me\n1774229815 Keyrxng: /agent I reset everything for testing. Can you recalculate the labels needed (stick to the predefined options) and once they are set, assign the issue to me\n=== End Linked Issue #16 Conversation ===\n\n=== Linked Issue #15 Specification === ubq-testing/bot-ai/15 ===\n\nDAO, UbiquityDAO.\n=== End Linked Issue #15 Specification ===\n\n=== Linked Issue #15 Conversation === ubq-testing/bot-ai #15 ===\n\n1774140523 Keyrxng: /help\n1774141222 Keyrxng: /agent are you operational?\n1774141632 Keyrxng: /agent are you operational?\n1774142790 Keyrxng: /agent are you operational?\n1774151530 Keyrxng: /agent set my wallet address to 0x959d25B75324fBE0ADc75a454Df286eaBc7B45a7\n1774152530 Keyrxng: /agent set my wallet address to 0x959d25B75324fBE0ADc75a454Df286eaBc7B45a7\n1774164766 Keyrxng: /start\n1774175050 Keyrxng: /agent can you assign me this issue?\n1774194241 Keyrxng: /agent calculate and set the time and priority labels\n1774206262 Keyrxng: /agent first calculate the time and priority then set the labels\n1774207392 Keyrxng: /agent first calculate the time and priority then set the labels\n1774208472 Keyrxng: /agent first calculate the time and priority then set the labels\n2227348291 Keyrxng: https://www.github.com/ubiquibot/ask-plugin/pull/1\n=== End Linked Issue #15 Conversation ===\n\n=== Linked Pull #1 Request === ubiquibot/ask-plugin/1 ===\n\nResolves https://github.com/ubiquibot/plugins-wishlist/issues/29\r\n\r\n- [worker logs](https://github.com/ubq-testing/bot-ai/issues/25#issuecomment-2226731460)\r\n\r\nI followed your prompt template and kept the system message short and sweet.\r\n\r\nIt seems it's able to lose the question being asked so I think it might be better to prioritize the question.\r\n\r\nI think filling the chat history slightly would do the trick\r\n\r\n1. system\r\n2. user - long prompt\r\n3. assistant - manually inserted short acknowledgement of the context received\r\n4. user - directly ask the question\r\n5. assistant - the real API response\r\n\r\n

---

\r\n\r\n- Should this plugin be able to read it's own comments or not?\r\n- Are we only going one level deep with the linked issue context?\r\n- Are there to be any kind of safe guards, formatting or anything (not including the spec prompt template) included in the system message or it's free reign with little guidance like it is now?\n=== End Linked Pull #1 Request ===\n\n=== Linked Issue #1 Conversation === ubiquibot/ask-plugin #1 ===\n\n1676753876 0x4007: ```suggestion\nname: \"command-gpt\"\n```\n\n1676754128 0x4007: Definitely just ignore all bot comments as we do in conversation rewards. \n1676754228 0x4007: Will this be a problem? It is not clear to me if the kernel will invoke this on any new comment. \n1676754270 0x4007: Shouldn't this be fully removed instead of `never`?\n1676754450 0x4007: Use a `string[]` as the buffer and `.join(\"\\n\")` at the end. += is too error prone. \n1676754745 0x4007: It would be useful context to explain the purpose of the plugin, and that it is being invoked inside of GitHub comments in order to help the contributors with their research. \n1676754853 0x4007: Why not all? Context length is not a problem anymore. \n1676754902 0x4007: I think that recursion is not a problem due to generous context length for 4o\n1676754978 0x4007: Just be sure to label the section with the org/repo name and ChatGPT should know it's not as relevant \n1676837551 Keyrxng: > Should this plugin be able to read it's own comments or not?\r\n\r\nWithout it seeing it's own answers it may try to re-answer a previous question and would be without relevant context. Originally I had it able to read it's own comments and had planned to do the same here\n1676837939 Keyrxng: Truthfully me either.\r\n\r\nThe kernel does match using the regex provided in the plugin config so as far as I'm aware this probably isn't required it just felt like a good to have, I'll confirm and remove if not needed.\n1676841685 Keyrxng: While we have a 128k window I don't think it's going to be as effective stuffing one prompt/chat history with near that amount of tokens. If we are adding more context which could potentially be 30/40k worth of tokens GPT will get lost in the sauce with it a little I think.\r\n\r\nAlso typically all issues are going to have at least one other linked issue which could easily produce deep enough recursion to max our tokens. Also that GPT output has to be factored into this 128k (likely always < 10k I'd imagine).\r\n\r\n

---

\r\n\r\nOne deep seems ideal to me, it obtains strictly the most relevant context without adding too much noise but if not one level deep I'd suggest to cap it somehow whether through a depth value or token limit\n1676842330 Keyrxng: I'm of a different mindset especially when linking to known 3rd party issues for popular packages which can often have hundreds of comments none of which are truly relevant other the what our commenter had to say before linking the issue.\r\n\r\nLike you always say signal over noise, I don't feel it's a benefit and will cause more issues than it will solve. But I'll remove this restriction and see how it fairs. \n1676842567 0x4007: I prefer to exclude for now. \n1676842699 0x4007: @gentlementlegen might know. \n1676842973 0x4007: I prefer to try utilizing the most amount of context. We can make assumed optimizations later. \n1676843159 0x4007: I prefer maximizing signal to noise in the context of human communication. \n \nThese LLMs on the other hand work their magic by dealing with a ton of data. I've seen better results when providing more context than less. \n1678675275 gentlementlegen: I think the kernel will have to be changed because currently plugins can only have one command which is problematic for the ones like `start stop` which is only working for one of the two commands.\r\n\r\nSecond, the plugin can be set in the wildcard, or the wrong event, so it is possible that it is called from the wrong event, or the wrong command payload.\r\n\r\nTL;DR it is not required but definitely safer to have the checks there.\n1680522525 Keyrxng: > ones like start stop which is only working for one of the two commands.\r\n\r\nWasn't it the case originally that the kernel was matching the command regex as opposed to `startsWith`? With regex `/(start|stop)` is technically one command so it shouldn't be treated like two as far as the kernel is concerned but the plugin would obviously have to split it.\r\n\r\n> TL;DR it is not required but definitely safer to have the checks there.\r\n\r\nI confirmed locally for me that it does fire on each comment so I'd say it is required or this plugin at least would comment an error for various reasons\n1680547173 gentlementlegen: Related PR for the fix is [here](https://github.com/ubiquity/ubiquibot-kernel/issues/75).\n1689084951 0x4007: `enabled` seems like a pointless config. Please remove that.\n1689085626 0x4007: If it is not a pull shouldn't this revert or throw? Doesn't make sense that it would treat an issue as a pull.\n1689086710 0x4007: You can recycle strings with a const in the beginning of this file.\r\n```suggestion\r\n```\n1689087649 0x4007: Why? Can't you just write the code to be in line with our styles?\n1689088291 0x4007: ```suggestion\r\n http.get(\"https://api.github.com/orgs/:org/repos\", ({ params: { org } }: { params: { org: string } }) => HttpResponse.json(db.repo.findMany({ where: { owner: { login: { equals: org } } } }))),\r\n```\r\n\r\nYou can make these all one liners succienct like this. \n1689089521 0x4007: Why not just use the correct object? Why cast as `any`?\n1689091491 0x4007: ```suggestion\r\n content: `You are a GitHub integrated chatbot tasked with assisting in research and discussion on GitHub issues and pull requests. Using the provided context, address the question being asked providing a clear and concise answer with no follow-up statements. The LAST comment in 'Issue Conversation' is the most recent one, focus on it as that is the question being asked. Use GitHub flavoured markdown in your response making effective use of lists, code blocks and other supported GitHub md features.`,\r\n```\r\n\r\nSeem unnecessary. \n1689573988 Keyrxng: Are you sure? In order to 'disable' they'd need to remove/comment out the plugin config completely from their `ubiquibot-config.yml`\r\n\r\nI get what you mean tho either install it if you want it or don't but this way it can be configured on a per repo basis \n1689585772 Keyrxng: just above it we return early if `!isPull` and we know it's a pull only if we can get a diff string from `octokit.pulls.get().data.diff`\n1689625716 Keyrxng: the `jest` import from `@jest/globals` isn't the same as the namespace through `@types/jest` and types as unknown\n1689626893 Keyrxng: Yeah it just means casting `issue_number` and `pull_number`\n=== End Linked Issue #1 Conversation ===\n\n=== Linked Pull Request Code Diff === ubiquibot/ask-plugin #1 ===\n\ndiff --git a/.cspell.json b/.cspell.json\nindex 213394b..b201567 100644\n--- a/.cspell.json\n+++ b/.cspell.json\n@@ -4,7 +4,25 @@\n \"ignorePaths\": [\"**/*.json\", \"**/*.css\", \"node_modules\", \"**/*.log\", \"./src/adapters/supabase/**/**.ts\"],\n \"useGitignore\": true,\n \"language\": \"en\",\n- \"words\": [\"Nektos\", \"dataurl\", \"devpool\", \"outdir\", \"servedir\", \"Supabase\", \"SUPABASE\", \"typebox\", \"ubiquibot\", \"Smee\"],\n+ \"words\": [\n+ \"mswjs\",\n+ \"Nektos\",\n+ \"dataurl\",\n+ \"devpool\",\n+ ```