Enhance Patchwork's Extensibility: Decoupling and Generalizing Core Components

[Greyisheep]

While attempting to extend the library by creating a GenerateUnitTests patchflow, several limitations in the current architecture were identified. These limitations hinder the easy extension of the library, which is one of the short-term goals of Patchwork.

Current Limitations

1. Tight Coupling: The CallCode2Prompt step is tightly coupled to the GenerateREADME patchflow. This makes it difficult to reuse or adapt for other patchflows.

2. Limited Functionality: The current CallCode2Prompt.py in the main branch has limited functionality, focusing only on README.md file handling.

3. Inflexible ModifyCode Class: The ModifyCode class in ModifyCode.py is not designed to handle multiple modes of operation, making it challenging to add new patchflow types.

4. Lack of Standardized Extension Method: There's no clear, standardized base model to add new patchflows to the library.

Proposed Solutions

1. Generalize CallCode2Prompt:

   • Refactor CallCode2Prompt to accept a mode parameter.
   • Implement separate handling methods for different modes (e.g., README, unit tests).
   • Create a flexible file selection mechanism based on the mode.

2. Enhance ModifyCode Class:

   • Implement a mode-based execution in the ModifyCode class.
   • Add separate methods for handling different types of modifications (e.g., updating README, creating unit test files).
   • Ensure the class can easily accommodate new modes in the future.

3. Standardize Patchflow Creation:

   • Create a base Patchflow class that new patchflows can inherit from.
   • Implement common methods and attributes in the base class.
   • Provide clear documentation on how to create new patchflows.

4. Update Existing Patchflows:

   • Refactor GenerateREADME to use the new generalized components.
   • Create a new GenerateUnitTests patchflow using the generalized components.

5. Improve Documentation:

   • Update the contributor guidelines to reflect the new extension process.
   • Provide examples of how to create new patchflows using the generalized components.

Discussion Points

1. Are there any other components that need generalization to improve extensibility?
2. How can we ensure backward compatibility while making these changes?
3. What level of abstraction is appropriate to balance flexibility and ease of use?

By addressing these points, we can significantly improve Patchwork's extensibility, making it easier for contributors to add new features and patchflows in the future.

Some Changes Already Being Worked on

Here's a link to a branch with the proposed changes already been worked.

[CTY-git]

Hi @Greyisheep , thank you for contributing to the project and attempting to add a new Patchflow.

In general, patchflows follow these series of patterns: Data Extraction -> Transform data into prompts -> Call LLMs with prompts -> Transform/Parse LLM response -> Load response into a series of actions.

For the creation of a unit test creating Patchflow, I believe the Step: ExtractCodeContexts would be used to extract for classes or methods in codes to generate unit tests for.

---

1. Tight Coupling: The CallCode2Prompt step is tightly coupled to the GenerateREADME patchflow. This makes it difficult to reuse or adapt for other patchflows.

2. Limited Functionality: The current CallCode2Prompt.py in the main branch has limited functionality, focusing only on README.md file handling.

This is because CallCode2Prompt is something we classify internally as an external tool use, something similar to ScanSemgrep and ScanDepscan. Each external tool used "Step" is indeed made for a specific patchflow in mind at the point of writing. This leads to the proposed solution:

1. Generalize CallCode2Prompt:
   • Refactor CallCode2Prompt to accept a mode parameter.
   • Implement separate handling methods for different modes (e.g., README, unit tests).
   • Create a flexible file selection mechanism based on the mode.

With the context of ScanSemgrep and ScanDepscan, perhaps a base class that encapsulates the idea of a external tool use would be appropriate. The base class handles how it will run an external tool, usually another CLI, and its mode handles how we handle the outputs of such an external tool from files or stdout.

---

2. Enhance ModifyCode Class:
   • Implement a mode-based execution in the ModifyCode class.
   • Add separate methods for handling different types of modifications (e.g., updating README, creating unit test files).
   • Ensure the class can easily accommodate new modes in the future.

Yes, you are right we should allow another mode of operations other than just making code changes in place. I believe for the creation of a unit test Patchflow, you would need a way to create new files rather than just editing current files.

---

3. Standardize Patchflow Creation:
   • Create a base Patchflow class that new patchflows can inherit from.
   • Implement common methods and attributes in the base class.
   • Provide clear documentation on how to create new patchflows.

This is currently in the works. The planned changes are to add additional status values, rather than just "No errors" -> Success and "an exception has been thrown" -> Failure, and to refactor some of the more common operations of setting up a step/patchflow. Do you have any specific functionality or feature you are looking for?