Rewrite Specification

[0x4007]

Sometimes issues are submitted but not finalized until the team contributes more ideas and research. This leads to the issue specification being out of sync with the latest information, leading to the contributors having to be sure to read the full conversation before starting the task and delivering their work.

• A simple tech solution to this would be some type of /rewrite specification command (I'm unsure whats the best default slash command for this.)
• This will feed the entire GitHub conversation into an LLM to rewrite the issue specification.
• This command can directly edit the issue specification with the results (we have revision history built in so no information is lost.)
• We can also place more weight on the issue author's comments as they likely have the clearest vision.
• It would be nice to think of a way for this to be automatically called instead of relying on manual invocation.
• Perhaps when a pricing related label is adjusted (like Time: in particular) as this is what I recall having to change based on new information being added to an issue.

Recommended Default Invocations

Remember invocations should be fully customizable based on a repo/org's config. However we should support defaults/recommended invocations based on our dogfooding to give the best default experience to new partners.

• Listen for /rewrite command
• Listen for Time: label change

[Keyrxng]

This is awfully similar to /research with a few changes to how the convo context is weighted.

I could add /rewrite as a slash command to the /research plugin

automatically called instead

then this plugin here (let's call it issue-spec-updater) could listen for Time: (and whatever else) changes then with it's uses: with: config calls into /rewrite?

---

Does it make sense to have both LLM QA commands in the same plugin, as /research can already do this (minus the direct issue body change) basically.

Added as a tool for the LLM it doesn't even need to be a slash command it can NLP any style of comment so long as it starts with /research i.e update the spec if it needs it

/research can be made agent-like with a couple tools, none are required for simple /research but I had it generating permits and dispatching workflows for my permit-module testing.

---

Remember invocations should be fully customizable based on a repo/org's config.

Customizable in the sense it can be active/disabled? If plugins are hosted in official org repos with specific regex handling for slash commands and then invocations shouldn't be customizable I thought

If added as a tool_call it would be 100% customizable (minus the /research prefix)

[0x4007]

The config should allow us to associate any webhook event, or any slash command, to any plugin.

I think it's better to have each plugin be a one-trick pony.

[Keyrxng]

• https://github.com/ubq-testing/issue-spec-rewrite/issues

Will experiment with different specs, system messages and context changes here

[gentlementlegen]

I think rewriting the spec is a bad idea as it loses meaningfulness compared to the comments that came later on as they wouldn't make much sense. It might be better to have the bot simply rewrite a new comment with the new specs. Also because the comment of the issuer is the one evaluated by the bot, it would be like rewarding the bot and not the actual user.

[Keyrxng]

I was thinking about this myself, if the bot edits the spec it'll likely fluff up the styling and formatting too which would increase the payout where it shouldn't have been

What about posting the comment and then updating the issue body with a link to the updated spec comment?

Also, should it pull in context from linked issues and referenced PRs too? I assumed it should as a lot of direction changes occur within the PR comments and it being more context aware is typically better. Although the spec only mentions the current issue and it's comments

[0x4007]

I think rewriting the spec is a bad idea as it loses meaningfulness compared to the comments that came later on as they wouldn't make much sense.

The whole point is to summarize and include those comments.

It might be better to have the bot simply rewrite a new comment with the new specs.

No it breaks the system of looking at the spec for the direction. Also for other functionality like relevance scoring it depends on the spec to be accurate.

Also because the comment of the issuer is the one evaluated by the bot, it would be like rewarding the bot and not the actual user.

It's fine. People already use some ChatGPT in their comments. As long as it adds value, it should be rewarded.

[Keyrxng]

People already use some ChatGPT in their comments

Wouldn't be me, I feel like a true reply guy posting anything that's AI generated 😂

Understood, I'll guard rail it into not fluffing too much and have it just renew the issue spec.