Open 0x4007 opened 6 months ago
This is awfully similar to /research
with a few changes to how the convo context is weighted.
I could add /rewrite
as a slash command to the /research
plugin
automatically called instead
then this plugin here (let's call it issue-spec-updater
) could listen for Time:
(and whatever else) changes then with it's uses: with:
config calls into /rewrite
?
Does it make sense to have both LLM QA commands in the same plugin, as /research
can already do this (minus the direct issue body change) basically.
Added as a tool
for the LLM it doesn't even need to be a slash command it can NLP any style of comment so long as it starts with /research
i.e update the spec if it needs it
/research
can be made agent-like with a couple tools, none are required for simple /research
but I had it generating permits and dispatching workflows for my permit-module
testing.
Remember invocations should be fully customizable based on a repo/org's config.
Customizable in the sense it can be active/disabled? If plugins are hosted in official org repos with specific regex handling for slash commands and then invocations shouldn't be customizable I thought
If added as a tool_call
it would be 100% customizable (minus the /research
prefix)
The config should allow us to associate any webhook event, or any slash command, to any plugin.
I think it's better to have each plugin be a one-trick pony.
Will experiment with different specs, system messages and context changes here
I think rewriting the spec is a bad idea as it loses meaningfulness compared to the comments that came later on as they wouldn't make much sense. It might be better to have the bot simply rewrite a new comment with the new specs. Also because the comment of the issuer is the one evaluated by the bot, it would be like rewarding the bot and not the actual user.
I was thinking about this myself, if the bot edits the spec it'll likely fluff up the styling and formatting too which would increase the payout where it shouldn't have been
What about posting the comment and then updating the issue body with a link to the updated spec comment?
Also, should it pull in context from linked issues and referenced PRs too? I assumed it should as a lot of direction changes occur within the PR comments and it being more context aware is typically better. Although the spec only mentions the current issue and it's comments
I think rewriting the spec is a bad idea as it loses meaningfulness compared to the comments that came later on as they wouldn't make much sense.
The whole point is to summarize and include those comments.
It might be better to have the bot simply rewrite a new comment with the new specs.
No it breaks the system of looking at the spec for the direction. Also for other functionality like relevance scoring it depends on the spec to be accurate.
Also because the comment of the issuer is the one evaluated by the bot, it would be like rewarding the bot and not the actual user.
It's fine. People already use some ChatGPT in their comments. As long as it adds value, it should be rewarded.
People already use some ChatGPT in their comments
Wouldn't be me, I feel like a true reply guy posting anything that's AI generated 😂
Understood, I'll guard rail it into not fluffing too much and have it just renew the issue spec.
Is this problem still active to b solved?
! No price label is set to calculate the duration
@0x4007 can you double check pricing and spec for this it's a little dated but still in-play isn't it?
I'm not sure on time estimate but the vector embeddings plugin took five days
Sometimes issues are submitted but not finalized until the team contributes more ideas and research. This leads to the issue specification being out of sync with the latest information, leading to the contributors having to be sure to read the full conversation before starting the task and delivering their work.
/rewrite
specification command (I'm unsure whats the best default slash command for this.)Time:
in particular) as this is what I recall having to change based on new information being added to an issue.Recommended Default Invocations
Remember invocations should be fully customizable based on a repo/org's config. However we should support defaults/recommended invocations based on our dogfooding to give the best default experience to new partners.
/rewrite
commandTime:
label change