search

ubiquity / ubiquibot

Putting the 'A' in 'DAO'
https://github.com/marketplace/ubiquibot
MIT License
17 stars 60 forks source link

Proposal: Crediting "On Topic Comment Incentives" Only #740

Open 0x4007 opened 1 year ago

0x4007 commented 1 year ago

This requires some experimentation but it would make sense to only offer credit to comment incentives that are on topic.

When the comment incentives are being calculated, we should rely on ChatGPT to filter out any comments that are off topic.

In order to save on network requests, we can consider the following:

  1. Pass in every comment from users in a single array, and associate each node ID.
  2. We tell ChatGPT to consider the issue specification as the topic of conversation, and then ask it to go through every comment to see if any are off topic.
  3. Filter out off topic comments (referenced via comment node IDs)
  4. We should be left with an array of "on topic" node IDs which can be passed into the comment calculation function.

This could help make dealing with abuse/spam comments hands off and seamless.

I'm not sure how to handle long conversations that exceed the context window.

Perhaps if the network request fails for any reason, we simply assume everything is on topic and calculate as normal (with a warning that the bot could not verify on-topic conversations)

kamaalsultan commented 1 year ago

I'm not sure how to handle long conversations that exceed the context window.

I think we can request several times in case the conversation is too long. We can trim the issue specification (this is called summarizing) to reduce the size of the request, split the comments into convenient sizes and we can send requests several times. I don't see any necessity of sending request exactly once...

0x4007 commented 1 year ago

Instant runs (with caching if necessary) scales much better.

kamaalsultan commented 1 year ago

Yes, so I am talking in only case the conversation is too long... And I am not sure what you mean by instant runs. Obviously, we need to use GPT API, isn't it? I don't think caching solves multiple request problems in this case since the issues and comments are all different.