ubiquity-os-marketplace / text-vector-embeddings

0 stars 6 forks source link

Task Matchmaking #7

Closed 0x4007 closed 1 month ago

0x4007 commented 1 month ago

When a new task is posted to our network, UbiquityOS should recommend qualified contributors.

For example it should look something like:

### 0x4007
- ubiquity/ubiquity-dollar#123 87% match. 
### rndquu
- ubiquity/ubiquity-dollar#250 77% match. 

Eventually this may be configurable to be network wide (global) organization wide, or repository wide. But for this first version, we will do network wide.


Another configuration of this idea could be to recommend the contributor in the organization with the highest match, even if it doesn't hit the threshold. If within the organization, perhaps tagging is appropriate. Perhaps it makes sense to handle this in a separate task.

sshivaditya2019 commented 1 month ago

@0x4007 So to sum up:

So, to find the who completed the task. Should assignees be used or is there a better way to do this ? I think theres a pull request field in the payload object.

0x4007 commented 1 month ago

First use assignees and fallback to pull author since many organizations are not as strict about assignment.

The plan is to separately build a scraper basically so when a new person joins our DevPool we can scrape all the public data on their profile to know what types of projects they completed.

sshivaditya2019 commented 1 month ago

First use assignees and fallback to pull author since many organizations are not as strict about assignment.

The plan is to separately build a scraper basically so when a new person joins our DevPool we can scrape all the public data on their profile to know what types of projects they completed.

Is scraper a part of this ticket ? If so, could you add more details about it to the issue spec. So, we would run a similarity search against the issues. Return the username of the issues.assignees which most similar (More than 75% threshold) in descending order. Is that right ?

So, right now issue-dedup would send a comment if its more than 75% similar. So is it supposed to be in the same comment ? Or Like how is it supposed to work ? Should there be two comments ?

Also, Another configuration of this idea could be to recommend the contributor in the organization with the highest match, even if it doesn't hit the threshold. So the 75% threshold is only for task similarity ?

0x4007 commented 1 month ago

First use assignees and fallback to pull author since many organizations are not as strict about assignment.

The plan is to separately build a scraper basically so when a new person joins our DevPool we can scrape all the public data on their profile to know what types of projects they completed.

Is scraper a part of this ticket ? If so, could you add more details about it to the issue spec.

Different project. Can make the spec later. I imagine we could ask them to install UbiquityOS and then we can read all the issues from their private repositories.

So, we would run a similarity search against the issues. Return the username of the issues.assignees which most similar (More than 75% threshold) in descending order. Is that right ?

Yes

So, right now issue-dedup would send a comment if its more than 75% similar. So is it supposed to be in the same comment ? Or Like how is it supposed to work ? Should there be two comments ?

Multiple comments isn't preferred. If we can edit the single comment it would look better. For issues similarity we can also consider editing the specification and adding a footer regarding similarities.

Also, Another configuration of this idea could be to recommend the contributor in the organization with the highest match, even if it doesn't hit the threshold. So the 75% threshold is only for task similarity ?

This can be handled in a separate task

sshivaditya2019 commented 1 month ago

/start

ubiquity-os[bot] commented 1 month ago
Deadline Thu, Sep 26, 4:10 PM UTC
Beneficiary 0xDAba6e01D15Db560b88C8F426b016801f79e1F69

[!TIP]

  • Use /wallet 0x0000...0000 if you want to update your registered payment wallet address.
  • Be sure to open a draft pull request as soon as possible to communicate updates on your progress.
  • Be sure to provide timely updates to us when requested, or you will be automatically unassigned from the task.
sshivaditya2019 commented 1 month ago

I attempted to address a few issues, but I’m wondering if the kernel supports issues.label_added. I’m not receiving messages from the webhook; while the webhook is invoked, it seems the kernel isn’t processing them.

Currently, when an issue is assigned, the payload doesn’t update. Is there another plugin that handles this update? If I add webhook support for issues.created and issues.updated, it might increase coupling. The intention was to keep the generate-vector-embeddings plugin standalone. Should this functionality be integrated into the generate-vector-plugin, or should it be a separate plugin? Just to clarify, the assignment feature doesn’t require embeddings.

0x4007 commented 1 month ago

I attempted to address a few issues, but I’m wondering if the kernel supports issues.label_added. I’m not receiving messages from the webhook; while the webhook is invoked, it seems the kernel isn’t processing them.

issues.label_added is certainly supported. That is how our pricing is set based on the priority and time labels.

Currently, when an issue is assigned, the payload doesn’t update. Is there another plugin that handles this update? If I add webhook support for issues.created and issues.updated, it might increase coupling. The intention was to keep the generate-vector-embeddings plugin standalone. Should this functionality be integrated into the generate-vector-plugin, or should it be a separate plugin? Just to clarify, the assignment feature doesn’t require embeddings.

We do want to decrease coupling but at this point I am leaning towards stacking tech debt so that we can ship more capabilities faster. We had quite a lull in the last couple of months and I want to speed things along. So do what you think is faster to ship and we can clean tech debt later.

You are also free to create a separate plugin if you think that makes sense! Perhaps @gentlementlegen can provide some advice on how to chain outputs together?

gentlementlegen commented 1 month ago

When multiple plugins are specified under a uses key, they will run sequentially and send their output to the following plugin.

0x4007 commented 1 month ago

I’m not receiving messages from the webhook; while the webhook is invoked, it seems the kernel isn’t processing them.

Are you using your own GitHub App? Have you correctly set the permissions? If you are using our kernel, the official instance, it definitely relays label events.

sshivaditya2019 commented 1 month ago

I’m not receiving messages from the webhook; while the webhook is invoked, it seems the kernel isn’t processing them.

Are you using your own GitHub App? Have you correctly set the permissions? If you are using our kernel, the official instance, it definitely relays label events.

I fixed that, it was issues.labled, I was using something else.

ubiquity-os[bot] commented 1 month ago

 [ 612.818 WXDAI ] 

@sshivaditya2019
Contributions Overview
ViewContributionCountReward
IssueTask1600
IssueComment412.818
ReviewComment80
Conversation Incentives
CommentFormattingRelevanceReward
@0x4007 So to sum up:- `Issues` Database, would have t…
4.13
content:
  p:
    symbols:
      \b\w+\b:
        count: 66
        multiplier: 0.1
    score: 1
  ul:
    symbols:
      \b\w+\b:
        count: 1
        multiplier: 0.1
    score: 1
  li:
    symbols:
      \b\w+\b:
        count: 2
        multiplier: 0.1
    score: 1
  code:
    symbols:
      \b\w+\b:
        count: 4
        multiplier: 0.1
    score: 1
multiplier: 1
0.83.304
Is scraper a part of this ticket ? If so, could you add more det…
6.54
content:
  p:
    symbols:
      \b\w+\b:
        count: 96
        multiplier: 0.1
    score: 1
  code:
    symbols:
      \b\w+\b:
        count: 28
        multiplier: 0.1
    score: 1
multiplier: 1
0.95.886
I attempted to address a few issues, but I’m wondering if the ke…
5.75
content:
  p:
    symbols:
      \b\w+\b:
        count: 98
        multiplier: 0.1
    score: 1
  code:
    symbols:
      \b\w+\b:
        count: 12
        multiplier: 0.1
    score: 1
multiplier: 1
0.63.45
I fixed that, it was `issues.labled`, I was using someth…
0.89
content:
  p:
    symbols:
      \b\w+\b:
        count: 10
        multiplier: 0.1
    score: 1
  code:
    symbols:
      \b\w+\b:
        count: 2
        multiplier: 0.1
    score: 1
multiplier: 1
0.20.178
Resolves #7 - Adds a `issue` matching option. - Wo…
0
content:
  p:
    symbols:
      \b\w+\b:
        count: 20
        multiplier: 0
    score: 1
  ul:
    symbols:
      \b\w+\b:
        count: 1
        multiplier: 0
    score: 1
  li:
    symbols:
      \b\w+\b:
        count: 3
        multiplier: 0
    score: 1
  code:
    symbols:
      \b\w+\b:
        count: 7
        multiplier: 0
    score: 1
multiplier: 0
0.8-
Adding the space, somehow breaks it. I tried without space, it d…
0
content:
  p:
    symbols:
      \b\w+\b:
        count: 16
        multiplier: 0.2
    score: 1
multiplier: 0
0.5-
It works well both with and without the space, based on my tests.
0
content:
  p:
    symbols:
      \b\w+\b:
        count: 13
        multiplier: 0.2
    score: 1
multiplier: 0
0.4-
Example issue:[New Issue](https://github.com/sshivaditya2019/t…
0
content:
  p:
    symbols:
      \b\w+\b:
        count: 42
        multiplier: 0.2
    score: 1
  a:
    symbols:
      \b\w+\b:
        count: 4
        multiplier: 0.2
    score: 1
  code:
    symbols:
      \b\w+\b:
        count: 8
        multiplier: 0.2
    score: 1
multiplier: 0
0.7-
If some is assigned an issue that particular issue's payload is …
0
content:
  p:
    symbols:
      \b\w+\b:
        count: 60
        multiplier: 0.2
    score: 1
multiplier: 0
0.6-
Fixed, Now, its using graphQL for query, listens only for `i…
0
content:
  p:
    symbols:
      \b\w+\b:
        count: 15
        multiplier: 0.2
    score: 1
  code:
    symbols:
      \b\w+\b:
        count: 6
        multiplier: 0.2
    score: 1
  a:
    symbols:
      \b\w+\b:
        count: 4
        multiplier: 0.2
    score: 1
multiplier: 0
0.9-
Updated the comment structure, the comments should start with &#…
0
content:
  p:
    symbols:
      \b\w+\b:
        count: 25
        multiplier: 0.2
    score: 1
  code:
    symbols:
      \b\w+\b:
        count: 1
        multiplier: 0.2
    score: 1
multiplier: 0
0.9-
Its working, you can check the updated comment in the ([Old Issu…
0
content:
  p:
    symbols:
      \b\w+\b:
        count: 10
        multiplier: 0.2
    score: 1
  a:
    symbols:
      \b\w+\b:
        count: 2
        multiplier: 0.2
    score: 1
multiplier: 0
0.6-

 [ 67.76 WXDAI ] 

@0x4007
Contributions Overview
ViewContributionCountReward
IssueSpecification151.76
IssueComment414.649
ReviewComment71.351
Conversation Incentives
CommentFormattingRelevanceReward
When a new task is posted to our network, UbiquityOS should reco…
51.76
content:
  p:
    symbols:
      \b\w+\b:
        count: 198
        multiplier: 0.1
    score: 1
  ul:
    symbols:
      \b\w+\b:
        count: 3
        multiplier: 0.1
    score: 0
  li:
    symbols:
      \b\w+\b:
        count: 27
        multiplier: 0.1
    score: 1
  code:
    symbols:
      \b\w+\b:
        count: 21
        multiplier: 0.1
    score: 5
  pre:
    symbols:
      \b\w+\b:
        count: 1
        multiplier: 0.1
    score: 0
  hr:
    symbols:
      \b\w+\b:
        count: 1
        multiplier: 0.1
    score: 0
multiplier: 3
151.76
First use assignees and fallback to pull author since many organ…
5.75
content:
  p:
    symbols:
      \b\w+\b:
        count: 52
        multiplier: 0.2
    score: 1
multiplier: 1
0.42.3
Different project. Can make the spec later. I imagine we could a…
7.31
content:
  p:
    symbols:
      \b\w+\b:
        count: 69
        multiplier: 0.2
    score: 1
multiplier: 1
0.53.655
`issues.label_added` is certainly supported. That is how…
11.01
content:
  p:
    symbols:
      \b\w+\b:
        count: 98
        multiplier: 0.2
    score: 1
  code:
    symbols:
      \b\w+\b:
        count: 4
        multiplier: 0.2
    score: 1
  a:
    symbols:
      \b\w+\b:
        count: 3
        multiplier: 0.2
    score: 1
multiplier: 1
0.77.707
Are you using your own GitHub App? Have you correctly set the pe…
3.29
content:
  p:
    symbols:
      \b\w+\b:
        count: 27
        multiplier: 0.2
    score: 1
multiplier: 1
0.30.987
Did you test this? The syntax doesn't look correct. Pretty sure …
0.32
content:
  p:
    symbols:
      \b\w+\b:
        count: 20
        multiplier: 0.1
    score: 1
multiplier: 0.25
0.70.224
```suggestionconst commentLines: string[] = [">…
0.22
content:
  pre:
    symbols:
      \b\w+\b:
        count: 1
        multiplier: 0.1
    score: 0
  code:
    symbols:
      \b\w+\b:
        count: 13
        multiplier: 0.1
    score: 1
multiplier: 0.25
0.80.176
```suggestioncommentLines.push(`> ### [$…
0.15
content:
  pre:
    symbols:
      \b\w+\b:
        count: 1
        multiplier: 0.1
    score: 0
  code:
    symbols:
      \b\w+\b:
        count: 8
        multiplier: 0.1
    score: 1
multiplier: 0.25
0.80.12
I don't understand what you mean by this.Only closed as comple…
0.7
content:
  p:
    symbols:
      \b\w+\b:
        count: 50
        multiplier: 0.1
    score: 1
multiplier: 0.25
0.60.42
I'm sorry but I have little context on the "payload object" I ha…
0.55
content:
  p:
    symbols:
      \b\w+\b:
        count: 38
        multiplier: 0.1
    score: 1
multiplier: 0.25
0.30.165
QA looking good. Please check my comment under "old issue" regar…
0.32
content:
  p:
    symbols:
      \b\w+\b:
        count: 20
        multiplier: 0.1
    score: 1
multiplier: 0.25
0.70.224
@gentlementlegen can you install this plugin
0.11
content:
  p:
    symbols:
      \b\w+\b:
        count: 6
        multiplier: 0.1
    score: 1
multiplier: 0.25
0.20.022

 [ 0 WXDAI ] 

@gentlementlegen
Contributions Overview
ViewContributionCountReward
IssueComment10
Conversation Incentives
CommentFormattingRelevanceReward
When multiple plugins are specified under a `uses` key, …
0
content:
  p:
    symbols:
      \b\w+\b:
        count: 20
        multiplier: 0.1
    score: 1
  code:
    symbols:
      \b\w+\b:
        count: 1
        multiplier: 0.1
    score: 1
multiplier: 1
--