design and deploy related entities API

[tatianab]

Design and deploy an API that returns entities (issues, docs, etc) that are related to the given document. Consider allowing the following parameters/filters:

• [x] filter: source/type of document (GitHub project, Gerrit project, crawled webpage etc)
• [x] cutoff: the similarity value (0-1) at which entities should be considered related
• [x] max number of related entities to return

[jba]

Suggested design of the API:

Endpoint POST /api/search?threshold=T&limit=L Threshold is the lowest score to return, assuming the max score is 1 and the min is 0. Limit is the max number of results to return.

The post body is the JSON for the EmbedDoc struct:

 type EmbedDoc struct {
        Title string // title of document
        Text  string // text of document
  }

This is the search query, which may itself be some document (like a GitHub issue).

The response is the JSON for this Go:

[]SearchResult

type SearchResult struct {
    Kind string // type of result, e.g. "GerritCL", "GitHubIssue", etc.
    ID string    // always a URL, for now
    Score float64
}

Responses are by decreasing score.

Notes:

• The system doesn't store the kind, but it can be determined from the ID.

• We'll omit the filter for now, since it is tricky to implement with the current structure of the vector DB and may not be important.