search

ubiquity-os-marketplace / command-ask

0 stars 4 forks source link

Evals #37

Open sshivaditya2019 opened 5 days ago

sshivaditya2019 commented 5 days ago

Resolves #185

github-actions[bot] commented 5 days ago

Unused dependencies (1)

Filename dependencies
package.json autoevals
sshivaditya2019 commented 3 days ago

@0x4007 Evaluations cannot be configured as action requests since repository secrets cannot be accessed in pull requests (without pull request workflow trigger which I think is not safe). I have currently implemented Levenshtein and ContextRelevance metrics. Are there any other metrics that should be added to the evals ?

0x4007 commented 3 days ago

@rndquu knows the solution for secrets and pull requests I think. I'm not sure because I never implemented evals before. I suppose you might know best off hand and without research.

ubiquity-os-beta[bot] commented 1 day ago

@sshivaditya2019, this task has been idle for a while. Please provide an update.