-
Being the only thing even tangentially related to GPT-4 made open-source, OpenAI Evals can help assess the performance of an LLM on a growing set of tasks. Maybe this could be used to measure how clos…
-
@uc-cfss/fa18 As I mentioned in-class, pick any two pull requests at random and provide feedback. Starting next week you should have your evals auto-assigned to you as I outlined in class. Also make s…
-
https://www.nytimes.com/2024/02/01/technology/perplexity-search-ai-google.html
> Perplexity also allows users to search within a specific set of sources, such as academic papers, YouTube videos or …
-
-
**Describe the bug**
I noticed while running promptflow evaluate API, it is trying to upload my entire working directory to remote, and below case it failed because there is a special file which is…
-
Automatically scrape from [all existing openai/evals](https://github.com/openai/evals/tree/main/evals/registry/data) to have all of the listed ones as zeno viewable.
Could potentially even webscrap…
-
https://firefox-ci-tc.services.mozilla.com/tasks/CEUR_rZ1Qty22JNz3JC-mw
We shouldn't run evals for the pre-trained models though. It wasn't the case before, so something got broken in training cont…
-
right now the eval works by continuing appending new style tags to the bottom of the document. unfortunately, this means that every new style that gets applied must also define a whole bunch of `none…
bling updated
10 years ago
-
After a quick investigation, I see that the original parallel corpus was filtered from 70M to 35M sentences.
Serbian is digraphic and uses both Latin and Cyrillic scripts.
I see that datasets li…
-
If the tokenizer prepends `_` as sow token, it will make single token evals fail.
Reported by @anton-l