-
right now the eval works by continuing appending new style tags to the bottom of the document. unfortunately, this means that every new style that gets applied must also define a whole bunch of `none…
bling updated
10 years ago
-
Automatically scrape from [all existing openai/evals](https://github.com/openai/evals/tree/main/evals/registry/data) to have all of the listed ones as zeno viewable.
Could potentially even webscrap…
-
### 💻 The Brief
AI Tech to use/discover:
- [TruLens-Eval](https://lablab.ai/tech/truera/trulens)
- [GPT-4 Turbo](https://clarifai.com/openai/chat-completion/models/gpt-4-turbo)
That is the cha…
-
Hi, thank you for the wonderful paper and codebase! I had one clarification question: it looks like there is an extra set of forward passes for the SigLIP ViT blocks - is this intentional for the sigl…
-
I think should eval to true like javascript
```
{{{ if emptyArray }}}
doesnt hit there
{{{ end }}}
```
-
When running
`python evals/lm_harness_eval.py --model mamba --model_args pretrained=state-spaces/mamba-130m --tasks lambada_openai,hellaswag,piqa,arc_easy,arc_challenge,winogrande --device cuda --…
-
https://www.nytimes.com/2024/02/01/technology/perplexity-search-ai-google.html
> Perplexity also allows users to search within a specific set of sources, such as academic papers, YouTube videos or …
-
I have been working with the HyperTuning from recbole.trainer.
On the provided documentation, I cannot find what are the parameters `early_stop` and `max_evals` and what is their relation to the pa…
-
If we are running classification tasks with LLM, how can we calculate overall precision, recall and F1 score from the evals?
It is not clear if derived metrics allow us to do that. Any suggestions?…
-
related to https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard but more meaningful scores
https://github.com/h2oai/h2ogpt/blob/ba6cad3207f8319b5c5f4b1e9099d7b909fdb661/generate.py#L132…