-
I've been reading your paper, interesting work.
I have a question about how you compute perplexities, especially over datasets that are already tokenized (e.g., wikitext-103). I understand that you…
-
Measuring perplexity takes forever, is that working?
-
When I run predict.py, an error message 'No module named 'perplexity'' appears. how can I download this library?
-
Does the TSNE algorithm implemented in cmml has a maximal perplexity? I used the following code to create a TSNE object, the perplexity parameter seems to have no impact on the result.
from cuml.m…
-
`bash scripts/llama_7b.sh`
the source model: wikitext perplexity is 5.67702
prune this model, sparsity 50%, get wikitext perplexity is 7.09153509
but the paper is : 50% 7.26
why?
-
I need to get ppl per sentence for millions of lines. Splitting them into files each containing one sentence would be time consuming. Is it possible to achieve this by modifying dataloader? For exampl…
-
**Current Documentation**:
[API documentation](https://github.com/ollama/ollama/blob/4759d879f2376ffb9b82f296e442ec8ef137f27b/docs/api.md?plain=1#L79) states:
> A stream of JSON objects is retur…
-
The sensitivity scores in `results/sensitivities/ppl_sentivities_ppl.csv` represent the importance of each layer: how the perplexity performance drops when a layer is dropped. For each layer, we have …
-
Hello, I'm opening this issue because I'm still having problems with reproducing the llama 2-7b results (both without pruning and using wanda). Here are my intermediate and final perplexity results wi…
-
https://www.perplexity.ai/ https://api.perplexity.ai/chat/completions
Perplexity is a popular AI service with its LLMs. Having it with llm as a plugin will add much value to the project.
I tried t…