-
I'd like to use Phi-2 to compute perplexity of the prompts over an entire dataset. Is there an API for this? In the short term, I'm happy to fork https://github.com/vllm-project/vllm/blob/d0215a58e785…
-
微调之前估计效果:
***** eval metrics *****
eval_loss = 2.5866
eval_runtime = 0:02:10.44
eval_samples = 1000
eval_samples_per_second = 7.666
e…
-
-
Hi
How can we use the login code emailed to us when logging into Perplexity.
It opens in a browser and not recognized by the app.
Regards
-
**Is your feature request related to a problem? Please describe.**
Make one for perplexity
**Describe the solution you'd like**
They are fast
**Describe alternatives you've considered**
chat.…
-
Hi, can you please include perplexity evaluation from llama.cpp to koboldcpp? There is a separate script for that called perplexity in llama.cpp. Currently looks like this script is not present comple…
-
Hi! Thanks for you work. In the implementation of VQVAE (https://github.com/deepmind/sonnet/blob/v2/sonnet/src/nets/vqvae.py#L89C1-L89C1), perplexity is used as an evaluation measure for VQ codebook. …
-
Hello,
we have noticed some unexpected behaviors when fine-tuning a llama 3 model on 1 gpu and when fine-tuning the same model on the same data set with 2 gpus in parallel mode. See the attached te…
-
Hi,
Thank you for sharing your work.
The re-produced perplexity for ptb dataset using your code is not matched with the paper. The reproduced is 27.8, while in the paper it is around 9. Please c…
-
如图。