-
Should we do these models for the launch of the Arena? We need them all to be loadable via MTEB, so it'd be great if you can help add them to https://github.com/embeddings-benchmark/mteb/tree/main/mte…
-
When running a model with ```--model-mode flashinfer``` (I have tested ```mistralai/Mistral-7B-Instruct-v0.2```), for a large batch (eg 50,000 text input), I usually see that the throughput is high th…
-
Have anyone ever ran into the issue where after finetuning the output doesn't know when to end, only ends until max new token is reached? Does it has to do with the tokenizer is not adding an eos toke…
-
### What happened?
Sample times are greatly increased with --top-k 0, especially with Gemma models.
### Name and Version
version: 3570 (4134999e)
built with Apple clang version 15.0.0 (clang…
-
### System Info
- GPU Name: EC2 g5.12xl w/ 4 NVIDIA A10G
- TensorRT-LLM: 0.8.0
- Nvidia Driver: 535.161.08
- Container: nvidia/cuda:12.1.0-devel-ubuntu22.04
- OS: Ubuntu 22.04
### Who can he…
-
```text
2024-05-16 03:24:16.542 | INFO | hydit.inference:__init__:160 - Got text-to-image model root path: ckpts/t2i
2024-05-16 03:24:21.606 | INFO | hydit.inference:__init__:172 - Loading C…
-
This is the first of several updates introducing a key component to the gameplay: Fuel.
New Features:
- Power balance is now computed at the level of each reactor.
- Reactors will consume fuel wh…
-
Radeon RX 6700 XT, Ryzen 5700X ECO, model mistral-7b-instruct-v0.2 fully oflloaded to the GPU. EDIT: ROCm 6.0
Some quantization methods implementations in the Vulkan backend provide relatively slow…
-
### Your current environment
```text
Collecting environment information...
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS…
-
Hi,
I'm using Langflow to create a ChatBot based on Mistral 7B, but i can't find any documentation or example of the module "Hugging Face API" on Langflow, and what are the exact values to put in End…