-
Hi, I am trying to obtain the prompt logprobs of 'meta-llama/llama-2-13b-chat-hf' but run into a size mismatch error.
Here are the sampling params:
```
SamplingParams(temperature=0, max_tokens=5…
-
### Your current environment
The output of `python collect_env.py`
```text
Your output of `python collect_env.py` here
```
### 🐛 Describe the bug
(base) bob@test-ESC8000A-E11:~$ python…
-
When I use `logprobs > 0`, the `IndexError` occurs after 500~1000 iters.
## Trace
```
Traceback (most recent call last):
File "/path/to/repo/src/vllm-generate.py", line 179, in
outputs…
-
### Motivation.
TLDR; There is high CPU overhead associated with each decode batch due to the processing and generation of input/output. Multi-step decoding will be able to amortize all these overh…
-
### Your current environment
The output of `python collect_env.py`
```text
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A…
-
Hey,
In my workflow the graphics API developer is entirely separate from the GLSL shader writer (who is usually an artist with limited GPU hardware knowledge). They don't want to have to think about …
-
I use example [search engine response time](https://github.com/sqshq/sampler#runchart) but get error
` FAILED TO PARSE A NUMBER
strconv.ParseFloat: parsing "D:\\utilites\\sampler>": invalid syntax
…
qw569 updated
3 years ago
-
It would be very helpful to do full volume validation during training. This would involve running one of the Aggregators instead of the Samplers mid-training and using the new `niftynet.engine.signal`…
-
Hi,
I'm not a contributor, but I'm a member of MIT's Venture team. I'm trying to do some comparisons of BLOG and Venture on open-universe problems. I'm using the latest version of BLOG downloaded from…
-
** Feature request**
Some rail estimators also have an "engine". This is a means to create samples of photometry. In the first instance this would be a means to sample uniformly from the lephare mo…