-
### Proposal
This is a followed up to PR #2459, which implemented batch scraping support within the same region. Some Promitor deployments target resources across multiple regions, and such deploymen…
-
## Checklist
- [x ] I'm asking a question
- [x ] I've looked through the README and FAQ for similar questions
- [x ] I've searched the bugtracker for similar questions including closed …
-
so for now i work with low-vram
VRAM is 8GB and free 8GB
and the GPU load is only 40%
can i raise up the batch size
-
Assume the following scenario: node with 8 shards, batch with 8 sstables, 1 is really big, 7 are small.
As only a single shard works on load&stream of given sstable, we would end up in a situation wh…
-
We could use the batch API and reap the benefits it provides since right now we do not need the real-time embeddings that we currently have, which is awesome but more expensive than what it needs to b…
-
I cannot access the `Batch` API changes made in [PR #334](https://github.com/aallam/openai-kotlin/pull/334) from the `OpenAI` interface when using `openai-client` 3.8.2. The `OpenAI` interface does no…
-
Hey everyone! It should function finally! Please update Unsloth via (if you're on a local machine - Colab / Kaggle no need to update, just refresh)
>
> ```shell
> pip uninstall unsloth -y
> pip i…
-
While we support batched inference like other constrained decoding libraries, the current implementation can be parallelized further. In particular, we can mask logits in batch and run several `kbnf` …
-
### Where is the problem?
https://docs.konghq.com/gateway/latest/kong-plugins/queue/#queue-flush-on-shutdown
### What happened?
It is not clear whether `flushed` means sending the queue reque…
-
Need option for batch normals update. For all blocks in nif file. Like tangents update.