-
Hello! First of all thank you for your amazing work and models. Also a big fan of your twitter!
I've tested out the OCR and it works great, but i've got a question about the `run_ocr` function. Jud…
-
When reading a large result set... I.e. SELECT * FROM x when x has a lot of data, buenavista will collect all the data before a client sees the first row.
-
I have implemented an inference API using ONNX Runtime and FastAPI to process multiple prompts in batches, with the goal of improving efficiency. However, I've observed that performance is significant…
-
### Tested versions
reproduced
- v4.3.stable.official [77dcf97d8]
### System information
Ubuntu 22.04.4 LTS, GTX1060 6GB, NVIDIA 535.183.01, Compatibility mode
### Issue description
noticed that…
-
With current dev branch (3383b98f1b9a2e60ec0bda2fcf463ba271926596) I'm seeing failures with the `run_test.sh` script (I'm on CentOS 9):
```
$ autoconf
$ ./configure
$ make -j$(nproc)
$ ./run…
-
**Is your feature request related to a problem? Please describe.**
In real-world use of the initial implementation of telemetry batching it was found that bursty telemetry easily overran the buffers …
-
### 🚀 The feature
Implement batched nms!
### Motivation, pitch
My motivation to create this issue was this warning:
```
[W BatchedFallback.cpp:84] Warning: There is a performance drop because w…
-
After reading through the docs it's not clear to me if there is a strict requirement that all batches be sync operations. Sometimes it's important to batch changes in async contexts. Can I use `startB…
-
InboxRemoveTable:
- needs 'if table exists' on user_role before asserting removal of rows-by-where
-
Implement a continuous batching test in Llama which demonstrates our continuous batching algorithm:
```python
while True:
if not is_full(current_batch) and not prefill_q.empty():
model_pre…