-
I think BatchedDataLoader is dealing with the case files are larger than memory, so it streams rows from disk into memory, and shuffles data in the meanwhile.
However, if in-memory cache option is …
-
Some developers prefer not to constantly use `this` in classes, as classes in general can lead to larger files and complicate code splitting. They much prefer a functional approach. I believe it would…
-
### Motivation
在vllm部署推理时，基于kv cache的长度限制，很可能会出现如下情况：
> ValueError: The model's max seq len (19008) is larger than the maximum number of tokens that can be stored in KV cache (3840). Try increas…
-
### System information
Type | Version/Name
--- | ---
Distribution Name | Proxmox VE (Debian GNU/Linux 12 (bookworm))
Distribution Version | proxmox-ve 8.2.4
Kernel Version | Linux erp…
-
> One should never rely on the number of bytes actually allocated corresponding to the number requested.
The number of bytes allocated is guaranteed to be the same (or more? I guess it's rounded up…
-
Hello, @Snosixtyboo @ameuleman my device is 4090 24G.
First,when using the SIBR viewer to view my trained model (model size is 4G), I found that the gpu memory is about 22G, if this is the case, if…
-
An interesting and counterintuitive observation we should make is that trying to achieve the highest possible levels of compression for call_genotype is actually pointless. From @benjeffery's experime…
-
### What is the issue?
**Description:**
I encountered an issue where the **LLaMA 3.2 Vision 11b** model loads entirely in CPU RAM, without utilizing the GPU memory as expected. The issue occurs on m…
-
**Severity**: Medium
**Vulnerability Details**:
Even after fixing the dynamic size allocation, there is a bug where retData is still pre-allocated to a fixed size (2 * 32 bytes). This allocation s…
-
Hi, this is great news after 14 years!
i've tried the executable on an Epyc 7282 server 2019 with 128 GB RAM and on djvu files larger than ~ 25MB it gives for example:
Processing page 81 of 81
…