-
After releasing online FFN offloading, we have found new issues in:
- [x] Decoding bug: #77.
- [x] Python module issue: #55, #78.
- [ ] Inaccuracy when offloading under a VRAM budget: #26, #38.
Some …
-
# Summary
We enabled node status offload and workflows archiving, and we have observed some performance and stability issues.
- there are many slow queries of mysql when running thousands of wor…
-
There's a new cache technique mentioned in the paper https://arxiv.org/abs/2312.17238. (github: https://github.com/dvmazur/mixtral-offloading)
They introduced LRU cache to cache experts based on patt…
-
In addition to LTO/Graphite, I also build with [Auto-Parallelisation](https://gcc.gnu.org/wiki/AutoParInGCC) where possible. I've converted my own custom flag management hacks over to gentooLTO inclu…
-
In GitLab by @bf2799 on Feb 8, 2022, 19:33
Send what data is being offloaded before any data is offloaded so offloader knows what is coming. This will help generalize things per board.
-
Currently, we only support `Continue` and `Pause` actions, which is quite limiting.
Most notably, we require forwarding HTTP headers before processing HTTP body, which prevents building plugins tha…
-
For testing and finding optimal quality it will be nice to have all these arguments and Gradio fields:
in priority order:
seed
negative_prompt:
num_inference_steps
video_num_inference_steps
…
ptits updated
4 weeks ago
-
Вобще я новый пользователь линукса, вот установил можно сказать на днях первый раз и столкнулся с проблемой дса(не робит войс), тк на винде все просто и не нужно ничего настраивать, достаточно просто …
-
Hello,
I wanted to ask you @lsalzman, whether ENet could see an optimization with GSO and sendmmsg (instead of the plain 'sendmsg') in order to optimize throughput?
( see: https://blog.cloudflare…
-
Thanks for your excellent work! But i met some questions when i try to use your framework.
I try to run `offloading.py` and `offloading_TP.py` on RTX4090 * 4 machine. As shown in the figure below, …