-
### Your current environment
```text
The output of `python collect_env.py`
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" f…
-
hope can use llama3.1 soon on ollama
-
**Describe the bug**
Traceback (most recent call last):
File "/apps1/zhangfan/anaconda3/envs/new_swift/lib/python3.10/site-packages/swift/cli/rlhf.py", line 5, in
rlhf_main()
File "/apps1…
-
The growth in size of open-source models is outpacing the growth of memory capacity of Mac computers. The latest 70B version of Llama 3 is already pushing the limits of a fully loaded Mac Pro. The upc…
-
### 🐛 Describe the bug
PR 115 - 969bd835 Do reshape_and_cache in bulk for prompt (#115) will result in HPUGraph forward replayV3 failure in TensorParallel mode.
=== Details ===
* Code Base
…
-
My environment seems to be installed successfully and I can open the chat website, but I'm unsure where to set the path for the large model?"
![image](https://github.com/user-attachments/assets/1ab…
-
I am facing difficulties in specifying GPU usage for different models for LLM inference pipeline using vLLM. Specifically, I have 4 RTX 4090 GPUs available, and I aim to run a LLM with a size of 42GB …
-
### Your current environment
```text
Collecting environment information...
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
…
-
Dear Flapi developers,
First and foremost, I want to express my sincere gratitude for developing Flapi. This library has the potential to greatly enhance workflow automation in FL Studio, and I'm…
LL1yt updated
3 weeks ago
-
Ray (https://github.com/ray-project/ray) becomes popular choice of running distributed Python ML applications. Its Python interface is easy to scale up the workload from local laptop to distributed cl…