-
Opening this up to track the development of the new caching behaviour I'm planning to implement. This will leverage 2 significant improvements
- Reduced llama state size which is now a function of …
-
When I modified “run_example.sh” and changed backend to vllm,
I got the error message down below, I will do some some check whether the error comes from server side or client side.
I notice t…
-
![QQ截图20240611082851](https://github.com/heshengtao/comfyui_LLM_party/assets/130342241/899689dc-9b08-4ca6-9ed9-7d1152917738)
-
### Description
I am using Microsoft HostedWebAssembly under .NET8, that means the solution has three projects, one client, one server and one shared. So client communicates to server though web ap…
-
### The model to consider.
https://huggingface.co/microsoft/Phi-3-medium-128k-instruct
### The closest model vllm already supports.
The older phi model (including phi-3-mini) has been supported
##…
-
CPU version works fine with the corresponding model and nuget package. DirectML version throws the exception below.
Model: https://huggingface.co/microsoft/Phi-3-mini-128k-instruct-onnx/tree/main/d…
AshD updated
3 months ago
-
After start pretrain, there is a bug
Traceback (most recent call last):
File "/data2/LLaVA-pp/LLaVA/llava/train/train_mem.py", line 4, in
train(attn_implementation="flash_attention_2")
…
-
**Describe the bug**
Running a docker build seems to fail with the error `failed to read /mistralrs/mistralrs-bench/Cargo.toml`
```
[+] Building 2.0s (18/20) …
-
With many claiming that phi3 mini is uncannily good for it's size, and with larger, actually-useful phi3 models on the way, adding support for this arch is almost certainly worthwhile.
-
```shell
🏂~/rust/mistral.rs master ❯ cargo run --release --features metal -- -i plain -m
microsoft/Phi-3-mini-128k-instruct -a phi3
Finished release [optimized] target(s) in 0.37s
Ru…