-
Python 2.7 is pretty much at the end of it's support life-span (and way past the end of development). We dev'd for 2.7 originally b/c of external constraints in our first runtime environments. Those a…
-
The current embed is... ugly:
The custom embed should show:
1. artist name;
2. song name;
3. anime name;
4. listeners;
5. maybe who requested it as well.
…
-
Hello!
I would like to ask if there are any plans to implement support with 4 and 8 bits to be able to infer with the model?
Thank you!
-
**Describe the bug**
我使用megatron-LM V2.5 patch ,执行命令为
`FMOE_FASTER_SHADOW_ENABLE=1 FMOE_FASTER_SCHEDULE_ENABLE=1 FMOE_FASTER_GROUP_SIZE=4 bash pretrain_gpt_distributed.sh`
用单机8卡跑gpt2+moe，设置了一共16个e…
-
Modifying 2 files is all you need.
pyproject.toml
llama-cpp-python = "^0.2.11" to llama-cpp-python = "^0.2.23"
poetry.lock
Search llama-cpp-python and update 2 values
version = "0.2.23"
…
-
A new technique for broadening the capabilities of LLMs _without_ massively increasing resource requirements has recently emerged. Instead of shuffling tokens in between expert models like a standard…
-
### Your current environment
vllm: v0.4.1
GPU : V100 32G
### 🐛 Describe the bug
on vllm v0.4.1
The output results of the MoE kinds model(like mixtral-8x7b ...etc ) are incorrect o…
-
Lol, I had fun with this code. But it wasn't all that suited to my use without the position embedding interpolation and some registers. So I added them. See if you feel like you want to include them i…
-
As a user (parent role) I want to see notifications so that to understand all changes and activities at the portal
**Role**
user (parent role)
**Precondition**
Users profile is created and …
-
I have this problem after deploying to vercel, error file /characters/build/id.json does not exist. Refresh it an…