-
Tracking updates of cloud.google.com
-
I am training model based on "mistralai/Mistral-7B-Instruct-v0.1"
but I am unable to serve it using docker image vllm/vllm-openai:latest
I am exe using
python3 -m vllm.entrypoints.openai.api_se…
-
**Describe the bug**
Following the instructions in [`examples/mistral`](https://github.com/microsoft/Olive/tree/main/examples/mistral) does not result in a quantized onnx model. After running the wor…
-
hi
在 megatron 中使用分布式优化器时,优化器需要执行内存操作和通信同步,这使得flash checkpoint可能会降级到分钟级别的效率。请问,对于这个问题,我们有什么优化方案和计划吗?
when use distributed optimizer in megatron, the optimizer needs to perform memory operations a…
FKyms updated
4 months ago
-
Hello @sgugger ,
Came across multiple related issues regarding this - https://github.com/huggingface/accelerate/issues/242, https://github.com/huggingface/accelerate/issues/154 . They were all clos…
-
**想要在8张40G 的A100 上微调Mixtral 8x7b. 发现OOM,之前在单卡A100 80G 上用相同的配置都没问题呀**
使用的命令是: NPROC_PER_NODE=8 xtuner train mixtral_config.py -- deepspeed deepspeed_zero2 > output.txt 2>&1
从sample output之后开始报错…
-
### Your current environment
Please note this is inside a Docker container built locally, so the collect_env.py was run as `docker exec -it vllm-openai sh -c '/usr/bin/python3 /vllm-workspace/collect…
-
The goal of this feature is to provide search and autocompletion across packages. We will add a new configuration, called `related_deps`, which is a list of package names we find related. We will impr…
-
Traceback (most recent call last):
File "/icooper/tools/miniconda3/envs/internLM/lib/python3.10/site-packages/datasets/utils/py_utils.py", line 649, in iflatmap_unordered
yield queue.get(timeo…
-
This has been around some of the tech news sources but [lifehacker](https://lifehacker.com/tech/slacks-ai-features-analyze-private-conversations) seems to sum it up quite nicely.
We should have a d…