-
### Your current environment
alexr@roxy:~$ python collect_env.py
Collecting environment information...
PyTorch version: 2.0.0+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM us…
-
So transformers has introduced support for speculative decoding of ngrams.
https://github.com/huggingface/transformers/pull/27979
It's as simple as passing `prompt_lookup_num_tokens=10` to `mode…
-
Issue
---
It is expected that `llama_decode` should take more time if more tokens are present in the batch, but on my system (Apple M1 Max 32GB) with `mistral-7b-instruct-v0.2.Q4_0.gguf` model, the …
-
### Your current environment
```text
The output of `python collect_env.py`
Collecting environment information...
WARNING 07-11 22:54:46 _custom_ops.py:14] Failed to import from vllm._C with Modu…
-
### Your current environment
```text
Collecting environment information...
PyTorch version: 2.3.0+cu121 …
CsRic updated
1 month ago
-
Hi all,
It seems that some chipsets (or mainboards?) do not have an active ME watchdog timer. I have an ASUS Rampage IV Extreme and was able to wipe the ME region (all 0xFF) with almost no ill effect…
ghost updated
2 years ago
-
### System Info
using latest docker image
### Information
- [X] Docker
- [ ] The CLI directly
### Tasks
- [ ] An officially supported command
- [ ] My own modifications
### Reproduction
similar…
-
### Docker image
intelanalytics/bigdl-llm-cpu:2.5.0-SNAPSHOT
### conda env
bigdl-speculative-py39
### model
chatglm3-6b
### ipex version
IPEX 2.2.0+cpu
With export BIGDL_OPT_IPEX=true, the o…
-
As mentioned in https://github.com/syncthing/syncthing-android/issues/192 , some filenames are not accepted by windows hosts because they contain '[special characters](https://msdn.microsoft.com/en-us…
-
I find an artice [Accelerating Generative AI with PyTorch II: GPT, Fast](https://pytorch.org/blog/accelerating-generative-ai-2/)
The optimization used in this article is as shown below
![image](http…