-
/kind feature
**Describe the solution you'd like**
[A clear and concise description of what you want to happen.]
Support the ollama server as a runtime (not sure if it has been asked elsewhere)…
-
### Your current environment
Collecting environment information...
INFO 08-28 14:32:56 importing.py:10] Triton not installed; certain GPU-related functions will not be available.
WARNING 08-28 14:3…
-
This 1-element (scalar) kernel works on CPU, but gives a `Error: CUDA error: CUDA_ERROR_ILLEGAL_ADDRESS cuLaunchKernel failed` on CUDA using both Li2018 and Anderson2021 autoschedulers.
```py
impo…
-
### Your current environment
```text
python collect_env.py
Collecting environment information...
PyTorch version: 2.2.1+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to…
-
### Your current environment
```text
Collecting environment information...
WARNING 07-23 19:11:42 _custom_ops.py:14] Failed to import from vllm._C with ModuleNotFoundError("No module named 'vllm.…
-
### Contact Details
github
### What happened?
I came here to report the issue / bug / my incompetence around the error of: `llama_model_load: error loading model: done_getting_tensors: wrong numbe…
-
Hi, I'm getting a compilation error using icc 2021.4.0 in a Centos 8 container.
```
# icc -V
Intel(R) C Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.4.0 Buil…
-
There are two variants:
* AVX512_VNNI (Tiger Lake, Rocket Lake) - 512bit/256bit/128bit
* AVX_VNNI - (upcoming Alder Lake) - 256bit/128bit
VNNI replaces 3 simd instructions with one instruction.
…
-
Command used : python3 build_ngtf.py --target_arch skylake-avx512 --build_plaidml_backend
OS : Ubuntu 16.04
Build error screenshot attached :
![error](https://user-images.githubusercontent.com/288…
-
### Problem description
Here is my CMAKE_CXX_FLAGS opinion.
Setting CMAKE_CXX_FLAGS=-mavx2 -mfma -mavx -mf16c -mlzcnt -std=c++17 -mbmi2 -mavx512f
```
FAILED: velox/common/base/tests/CMakeFiles/v…