search

intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, Phi, MiniCPM, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, GraphRAG, DeepSpeed, vLLM, FastChat, Axolotl, etc.
Apache License 2.0
6.55k stars 1.25k forks source link

Ollama returns incorrect result #12134

Open yunbiaolin opened 3 days ago

yunbiaolin commented 3 days ago

followed all steps in https://github.com/intel-analytics/ipex-llm/blob/main/docs/mddocs/Quickstart/ollama_quickstart.md.
below are the failure case.

my environment is: OS: windows 11 Graphics: ARC 770 driver 31.0.101.5534. oneAPI: 2024.2

(llm-cpp) C:\Users\SAS>ollama.exe run qwen2:latest

你好 _REGSITER: 你好！有什么我能为你做的吗？

你好

当然，"Regsiter"的意思是注册。如果您在尝试进行账户注册时遇到问题，或者有任何与注册相关的问题，随时可以问我。无论是提 供技术支持、解释步骤、解答疑问还是提供帮助，我都尽力而为。请具体说明您需要的帮助内容，我会为您提供详细的指导和答案。

time=2024-09-27T13:46:13.117+08:00 level=INFO source=gpu.go:168 msg="looking for compatible GPUs" time=2024-09-27T13:46:13.132+08:00 level=INFO source=gpu.go:280 msg="no compatible GPUs were discovered" time=2024-09-27T13:46:13.153+08:00 level=INFO source=memory.go:309 msg="offload to cpu" layers.requested=-1 layers.model=29 layers.offload=0 layers.split="" memory.available="[15.6 GiB]" memory.required.full="4.9 GiB" memory.required.partial="0 B" memory.required.kv="448.0 MiB" memory.required.allocations="[4.9 GiB]" memory.weights.total="3.9 GiB" memory.weights.repeating="3.4 GiB" memory.weights.nonrepeating="426.4 MiB" memory.graph.full="478.0 MiB" memory.graph.partial="730.4 MiB" time=2024-09-27T13:46:13.158+08:00 level=INFO source=server.go:395 msg="starting llama server" cmd="C:\Users\SAS\dist\windows-amd64\lib\ollama\runners\cpu_avx2\ollama_llama_server.exe --model C:\Users\SAS\.ollama\models\blobs\sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 --ctx-size 8192 --batch-size 512 --embedding --log-disable --n-gpu-layers 999 --no-mmap --parallel 4 --port 11250" time=2024-09-27T13:46:13.163+08:00 level=INFO source=sched.go:450 msg="loaded runners" count=1 time=2024-09-27T13:46:13.163+08:00 level=INFO source=server.go:595 msg="waiting for llama runner to start responding" time=2024-09-27T13:46:13.163+08:00 level=INFO source=server.go:629 msg="waiting for server to become available" status="llm server error" INFO [wmain] build info build=1 commit="1810c22" tid="8200" timestamp=1727415973 INFO [wmain] system info n_threads=12 n_threads_batch=-1 system_info="AVX = 0 AVX_VNNI = 0 AVX2 = 0 AVX512 = 0 AVX512_VBMI = 0 AVX512_VNNI = 0 AVX512_BF16 = 0 FMA = 0 NEON = 0 SVE = 0 ARM_FMA = 0 F16C = 0 FP16_VA = 0 WASM_SIMD = 0 BLAS = 1 SSE3 = 0 SSSE3 = 0 VSX = 0 MATMUL_INT8 = 0 LLAMAFILE = 1 " tid="8200" timestamp=1727415973 total_threads=20 INFO [wmain] HTTP server listening hostname="127.0.0.1" n_threads_http="19" port="11250" tid="8200" timestamp=1727415973 llama_model_loader: loaded meta data with 21 key-value pairs and 339 tensors from C:\Users\SAS.ollama\models\blobs\sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = qwen2 llama_model_loader: - kv 1: general.name str = Qwen2-7B-Instruct llama_model_loader: - kv 2: qwen2.block_count u32 = 28 llama_model_loader: - kv 3: qwen2.context_length u32 = 32768 llama_model_loader: - kv 4: qwen2.embedding_length u32 = 3584 llama_model_loader: - kv 5: qwen2.feed_forward_length u32 = 18944 llama_model_loader: - kv 6: qwen2.attention.head_count u32 = 28 llama_model_loader: - kv 7: qwen2.attention.head_count_kv u32 = 4 llama_model_loader: - kv 8: qwen2.rope.freq_base f32 = 1000000.000000 llama_model_loader: - kv 9: qwen2.attention.layer_norm_rms_epsilon f32 = 0.000001 llama_model_loader: - kv 10: general.file_type u32 = 2 llama_model_loader: - kv 11: tokenizer.ggml.model str = gpt2 llama_model_loader: - kv 12: tokenizer.ggml.pre str = qwen2 llama_model_loader: - kv 13: tokenizer.ggml.tokens arr[str,152064] = ["!", "\"", "#", "$", "%", "&", "'", ... llama_model_loader: - kv 14: tokenizer.ggml.token_type arr[i32,152064] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... llama_model_loader: - kv 15: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",... llama_model_loader: - kv 16: tokenizer.ggml.eos_token_id u32 = 151645 llama_model_loader: - kv 17: tokenizer.ggml.padding_token_id u32 = 151643 llama_model_loader: - kv 18: tokenizer.ggml.bos_token_id u32 = 151643 llama_model_loader: - kv 19: tokenizer.chat_template str = {% for message in messages %}{% if lo... llama_model_loader: - kv 20: general.quantization_version u32 = 2 llama_model_loader: - type f32: 141 tensors llama_model_loader: - type q4_0: 197 tensors llama_model_loader: - type q6_K: 1 tensors llm_load_vocab: special tokens cache size = 421 time=2024-09-27T13:46:13.428+08:00 level=INFO source=server.go:629 msg="waiting for server to become available" status="llm server loading model" llm_load_vocab: token to piece cache size = 0.9352 MB llm_load_print_meta: format = GGUF V3 (latest) llm_load_print_meta: arch = qwen2 llm_load_print_meta: vocab type = BPE llm_load_print_meta: n_vocab = 152064 llm_load_print_meta: n_merges = 151387 llm_load_print_meta: vocab_only = 0 llm_load_print_meta: n_ctx_train = 32768 llm_load_print_meta: n_embd = 3584 llm_load_print_meta: n_layer = 28 llm_load_print_meta: n_head = 28 llm_load_print_meta: n_head_kv = 4 llm_load_print_meta: n_rot = 128 llm_load_print_meta: n_swa = 0 llm_load_print_meta: n_embd_head_k = 128 llm_load_print_meta: n_embd_head_v = 128 llm_load_print_meta: n_gqa = 7 llm_load_print_meta: n_embd_k_gqa = 512 llm_load_print_meta: n_embd_v_gqa = 512 llm_load_print_meta: f_norm_eps = 0.0e+00 llm_load_print_meta: f_norm_rms_eps = 1.0e-06 llm_load_print_meta: f_clamp_kqv = 0.0e+00 llm_load_print_meta: f_max_alibi_bias = 0.0e+00 llm_load_print_meta: f_logit_scale = 0.0e+00 llm_load_print_meta: n_ff = 18944 llm_load_print_meta: n_expert = 0 llm_load_print_meta: n_expert_used = 0 llm_load_print_meta: causal attn = 1 llm_load_print_meta: pooling type = 0 llm_load_print_meta: rope type = 2 llm_load_print_meta: rope scaling = linear llm_load_print_meta: freq_base_train = 1000000.0 llm_load_print_meta: freq_scale_train = 1 llm_load_print_meta: n_ctx_orig_yarn = 32768 llm_load_print_meta: rope_finetuned = unknown llm_load_print_meta: ssm_d_conv = 0 llm_load_print_meta: ssm_d_inner = 0 llm_load_print_meta: ssm_d_state = 0 llm_load_print_meta: ssm_dt_rank = 0 llm_load_print_meta: ssm_dt_b_c_rms = 0 llm_load_print_meta: model type = ?B llm_load_print_meta: model ftype = Q4_0 llm_load_print_meta: model params = 7.62 B llm_load_print_meta: model size = 4.12 GiB (4.65 BPW) llm_load_print_meta: general.name = Qwen2-7B-Instruct llm_load_print_meta: BOS token = 151643 '< endoftext >' llm_load_print_meta: EOS token = 151645 '< im_end >' llm_load_print_meta: PAD token = 151643 '< endoftext >' llm_load_print_meta: LF token = 148848 'ÄĬ' llm_load_print_meta: EOT token = 151645 '< im_end >' llm_load_print_meta: max token length = 256 ggml_sycl_init: GGML_SYCL_FORCE_MMQ: no ggml_sycl_init: SYCL_USE_XMX: yes ggml_sycl_init: found 1 SYCL devices: llm_load_tensors: ggml ctx size = 0.30 MiB llm_load_tensors: offloading 28 repeating layers to GPU llm_load_tensors: offloading non-repeating layers to GPU llm_load_tensors: offloaded 29/29 layers to GPU llm_load_tensors: SYCL0 buffer size = 3928.07 MiB llm_load_tensors: SYCL_Host buffer size = 292.36 MiB llama_new_context_with_model: n_ctx = 8192 llama_new_context_with_model: n_batch = 512 llama_new_context_with_model: n_ubatch = 512 llama_new_context_with_model: flash_attn = 0 llama_new_context_with_model: freq_base = 1000000.0 llama_new_context_with_model: freq_scale = 1 [SYCL] call ggml_check_sycl ggml_check_sycl: GGML_SYCL_DEBUG: 0 ggml_check_sycl: GGML_SYCL_F16: no found 1 SYCL devices: Max Max Global compute Max work sub mem ID Device Type Name Version units group group size Driver version
0 [level_zero:gpu:0] Intel Arc A770 Graphics 1.3 512 1024 32 16704M 1.3.29283

llama_kv_cache_init: SYCL0 KV buffer size = 448.00 MiB llama_new_context_with_model: KV self size = 448.00 MiB, K (f16): 224.00 MiB, V (f16): 224.00 MiB llama_new_context_with_model: SYCL_Host output buffer size = 2.38 MiB llama_new_context_with_model: SYCL0 compute buffer size = 500.00 MiB llama_new_context_with_model: SYCL_Host compute buffer size = 23.01 MiB llama_new_context_with_model: graph nodes = 1014 llama_new_context_with_model: graph splits = 2 INFO [wmain] model loaded | tid="8200" timestamp=1727415979 time=2024-09-27T13:46:19.445+08:00 level=INFO source=server.go:634 msg="llama runner started in 6.28 seconds" [GIN] 2024/09/27 - 13:46:19 | 200 | 6.3390885s | 127.0.0.1 | POST "/api/chat" INFO [print_timings] prompt eval time = 214.18 ms / 20 tokens ( 10.71 ms per token, 93.38 tokens per second) | n_prompt_tokens_processed=20 n_tokens_second=93.3776565943301 slot_id=0 t_prompt_processing=214.184 t_token=10.7092 task_id=4 tid="8200" timestamp=1727415983 INFO [print_timings] generation eval time = 312.33 ms / 13 runs ( 24.03 ms per token, 41.62 tokens per second) | n_decoded=13 n_tokens_second=41.62290924925078 slot_id=0 t_token=24.02523076923077 t_token_generation=312.32800000000003 task_id=4 tid="8200" timestamp=1727415983 INFO [print_timings] total time = 526.51 ms | slot_id=0 t_prompt_processing=214.184 t_token_generation=312.32800000000003 t_total=526.5120000000001 task_id=4 tid="8200" timestamp=1727415983 [GIN] 2024/09/27 - 13:46:23 | 200 | 550.0716ms | 127.0.0.1 | POST "/api/chat" INFO [print_timings] prompt eval time = 699.86 ms / 43 tokens ( 16.28 ms per token, 61.44 tokens per second) | n_prompt_tokens_processed=43 n_tokens_second=61.44059623097663 slot_id=0 t_prompt_processing=699.863 t_token=16.275883720930235 task_id=25 tid="8200" timestamp=1727415998 INFO [print_timings] generation eval time = 1699.65 ms / 66 runs ( 25.75 ms per token, 38.83 tokens per second) | n_decoded=66 n_tokens_second=38.83159267777368 slot_id=0 t_token=25.75222727272727 t_token_generation=1699.647 task_id=25 tid="8200" timestamp=1727415998 INFO [print_timings] total time = 2399.51 ms | slot_id=0 t_prompt_processing=699.863 t_token_generation=1699.647 t_total=2399.51 task_id=25 tid="8200" timestamp=1727415998 [GIN] 2024/09/27 - 13:46:38 | 200 | 2.4166555s | 127.0.0.1 | POST "/api/chat"

(llm-cpp) C:\Users\SAS>set ADVISOR_2024_DIR=C:\Program Files (x86)\Intel\oneAPI\advisor\2024.2\ ALLUSERSPROFILE=C:\ProgramData APM=C:\Program Files (x86)\Intel\oneAPI\advisor\2024.2\perfmodels APPDATA=C:\Users\SAS\AppData\Roaming CLASSPATH=c:\Program Files (x86)\Intel\oneAPI\dal\latest\share\java\onedal.jar; CMAKE_PREFIX_PATH=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env..;c:\Program Files (x86)\Intel\oneAPI\ipp\latest\lib\cmake\ipp;c:\Program Files (x86)\Intel\oneAPI\dpl\latest\lib\cmake\oneDPL;c:\Program Files (x86)\Intel\oneAPI\dnnl\latest\env..\lib\cmake;c:\Program Files (x86)\Intel\oneAPI\dal\latest;c:\Program Files (x86)\Intel\oneAPI\compiler\latest; CMPLR_ROOT=c:\Program Files (x86)\Intel\oneAPI\compiler\latest CommandPromptType=Native CommonProgramFiles=C:\Program Files\Common Files CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files CommonProgramW6432=C:\Program Files\Common Files COMPUTERNAME=DESKTOP-1NCU6BB ComSpec=C:\Windows\system32\cmd.exe CONDA_DEFAULT_ENV=llm-cpp CONDA_EXE=C:\Users\SAS\miniforge-pypy3\Scripts\conda.exe CONDA_PREFIX=C:\Users\SAS\miniforge-pypy3\envs\llm-cpp CONDA_PREFIX_1=C:\Users\SAS\miniforge-pypy3 CONDA_PROMPT_MODIFIER=(llm-cpp) CONDA_PYTHON_EXE=C:\Users\SAS\miniforge-pypy3\python.exe CONDA_SHLVL=2 CPATH=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env..\include;c:\Program Files (x86)\Intel\oneAPI\ocloc\latest\include;c:\Program Files (x86)\Intel\oneAPI\mkl\latest\include;c:\Program Files (x86)\Intel\oneAPI\ippcp\latest\include;c:\Program Files (x86)\Intel\oneAPI\ipp\latest\include;c:\Program Files (x86)\Intel\oneAPI\dpl\latest\include;c:\Program Files (x86)\Intel\oneAPI\dpcpp-ct\latest\env..\include;c:\Program Files (x86)\Intel\oneAPI\dev-utilities\latest\include;c:\Program Files (x86)\Intel\oneAPI\dal\latest\include\dal;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\include; DALROOT=c:\Program Files (x86)\Intel\oneAPI\dal\latest DAL_MAJOR_BINARY=2 DAL_MINOR_BINARY=0 DevEnvDir=C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\ DIAGUTIL_PATH=c:\Program Files (x86)\Intel\oneAPI\debugger\latest\env\..\etc\debugger\sys_check; DNNLROOT=c:\Program Files (x86)\Intel\oneAPI\dnnl\latest\env.. DPL_ROOT=c:\Program Files (x86)\Intel\oneAPI\dpl\latest DriverData=C:\Windows\System32\Drivers\DriverData EFC_7224=1 ERRORSTATE=0 ExtensionSdkDir=C:\Program Files (x86)\Microsoft SDKs\Windows Kits\10\ExtensionSDKs EXTERNAL_INCLUDE=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\include;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ATLMFC\include;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\shared;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\winrt;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\cppwinrt;C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um FPS_BROWSER_APP_PROFILE_STRING=Internet Explorer FPS_BROWSER_USER_PROFILE_STRING=Default Framework40Version=v4.0 FrameworkDir=C:\Windows\Microsoft.NET\Framework64\ FrameworkDir64=C:\Windows\Microsoft.NET\Framework64\ FrameworkVersion=v4.0.30319 FrameworkVersion64=v4.0.30319 FSHARPINSTALLDIR=C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\FSharp\Tools HOMEDRIVE=C: HOMEPATH=\Users\SAS HTMLHelpDir=C:\Program Files (x86)\HTML Help Workshop INCLUDE=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env..\include;c:\Program Files (x86)\Intel\oneAPI\ocloc\latest\include;c:\Program Files (x86)\Intel\oneAPI\mkl\latest\include;c:\Program Files (x86)\Intel\oneAPI\ippcp\latest\include;c:\Program Files (x86)\Intel\oneAPI\ipp\latest\include;c:\Program Files (x86)\Intel\oneAPI\dpcpp-ct\latest\env..\include;c:\Program Files (x86)\Intel\oneAPI\dnnl\latest\env..\include;c:\Program Files (x86)\Intel\oneAPI\dev-utilities\latest\include;c:\Program Files (x86)\Intel\oneAPI\dal\latest\include\dal;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\include;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\include;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ATLMFC\include;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\shared;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\winrt;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\cppwinrt;C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um INTELGTDEBUGGERROOT=c:\Program Files (x86)\Intel\oneAPI\debugger\latest\env\.. INTEL_PYTHONPATH=C:\Program Files (x86)\Intel\oneAPI\advisor\2024.2\pythonapi INTEL_TARGET_ARCH=intel64 IPPCP_TARGET_ARCH=intel64 IPPCP_TARGET_BIN_ARCH=bin IPPCP_TARGET_LIB_ARCH=lib IPPCRYPTOROOT=c:\Program Files (x86)\Intel\oneAPI\ippcp\latest IPPROOT=c:\Program Files (x86)\Intel\oneAPI\ipp\latest IPP_TARGET_ARCH=intel64 LIB=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env..\lib\;c:\Program Files (x86)\Intel\oneAPI\mkl\latest\lib;c:\Program Files (x86)\Intel\oneAPI\ippcp\latest\lib;c:\Program Files (x86)\Intel\oneAPI\ipp\latest\lib;c:\Program Files (x86)\Intel\oneAPI\dnnl\latest\env..\lib;c:\Program Files (x86)\Intel\oneAPI\dal\latest\lib;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\lib\clang\18\lib\windows;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\opt\compiler\lib;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\lib;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ATLMFC\lib\x64;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\lib\x64;C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64;C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64;C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\um\x64 LIBPATH=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ATLMFC\lib\x64;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\lib\x64;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\lib\x86\store\references;C:\Program Files (x86)\Windows Kits\10\UnionMetadata\10.0.22621.0;C:\Program Files (x86)\Windows Kits\10\References\10.0.22621.0;C:\Windows\Microsoft.NET\Framework64\v4.0.30319 LIBRARY_PATH=c:\Program Files (x86)\Intel\oneAPI\ippcp\latest\lib;c:\Program Files (x86)\Intel\oneAPI\ipp\latest\lib; LOCALAPPDATA=C:\Users\SAS\AppData\Local LOGONSERVER=\DESKTOP-1NCU6BB MKLROOT=c:\Program Files (x86)\Intel\oneAPI\mkl\latest NETFXSDKDir=C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\ NLSPATH=c:\Program Files (x86)\Intel\oneAPI\mkl\latest\share\locale\1033; NUMBER_OF_PROCESSORS=20 OCLOC_ROOT=c:\Program Files (x86)\Intel\oneAPI\ocloc\latest OCL_ICD_FILENAMES=;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\bin\intelocl64_emu.dll;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\bin\intelocl64.dll OLLAMA_HOST=0.0.0.0 OLLAMA_NUM_GPU=999 ONEAPI_ROOT=c:\Program Files (x86)\Intel\oneAPI OneDrive=C:\Users\SAS\OneDrive opensslIncludeDir=C:\Program Files\OpenSSL\include OS=Windows_NT Path=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env..\bin\;c:\Program Files (x86)\Intel\oneAPI\ocloc\latest\bin;c:\Program Files (x86)\Intel\oneAPI\mkl\latest\bin;c:\Program Files (x86)\Intel\oneAPI\ippcp\latest\bin;c:\Program Files (x86)\Intel\oneAPI\ipp\latest\bin;c:\Program Files (x86)\Intel\oneAPI\dpcpp-ct\latest\env..\bin;c:\Program Files (x86)\Intel\oneAPI\dnnl\latest\env..\bin;c:\Program Files (x86)\Intel\oneAPI\dev-utilities\latest\bin;c:\Program Files (x86)\Intel\oneAPI\debugger\latest\env\..\opt\debugger\bin;c:\Program Files (x86)\Intel\oneAPI\dal\latest\bin;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\lib\ocloc;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\bin;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\bin\HostX64\x64;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\VC\VCPackages;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\TestWindow;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer;C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\bin\Roslyn;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\x64\;C:\Program Files (x86)\HTML Help Workshop;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\FSharp\Tools;C:\Program Files\Microsoft Visual Studio\2022\Community\Team Tools\DiagnosticsHub\Collector;C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\x64;C:\Program Files (x86)\Windows Kits\10\bin\x64;C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\amd64;C:\Windows\Microsoft.NET\Framework64\v4.0.30319;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\Tools\;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Library\mingw-w64\bin;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Library\usr\bin;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Library\bin;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Scripts;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\bin;C:\Users\SAS\miniforge-pypy3\condabin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files\dotnet;C:\Program Files\Git\cmd;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files\PuTTY;C:\Program Files\Rust stable MSVC 1.78\bin;C:\Program Files\CMake\bin;C:\Strawberry\c\bin;C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\Users\SAS\AppData\Local\bin\NASM;C:\Strawberry\perl\bin;C:\Users\SAS\miniforge3\Scripts;C:\Program Files\nodejs;C:\Program Files\Docker\Docker\resources\bin;C:\Users\SAS\bin;C:\Users\SAS.cargo\bin;C:\Users\SAS\AppData\Local\Microsoft\WindowsApps;C:\Users\SAS\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\SAS\AppData\Local\Microsoft\WinGet\Links;C:\Users\SAS\AppData\Roaming\npm;C:\Users\SAS.dotnet\tools;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\VC\Linux\bin\ConnectionManagerExe;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\vcpkg;C:\Program Files (x86)\Intel\oneAPI\advisor\2024.2\bin64;C:\Program Files (x86)\Intel\oneAPI\vtune\2024.2\bin64 PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PKG_CONFIG_PATH=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env..\lib\pkgconfig;c:\Program Files (x86)\Intel\oneAPI\mkl\latest\lib\pkgconfig;c:\Program Files (x86)\Intel\oneAPI\ippcp\latest\lib\pkgconfig;c:\Program Files (x86)\Intel\oneAPI\dpl\latest\lib\pkgconfig;c:\Program Files (x86)\Intel\oneAPI\dnnl\latest\env..\lib\pkgconfig;c:\Program Files (x86)\Intel\oneAPI\dal\latest\lib\pkgconfig;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\lib\pkgconfig; Platform=x64 PROCESSOR_ARCHITECTURE=AMD64 PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 151 Stepping 2, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=9702 ProgramData=C:\ProgramData ProgramFiles=C:\Program Files ProgramFiles(x86)=C:\Program Files (x86) ProgramW6432=C:\Program Files PROMPT=(llm-cpp) $P$G PSModulePath=C:\Program Files\WindowsPowerShell\Modules;C:\Windows\system32\WindowsPowerShell\v1.0\Modules PUBLIC=C:\Users\Public PYTHONPATH=C:\Program Files (x86)\Intel\oneAPI\advisor\2024.2\pythonapi SESSIONNAME=Console SETVARS_COMPLETED=1 SSL_CERT_DIR=C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Library\ssl\certs SSL_CERT_FILE=C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Library\ssl\cacert.pem SYCL_CACHE_PERSISTENT=1 SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1 SystemDrive=C: SystemRoot=C:\Windows TBBROOT=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env.. TBB_BIN_DIR=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env..\bin TBB_DLL_PATH=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env..\bin\ TBB_SCRIPT_DIR=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env\ TBB_TARGET_ARCH=intel64 TEMP=C:\Users\SAS\AppData\Local\Temp TMP=C:\Users\SAS\AppData\Local\Temp UCRTVersion=10.0.22621.0 UniversalCRTSdkDir=C:\Program Files (x86)\Windows Kits\10\ USERDOMAIN=DESKTOP-1NCU6BB USERDOMAIN_ROAMINGPROFILE=DESKTOP-1NCU6BB USERNAME=SAS USERPROFILE=C:\Users\SAS USE_INTEL_LLVM=0 VARSDIR=c:\Program Files (x86)\Intel\oneAPI\ocloc\latest\env\ VCIDEInstallDir=C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\VC\ VCINSTALLDIR=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\ VCPKG_ROOT=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\vcpkg VCToolsInstallDir=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ VCToolsRedistDir=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Redist\MSVC\14.38.33135\ VCToolsVersion=14.39.33519 VisualStudioVersion=17.0 VS170COMNTOOLS=C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\Tools\ VSCMD_ARG_app_plat=Desktop VSCMD_ARG_HOST_ARCH=x64 VSCMD_ARG_TGT_ARCH=x64 VSCMD_VER=17.9.6 VSINSTALLDIR=C:\Program Files\Microsoft Visual Studio\2022\Community\ VS_TARGET_ARCH=amd64 VTUNE_PROFILER_2024_DIR=C:\Program Files (x86)\Intel\oneAPI\vtune\2024.2\ VTUNE_PROFILER_DIR=C:\Program Files (x86)\Intel\oneAPI\vtune\2024.2\ windir=C:\Windows WindowsLibPath=C:\Program Files (x86)\Windows Kits\10\UnionMetadata\10.0.22621.0;C:\Program Files (x86)\Windows Kits\10\References\10.0.22621.0 WindowsSdkBinPath=C:\Program Files (x86)\Windows Kits\10\bin\ WindowsSdkDir=C:\Program Files (x86)\Windows Kits\10\ WindowsSDKLibVersion=10.0.22621.0\ WindowsSdkVerBinPath=C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\ WindowsSDKVersion=10.0.22621.0\ WindowsSDK_ExecutablePath_x64=C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\x64\ WindowsSDK_ExecutablePath_x86=C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\ ZES_ENABLE_SYSMAN=1 CONDA_OPENSSL_CERT_DIR_SET="1" CONDA_OPENSSL_CERT_FILE_SET="1" DOTNET_ADD_64BIT=1 DOTNET_PREFERRED_BITNESS=64 __MS_VC_INSTALL_PATH=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ __VSCMD_PREINIT_PATH=C:\Users\SAS\miniforge-pypy3\envs\llm-cpp;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Library\mingw-w64\bin;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Library\usr\bin;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Library\bin;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Scripts;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\bin;C:\Users\SAS\miniforge-pypy3\condabin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files\dotnet;C:\Program Files\Git\cmd;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files\PuTTY;C:\Program Files\Rust stable MSVC 1.78\bin;C:\Program Files\CMake\bin;C:\Strawberry\c\bin;C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\Users\SAS\AppData\Local\bin\NASM;C:\Strawberry\perl\bin;C:\Users\SAS\miniforge3\Scripts;C:\Program Files\nodejs;C:\Program Files\Docker\Docker\resources\bin;C:\Users\SAS\bin;C:\Users\SAS.cargo\bin;C:\Users\SAS\AppData\Local\Microsoft\WindowsApps;C:\Users\SAS\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\SAS\AppData\Local\Microsoft\WinGet\Links;C:\Users\SAS\AppData\Roaming\npm;C:\Users\SAS.dotnet\tools

sgwhat commented 1 day ago

Hi @yunbiaolin , we cannot reproduce your issue, ollama works fine on our windows arc device. Could you please provide more details how you install and run ollama?