search

intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, GraphRAG, DeepSpeed, Axolotl, etc
Apache License 2.0
6.74k stars 1.27k forks source link

Ollama returns incorrect result #12134

Open yunbiaolin opened 1 month ago

yunbiaolin commented 1 month ago

followed all steps in https://github.com/intel-analytics/ipex-llm/blob/main/docs/mddocs/Quickstart/ollama_quickstart.md.
below are the failure case.

my environment is: OS: windows 11 Graphics: ARC 770 driver 31.0.101.5534. oneAPI: 2024.2

(llm-cpp) C:\Users\SAS>ollama.exe run qwen2:latest

你好 _REGSITER: 你好！有什么我能为你做的吗？

你好

当然，"Regsiter"的意思是注册。如果您在尝试进行账户注册时遇到问题，或者有任何与注册相关的问题，随时可以问我。无论是提 供技术支持、解释步骤、解答疑问还是提供帮助，我都尽力而为。请具体说明您需要的帮助内容，我会为您提供详细的指导和答案。

time=2024-09-27T13:46:13.117+08:00 level=INFO source=gpu.go:168 msg="looking for compatible GPUs" time=2024-09-27T13:46:13.132+08:00 level=INFO source=gpu.go:280 msg="no compatible GPUs were discovered" time=2024-09-27T13:46:13.153+08:00 level=INFO source=memory.go:309 msg="offload to cpu" layers.requested=-1 layers.model=29 layers.offload=0 layers.split="" memory.available="[15.6 GiB]" memory.required.full="4.9 GiB" memory.required.partial="0 B" memory.required.kv="448.0 MiB" memory.required.allocations="[4.9 GiB]" memory.weights.total="3.9 GiB" memory.weights.repeating="3.4 GiB" memory.weights.nonrepeating="426.4 MiB" memory.graph.full="478.0 MiB" memory.graph.partial="730.4 MiB" time=2024-09-27T13:46:13.158+08:00 level=INFO source=server.go:395 msg="starting llama server" cmd="C:\Users\SAS\dist\windows-amd64\lib\ollama\runners\cpu_avx2\ollama_llama_server.exe --model C:\Users\SAS\.ollama\models\blobs\sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 --ctx-size 8192 --batch-size 512 --embedding --log-disable --n-gpu-layers 999 --no-mmap --parallel 4 --port 11250" time=2024-09-27T13:46:13.163+08:00 level=INFO source=sched.go:450 msg="loaded runners" count=1 time=2024-09-27T13:46:13.163+08:00 level=INFO source=server.go:595 msg="waiting for llama runner to start responding" time=2024-09-27T13:46:13.163+08:00 level=INFO source=server.go:629 msg="waiting for server to become available" status="llm server error" INFO [wmain] build info build=1 commit="1810c22" tid="8200" timestamp=1727415973 INFO [wmain] system info n_threads=12 n_threads_batch=-1 system_info="AVX = 0 AVX_VNNI = 0 AVX2 = 0 AVX512 = 0 AVX512_VBMI = 0 AVX512_VNNI = 0 AVX512_BF16 = 0 FMA = 0 NEON = 0 SVE = 0 ARM_FMA = 0 F16C = 0 FP16_VA = 0 WASM_SIMD = 0 BLAS = 1 SSE3 = 0 SSSE3 = 0 VSX = 0 MATMUL_INT8 = 0 LLAMAFILE = 1 " tid="8200" timestamp=1727415973 total_threads=20 INFO [wmain] HTTP server listening hostname="127.0.0.1" n_threads_http="19" port="11250" tid="8200" timestamp=1727415973 llama_model_loader: loaded meta data with 21 key-value pairs and 339 tensors from C:\Users\SAS.ollama\models\blobs\sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = qwen2 llama_model_loader: - kv 1: general.name str = Qwen2-7B-Instruct llama_model_loader: - kv 2: qwen2.block_count u32 = 28 llama_model_loader: - kv 3: qwen2.context_length u32 = 32768 llama_model_loader: - kv 4: qwen2.embedding_length u32 = 3584 llama_model_loader: - kv 5: qwen2.feed_forward_length u32 = 18944 llama_model_loader: - kv 6: qwen2.attention.head_count u32 = 28 llama_model_loader: - kv 7: qwen2.attention.head_count_kv u32 = 4 llama_model_loader: - kv 8: qwen2.rope.freq_base f32 = 1000000.000000 llama_model_loader: - kv 9: qwen2.attention.layer_norm_rms_epsilon f32 = 0.000001 llama_model_loader: - kv 10: general.file_type u32 = 2 llama_model_loader: - kv 11: tokenizer.ggml.model str = gpt2 llama_model_loader: - kv 12: tokenizer.ggml.pre str = qwen2 llama_model_loader: - kv 13: tokenizer.ggml.tokens arr[str,152064] = ["!", "\"", "#", "$", "%", "&", "'", ... llama_model_loader: - kv 14: tokenizer.ggml.token_type arr[i32,152064] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... llama_model_loader: - kv 15: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",... llama_model_loader: - kv 16: tokenizer.ggml.eos_token_id u32 = 151645 llama_model_loader: - kv 17: tokenizer.ggml.padding_token_id u32 = 151643 llama_model_loader: - kv 18: tokenizer.ggml.bos_token_id u32 = 151643 llama_model_loader: - kv 19: tokenizer.chat_template str = {% for message in messages %}{% if lo... llama_model_loader: - kv 20: general.quantization_version u32 = 2 llama_model_loader: - type f32: 141 tensors llama_model_loader: - type q4_0: 197 tensors llama_model_loader: - type q6_K: 1 tensors llm_load_vocab: special tokens cache size = 421 time=2024-09-27T13:46:13.428+08:00 level=INFO source=server.go:629 msg="waiting for server to become available" status="llm server loading model" llm_load_vocab: token to piece cache size = 0.9352 MB llm_load_print_meta: format = GGUF V3 (latest) llm_load_print_meta: arch = qwen2 llm_load_print_meta: vocab type = BPE llm_load_print_meta: n_vocab = 152064 llm_load_print_meta: n_merges = 151387 llm_load_print_meta: vocab_only = 0 llm_load_print_meta: n_ctx_train = 32768 llm_load_print_meta: n_embd = 3584 llm_load_print_meta: n_layer = 28 llm_load_print_meta: n_head = 28 llm_load_print_meta: n_head_kv = 4 llm_load_print_meta: n_rot = 128 llm_load_print_meta: n_swa = 0 llm_load_print_meta: n_embd_head_k = 128 llm_load_print_meta: n_embd_head_v = 128 llm_load_print_meta: n_gqa = 7 llm_load_print_meta: n_embd_k_gqa = 512 llm_load_print_meta: n_embd_v_gqa = 512 llm_load_print_meta: f_norm_eps = 0.0e+00 llm_load_print_meta: f_norm_rms_eps = 1.0e-06 llm_load_print_meta: f_clamp_kqv = 0.0e+00 llm_load_print_meta: f_max_alibi_bias = 0.0e+00 llm_load_print_meta: f_logit_scale = 0.0e+00 llm_load_print_meta: n_ff = 18944 llm_load_print_meta: n_expert = 0 llm_load_print_meta: n_expert_used = 0 llm_load_print_meta: causal attn = 1 llm_load_print_meta: pooling type = 0 llm_load_print_meta: rope type = 2 llm_load_print_meta: rope scaling = linear llm_load_print_meta: freq_base_train = 1000000.0 llm_load_print_meta: freq_scale_train = 1 llm_load_print_meta: n_ctx_orig_yarn = 32768 llm_load_print_meta: rope_finetuned = unknown llm_load_print_meta: ssm_d_conv = 0 llm_load_print_meta: ssm_d_inner = 0 llm_load_print_meta: ssm_d_state = 0 llm_load_print_meta: ssm_dt_rank = 0 llm_load_print_meta: ssm_dt_b_c_rms = 0 llm_load_print_meta: model type = ?B llm_load_print_meta: model ftype = Q4_0 llm_load_print_meta: model params = 7.62 B llm_load_print_meta: model size = 4.12 GiB (4.65 BPW) llm_load_print_meta: general.name = Qwen2-7B-Instruct llm_load_print_meta: BOS token = 151643 '< endoftext >' llm_load_print_meta: EOS token = 151645 '< im_end >' llm_load_print_meta: PAD token = 151643 '< endoftext >' llm_load_print_meta: LF token = 148848 'ÄĬ' llm_load_print_meta: EOT token = 151645 '< im_end >' llm_load_print_meta: max token length = 256 ggml_sycl_init: GGML_SYCL_FORCE_MMQ: no ggml_sycl_init: SYCL_USE_XMX: yes ggml_sycl_init: found 1 SYCL devices: llm_load_tensors: ggml ctx size = 0.30 MiB llm_load_tensors: offloading 28 repeating layers to GPU llm_load_tensors: offloading non-repeating layers to GPU llm_load_tensors: offloaded 29/29 layers to GPU llm_load_tensors: SYCL0 buffer size = 3928.07 MiB llm_load_tensors: SYCL_Host buffer size = 292.36 MiB llama_new_context_with_model: n_ctx = 8192 llama_new_context_with_model: n_batch = 512 llama_new_context_with_model: n_ubatch = 512 llama_new_context_with_model: flash_attn = 0 llama_new_context_with_model: freq_base = 1000000.0 llama_new_context_with_model: freq_scale = 1 [SYCL] call ggml_check_sycl ggml_check_sycl: GGML_SYCL_DEBUG: 0 ggml_check_sycl: GGML_SYCL_F16: no found 1 SYCL devices: Max Max Global compute Max work sub mem ID Device Type Name Version units group group size Driver version
0 [level_zero:gpu:0] Intel Arc A770 Graphics 1.3 512 1024 32 16704M 1.3.29283

llama_kv_cache_init: SYCL0 KV buffer size = 448.00 MiB llama_new_context_with_model: KV self size = 448.00 MiB, K (f16): 224.00 MiB, V (f16): 224.00 MiB llama_new_context_with_model: SYCL_Host output buffer size = 2.38 MiB llama_new_context_with_model: SYCL0 compute buffer size = 500.00 MiB llama_new_context_with_model: SYCL_Host compute buffer size = 23.01 MiB llama_new_context_with_model: graph nodes = 1014 llama_new_context_with_model: graph splits = 2 INFO [wmain] model loaded | tid="8200" timestamp=1727415979 time=2024-09-27T13:46:19.445+08:00 level=INFO source=server.go:634 msg="llama runner started in 6.28 seconds" [GIN] 2024/09/27 - 13:46:19 | 200 | 6.3390885s | 127.0.0.1 | POST "/api/chat" INFO [print_timings] prompt eval time = 214.18 ms / 20 tokens ( 10.71 ms per token, 93.38 tokens per second) | n_prompt_tokens_processed=20 n_tokens_second=93.3776565943301 slot_id=0 t_prompt_processing=214.184 t_token=10.7092 task_id=4 tid="8200" timestamp=1727415983 INFO [print_timings] generation eval time = 312.33 ms / 13 runs ( 24.03 ms per token, 41.62 tokens per second) | n_decoded=13 n_tokens_second=41.62290924925078 slot_id=0 t_token=24.02523076923077 t_token_generation=312.32800000000003 task_id=4 tid="8200" timestamp=1727415983 INFO [print_timings] total time = 526.51 ms | slot_id=0 t_prompt_processing=214.184 t_token_generation=312.32800000000003 t_total=526.5120000000001 task_id=4 tid="8200" timestamp=1727415983 [GIN] 2024/09/27 - 13:46:23 | 200 | 550.0716ms | 127.0.0.1 | POST "/api/chat" INFO [print_timings] prompt eval time = 699.86 ms / 43 tokens ( 16.28 ms per token, 61.44 tokens per second) | n_prompt_tokens_processed=43 n_tokens_second=61.44059623097663 slot_id=0 t_prompt_processing=699.863 t_token=16.275883720930235 task_id=25 tid="8200" timestamp=1727415998 INFO [print_timings] generation eval time = 1699.65 ms / 66 runs ( 25.75 ms per token, 38.83 tokens per second) | n_decoded=66 n_tokens_second=38.83159267777368 slot_id=0 t_token=25.75222727272727 t_token_generation=1699.647 task_id=25 tid="8200" timestamp=1727415998 INFO [print_timings] total time = 2399.51 ms | slot_id=0 t_prompt_processing=699.863 t_token_generation=1699.647 t_total=2399.51 task_id=25 tid="8200" timestamp=1727415998 [GIN] 2024/09/27 - 13:46:38 | 200 | 2.4166555s | 127.0.0.1 | POST "/api/chat"

(llm-cpp) C:\Users\SAS>set ADVISOR_2024_DIR=C:\Program Files (x86)\Intel\oneAPI\advisor\2024.2\ ALLUSERSPROFILE=C:\ProgramData APM=C:\Program Files (x86)\Intel\oneAPI\advisor\2024.2\perfmodels APPDATA=C:\Users\SAS\AppData\Roaming CLASSPATH=c:\Program Files (x86)\Intel\oneAPI\dal\latest\share\java\onedal.jar; CMAKE_PREFIX_PATH=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env..;c:\Program Files (x86)\Intel\oneAPI\ipp\latest\lib\cmake\ipp;c:\Program Files (x86)\Intel\oneAPI\dpl\latest\lib\cmake\oneDPL;c:\Program Files (x86)\Intel\oneAPI\dnnl\latest\env..\lib\cmake;c:\Program Files (x86)\Intel\oneAPI\dal\latest;c:\Program Files (x86)\Intel\oneAPI\compiler\latest; CMPLR_ROOT=c:\Program Files (x86)\Intel\oneAPI\compiler\latest CommandPromptType=Native CommonProgramFiles=C:\Program Files\Common Files CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files CommonProgramW6432=C:\Program Files\Common Files COMPUTERNAME=DESKTOP-1NCU6BB ComSpec=C:\Windows\system32\cmd.exe CONDA_DEFAULT_ENV=llm-cpp CONDA_EXE=C:\Users\SAS\miniforge-pypy3\Scripts\conda.exe CONDA_PREFIX=C:\Users\SAS\miniforge-pypy3\envs\llm-cpp CONDA_PREFIX_1=C:\Users\SAS\miniforge-pypy3 CONDA_PROMPT_MODIFIER=(llm-cpp) CONDA_PYTHON_EXE=C:\Users\SAS\miniforge-pypy3\python.exe CONDA_SHLVL=2 CPATH=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env..\include;c:\Program Files (x86)\Intel\oneAPI\ocloc\latest\include;c:\Program Files (x86)\Intel\oneAPI\mkl\latest\include;c:\Program Files (x86)\Intel\oneAPI\ippcp\latest\include;c:\Program Files (x86)\Intel\oneAPI\ipp\latest\include;c:\Program Files (x86)\Intel\oneAPI\dpl\latest\include;c:\Program Files (x86)\Intel\oneAPI\dpcpp-ct\latest\env..\include;c:\Program Files (x86)\Intel\oneAPI\dev-utilities\latest\include;c:\Program Files (x86)\Intel\oneAPI\dal\latest\include\dal;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\include; DALROOT=c:\Program Files (x86)\Intel\oneAPI\dal\latest DAL_MAJOR_BINARY=2 DAL_MINOR_BINARY=0 DevEnvDir=C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\ DIAGUTIL_PATH=c:\Program Files (x86)\Intel\oneAPI\debugger\latest\env\..\etc\debugger\sys_check; DNNLROOT=c:\Program Files (x86)\Intel\oneAPI\dnnl\latest\env.. DPL_ROOT=c:\Program Files (x86)\Intel\oneAPI\dpl\latest DriverData=C:\Windows\System32\Drivers\DriverData EFC_7224=1 ERRORSTATE=0 ExtensionSdkDir=C:\Program Files (x86)\Microsoft SDKs\Windows Kits\10\ExtensionSDKs EXTERNAL_INCLUDE=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\include;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ATLMFC\include;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\shared;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\winrt;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\cppwinrt;C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um FPS_BROWSER_APP_PROFILE_STRING=Internet Explorer FPS_BROWSER_USER_PROFILE_STRING=Default Framework40Version=v4.0 FrameworkDir=C:\Windows\Microsoft.NET\Framework64\ FrameworkDir64=C:\Windows\Microsoft.NET\Framework64\ FrameworkVersion=v4.0.30319 FrameworkVersion64=v4.0.30319 FSHARPINSTALLDIR=C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\FSharp\Tools HOMEDRIVE=C: HOMEPATH=\Users\SAS HTMLHelpDir=C:\Program Files (x86)\HTML Help Workshop INCLUDE=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env..\include;c:\Program Files (x86)\Intel\oneAPI\ocloc\latest\include;c:\Program Files (x86)\Intel\oneAPI\mkl\latest\include;c:\Program Files (x86)\Intel\oneAPI\ippcp\latest\include;c:\Program Files (x86)\Intel\oneAPI\ipp\latest\include;c:\Program Files (x86)\Intel\oneAPI\dpcpp-ct\latest\env..\include;c:\Program Files (x86)\Intel\oneAPI\dnnl\latest\env..\include;c:\Program Files (x86)\Intel\oneAPI\dev-utilities\latest\include;c:\Program Files (x86)\Intel\oneAPI\dal\latest\include\dal;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\include;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\include;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ATLMFC\include;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\shared;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\winrt;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\cppwinrt;C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um INTELGTDEBUGGERROOT=c:\Program Files (x86)\Intel\oneAPI\debugger\latest\env\.. INTEL_PYTHONPATH=C:\Program Files (x86)\Intel\oneAPI\advisor\2024.2\pythonapi INTEL_TARGET_ARCH=intel64 IPPCP_TARGET_ARCH=intel64 IPPCP_TARGET_BIN_ARCH=bin IPPCP_TARGET_LIB_ARCH=lib IPPCRYPTOROOT=c:\Program Files (x86)\Intel\oneAPI\ippcp\latest IPPROOT=c:\Program Files (x86)\Intel\oneAPI\ipp\latest IPP_TARGET_ARCH=intel64 LIB=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env..\lib\;c:\Program Files (x86)\Intel\oneAPI\mkl\latest\lib;c:\Program Files (x86)\Intel\oneAPI\ippcp\latest\lib;c:\Program Files (x86)\Intel\oneAPI\ipp\latest\lib;c:\Program Files (x86)\Intel\oneAPI\dnnl\latest\env..\lib;c:\Program Files (x86)\Intel\oneAPI\dal\latest\lib;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\lib\clang\18\lib\windows;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\opt\compiler\lib;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\lib;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ATLMFC\lib\x64;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\lib\x64;C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64;C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64;C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\um\x64 LIBPATH=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ATLMFC\lib\x64;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\lib\x64;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\lib\x86\store\references;C:\Program Files (x86)\Windows Kits\10\UnionMetadata\10.0.22621.0;C:\Program Files (x86)\Windows Kits\10\References\10.0.22621.0;C:\Windows\Microsoft.NET\Framework64\v4.0.30319 LIBRARY_PATH=c:\Program Files (x86)\Intel\oneAPI\ippcp\latest\lib;c:\Program Files (x86)\Intel\oneAPI\ipp\latest\lib; LOCALAPPDATA=C:\Users\SAS\AppData\Local LOGONSERVER=\DESKTOP-1NCU6BB MKLROOT=c:\Program Files (x86)\Intel\oneAPI\mkl\latest NETFXSDKDir=C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\ NLSPATH=c:\Program Files (x86)\Intel\oneAPI\mkl\latest\share\locale\1033; NUMBER_OF_PROCESSORS=20 OCLOC_ROOT=c:\Program Files (x86)\Intel\oneAPI\ocloc\latest OCL_ICD_FILENAMES=;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\bin\intelocl64_emu.dll;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\bin\intelocl64.dll OLLAMA_HOST=0.0.0.0 OLLAMA_NUM_GPU=999 ONEAPI_ROOT=c:\Program Files (x86)\Intel\oneAPI OneDrive=C:\Users\SAS\OneDrive opensslIncludeDir=C:\Program Files\OpenSSL\include OS=Windows_NT Path=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env..\bin\;c:\Program Files (x86)\Intel\oneAPI\ocloc\latest\bin;c:\Program Files (x86)\Intel\oneAPI\mkl\latest\bin;c:\Program Files (x86)\Intel\oneAPI\ippcp\latest\bin;c:\Program Files (x86)\Intel\oneAPI\ipp\latest\bin;c:\Program Files (x86)\Intel\oneAPI\dpcpp-ct\latest\env..\bin;c:\Program Files (x86)\Intel\oneAPI\dnnl\latest\env..\bin;c:\Program Files (x86)\Intel\oneAPI\dev-utilities\latest\bin;c:\Program Files (x86)\Intel\oneAPI\debugger\latest\env\..\opt\debugger\bin;c:\Program Files (x86)\Intel\oneAPI\dal\latest\bin;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\lib\ocloc;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\bin;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\bin\HostX64\x64;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\VC\VCPackages;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\TestWindow;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer;C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\bin\Roslyn;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\x64\;C:\Program Files (x86)\HTML Help Workshop;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\FSharp\Tools;C:\Program Files\Microsoft Visual Studio\2022\Community\Team Tools\DiagnosticsHub\Collector;C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\x64;C:\Program Files (x86)\Windows Kits\10\bin\x64;C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\amd64;C:\Windows\Microsoft.NET\Framework64\v4.0.30319;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\Tools\;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Library\mingw-w64\bin;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Library\usr\bin;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Library\bin;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Scripts;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\bin;C:\Users\SAS\miniforge-pypy3\condabin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files\dotnet;C:\Program Files\Git\cmd;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files\PuTTY;C:\Program Files\Rust stable MSVC 1.78\bin;C:\Program Files\CMake\bin;C:\Strawberry\c\bin;C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\Users\SAS\AppData\Local\bin\NASM;C:\Strawberry\perl\bin;C:\Users\SAS\miniforge3\Scripts;C:\Program Files\nodejs;C:\Program Files\Docker\Docker\resources\bin;C:\Users\SAS\bin;C:\Users\SAS.cargo\bin;C:\Users\SAS\AppData\Local\Microsoft\WindowsApps;C:\Users\SAS\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\SAS\AppData\Local\Microsoft\WinGet\Links;C:\Users\SAS\AppData\Roaming\npm;C:\Users\SAS.dotnet\tools;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\VC\Linux\bin\ConnectionManagerExe;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\vcpkg;C:\Program Files (x86)\Intel\oneAPI\advisor\2024.2\bin64;C:\Program Files (x86)\Intel\oneAPI\vtune\2024.2\bin64 PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PKG_CONFIG_PATH=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env..\lib\pkgconfig;c:\Program Files (x86)\Intel\oneAPI\mkl\latest\lib\pkgconfig;c:\Program Files (x86)\Intel\oneAPI\ippcp\latest\lib\pkgconfig;c:\Program Files (x86)\Intel\oneAPI\dpl\latest\lib\pkgconfig;c:\Program Files (x86)\Intel\oneAPI\dnnl\latest\env..\lib\pkgconfig;c:\Program Files (x86)\Intel\oneAPI\dal\latest\lib\pkgconfig;c:\Program Files (x86)\Intel\oneAPI\compiler\latest\lib\pkgconfig; Platform=x64 PROCESSOR_ARCHITECTURE=AMD64 PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 151 Stepping 2, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=9702 ProgramData=C:\ProgramData ProgramFiles=C:\Program Files ProgramFiles(x86)=C:\Program Files (x86) ProgramW6432=C:\Program Files PROMPT=(llm-cpp) $P$G PSModulePath=C:\Program Files\WindowsPowerShell\Modules;C:\Windows\system32\WindowsPowerShell\v1.0\Modules PUBLIC=C:\Users\Public PYTHONPATH=C:\Program Files (x86)\Intel\oneAPI\advisor\2024.2\pythonapi SESSIONNAME=Console SETVARS_COMPLETED=1 SSL_CERT_DIR=C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Library\ssl\certs SSL_CERT_FILE=C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Library\ssl\cacert.pem SYCL_CACHE_PERSISTENT=1 SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1 SystemDrive=C: SystemRoot=C:\Windows TBBROOT=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env.. TBB_BIN_DIR=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env..\bin TBB_DLL_PATH=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env..\bin\ TBB_SCRIPT_DIR=c:\Program Files (x86)\Intel\oneAPI\tbb\latest\env\ TBB_TARGET_ARCH=intel64 TEMP=C:\Users\SAS\AppData\Local\Temp TMP=C:\Users\SAS\AppData\Local\Temp UCRTVersion=10.0.22621.0 UniversalCRTSdkDir=C:\Program Files (x86)\Windows Kits\10\ USERDOMAIN=DESKTOP-1NCU6BB USERDOMAIN_ROAMINGPROFILE=DESKTOP-1NCU6BB USERNAME=SAS USERPROFILE=C:\Users\SAS USE_INTEL_LLVM=0 VARSDIR=c:\Program Files (x86)\Intel\oneAPI\ocloc\latest\env\ VCIDEInstallDir=C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\VC\ VCINSTALLDIR=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\ VCPKG_ROOT=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\vcpkg VCToolsInstallDir=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ VCToolsRedistDir=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Redist\MSVC\14.38.33135\ VCToolsVersion=14.39.33519 VisualStudioVersion=17.0 VS170COMNTOOLS=C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\Tools\ VSCMD_ARG_app_plat=Desktop VSCMD_ARG_HOST_ARCH=x64 VSCMD_ARG_TGT_ARCH=x64 VSCMD_VER=17.9.6 VSINSTALLDIR=C:\Program Files\Microsoft Visual Studio\2022\Community\ VS_TARGET_ARCH=amd64 VTUNE_PROFILER_2024_DIR=C:\Program Files (x86)\Intel\oneAPI\vtune\2024.2\ VTUNE_PROFILER_DIR=C:\Program Files (x86)\Intel\oneAPI\vtune\2024.2\ windir=C:\Windows WindowsLibPath=C:\Program Files (x86)\Windows Kits\10\UnionMetadata\10.0.22621.0;C:\Program Files (x86)\Windows Kits\10\References\10.0.22621.0 WindowsSdkBinPath=C:\Program Files (x86)\Windows Kits\10\bin\ WindowsSdkDir=C:\Program Files (x86)\Windows Kits\10\ WindowsSDKLibVersion=10.0.22621.0\ WindowsSdkVerBinPath=C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\ WindowsSDKVersion=10.0.22621.0\ WindowsSDK_ExecutablePath_x64=C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\x64\ WindowsSDK_ExecutablePath_x86=C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\ ZES_ENABLE_SYSMAN=1 CONDA_OPENSSL_CERT_DIR_SET="1" CONDA_OPENSSL_CERT_FILE_SET="1" DOTNET_ADD_64BIT=1 DOTNET_PREFERRED_BITNESS=64 __MS_VC_INSTALL_PATH=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ __VSCMD_PREINIT_PATH=C:\Users\SAS\miniforge-pypy3\envs\llm-cpp;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Library\mingw-w64\bin;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Library\usr\bin;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Library\bin;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\Scripts;C:\Users\SAS\miniforge-pypy3\envs\llm-cpp\bin;C:\Users\SAS\miniforge-pypy3\condabin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files\dotnet;C:\Program Files\Git\cmd;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files\PuTTY;C:\Program Files\Rust stable MSVC 1.78\bin;C:\Program Files\CMake\bin;C:\Strawberry\c\bin;C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\Users\SAS\AppData\Local\bin\NASM;C:\Strawberry\perl\bin;C:\Users\SAS\miniforge3\Scripts;C:\Program Files\nodejs;C:\Program Files\Docker\Docker\resources\bin;C:\Users\SAS\bin;C:\Users\SAS.cargo\bin;C:\Users\SAS\AppData\Local\Microsoft\WindowsApps;C:\Users\SAS\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\SAS\AppData\Local\Microsoft\WinGet\Links;C:\Users\SAS\AppData\Roaming\npm;C:\Users\SAS.dotnet\tools

sgwhat commented 1 month ago

Hi @yunbiaolin , we cannot reproduce your issue, ollama works fine on our windows arc device. Could you please provide more details how you install and run ollama?