HoiM
commented
9 hours ago Same problem when I run llava-1.5-7b-hf
Open l241025097 opened 10 hours ago
HoiM
commented
9 hours ago Same problem when I run llava-1.5-7b-hf
🐛 Bug
To Reproduce
Steps to reproduce the behavior:
编译 mlc_llm compile /path/to/internlm2_5-20b-q0f16-MLC/mlc-chat-config.json \ --device metal \ --overrides "max_batch_size=128;prefill_chunk_size=4096" \ -o /path/to/internlm2_5-20b-q0f16-MLC/libs/internlm2_5-20b-q0f16-MLC-metal.so
运行 mlc_llm chat \ /path/to/internlm2_5-20b-q0f16-MLC \ --model-lib /path/to/internlm2_5-20b-q0f16-MLC/libs/internlm2_5-20b-q0f16-MLC-metal.so
Expected behavior
能够正常回复“你好”。
Environment
Platform: arm64 Operating system: MacOS 15.1.1 Device: metal How you installed MLC-LLM : pip, source How you installed TVM-Unity: pip, source Python version: 3.11 TVM Unity Hash Tag: python -c "import tvm; print('\n'.join(f'{k}: {v}' for k, v in tvm.support.libinfo().items()))" [11:43:36] /Users/lyn/Documents/python/learn/mlc_llm/modules/tvm-unity/src/target/llvm/llvm_instance.cc:226: Error: Using LLVM 19.1.4 with -mcpu=apple-latest is not valid in -mtriple=arm64-apple-macos, using default -mcpu=generic [11:43:36] /Users/lyn/Documents/python/learn/mlc_llm/modules/tvm-unity/src/target/llvm/llvm_instance.cc:226: Error: Using LLVM 19.1.4 with -mcpu=apple-latest is not valid in -mtriple=arm64-apple-macos, using default -mcpu=generic [11:43:36] /Users/lyn/Documents/python/learn/mlc_llm/modules/tvm-unity/src/target/llvm/llvm_instance.cc:226: Error: Using LLVM 19.1.4 with -mcpu=apple-latest is not valid in -mtriple=arm64-apple-macos, using default -mcpu=generic CLML Target Version: 3 USE_NVTX: OFF USE_GTEST: AUTO SUMMARIZE: OFF TVM_DEBUG_WITH_ABI_CHANGE: OFF USE_IOS_RPC: OFF USE_MSC: OFF USE_ETHOSU: OFF CUDA_VERSION: NOT-FOUND USE_LIBBACKTRACE: AUTO DLPACK_PATH: 3rdparty/dlpack/include USE_TENSORRT_CODEGEN: OFF USE_OPENCL_EXTN_QCOM: NOT-FOUND USE_TARGET_ONNX: OFF USE_AOT_EXECUTOR: ON BUILD_DUMMY_LIBTVM: OFF USE_CUDNN: OFF USE_TENSORRT_RUNTIME: OFF USE_ARM_COMPUTE_LIB_GRAPH_EXECUTOR: OFF USE_THRUST: OFF USE_CCACHE: AUTO USE_ARM_COMPUTE_LIB: OFF USE_CPP_RTVM: OFF USE_OPENCL_GTEST: /path/to/opencl/gtest TVM_LOG_BEFORE_THROW: OFF USE_MKL: OFF USE_PT_TVMDSOOP: OFF MLIR_VERSION: NOT-FOUND USE_CLML: OFF USE_STACKVM_RUNTIME: OFF USE_GRAPH_EXECUTOR_CUDA_GRAPH: OFF ROCM_PATH: /opt/rocm USE_DNNL: OFF USE_MSCCL: OFF USE_NNAPI_RUNTIME: OFF USE_VITIS_AI: OFF USE_MLIR: OFF USE_RCCL: OFF USE_LLVM: llvm-config USE_VERILATOR: OFF USE_TF_TVMDSOOP: OFF USE_THREADS: ON USE_MSVC_MT: OFF BACKTRACE_ON_SEGFAULT: OFF USE_GRAPH_EXECUTOR: ON USE_NCCL: OFF USE_ROCBLAS: OFF GIT_COMMIT_HASH: 30f97b0df3a0078ac5e6be1e8ad50eadcc2dff43 USE_VULKAN: OFF USE_RUST_EXT: OFF USE_CUTLASS: OFF USE_CPP_RPC: OFF USE_HEXAGON: OFF USE_CUSTOM_LOGGING: OFF USE_UMA: OFF USE_FALLBACK_STL_MAP: OFF USE_SORT: ON USE_RTTI: ON GIT_COMMIT_TIME: 2024-11-15 11:16:12 -0500 USE_HIPBLAS: OFF USE_HEXAGON_SDK: /path/to/sdk USE_BLAS: none USE_ETHOSN: OFF USE_LIBTORCH: OFF USE_RANDOM: ON USE_CUDA: OFF USE_COREML: OFF USE_AMX: OFF BUILD_STATIC_RUNTIME: OFF USE_CMSISNN: OFF USE_KHRONOS_SPIRV: OFF USE_CLML_GRAPH_EXECUTOR: OFF USE_TFLITE: OFF USE_HEXAGON_GTEST: /path/to/hexagon/gtest PICOJSON_PATH: 3rdparty/picojson USE_OPENCL_ENABLE_HOST_PTR: OFF INSTALL_DEV: OFF USE_PROFILER: ON USE_NNPACK: OFF LLVM_VERSION: 19.1.4 USE_MRVL: OFF USE_OPENCL: OFF COMPILER_RT_PATH: 3rdparty/compiler-rt USE_NNAPI_CODEGEN: OFF RANG_PATH: 3rdparty/rang/include USE_SPIRV_KHR_INTEGER_DOT_PRODUCT: OFF USE_OPENMP: none USE_BNNS: OFF USE_FLASHINFER: OFF USE_CUBLAS: OFF USE_METAL: ON USE_MICRO_STANDALONE_RUNTIME: OFF USE_HEXAGON_EXTERNAL_LIBS: OFF USE_ALTERNATIVE_LINKER: AUTO USE_BYODT_POSIT: OFF USE_NVSHMEM: OFF USE_HEXAGON_RPC: OFF USE_MICRO: OFF DMLC_PATH: 3rdparty/dmlc-core/include INDEX_DEFAULT_I64: ON USE_RELAY_DEBUG: OFF USE_RPC: ON USE_TENSORFLOW_PATH: none TVM_CLML_VERSION: USE_MIOPEN: OFF USE_ROCM: OFF USE_PAPI: OFF USE_CURAND: OFF TVM_CXX_COMPILER_PATH: /Library/Developer/CommandLineTools/usr/bin/c++ HIDE_PRIVATE_SYMBOLS: ON
Additional context