Closed RiverFerry closed 5 months ago
文本模型可以运行
可能和这个有关 https://github.com/mlc-ai/mlc-llm/issues/665 ,怀疑和模型无响应(例如手机内存不足)有关
链接的apk下载不了了吗?点积下载链接显示如下:
UserDisable
可能和这个有关 mlc-ai/mlc-llm#665 ,怀疑和模型无响应(例如手机内存不足)有关
我更新了手机操作系统现在图片可以发送出去了,但是还是报错,这次的错误信息直接提示了内存不足,我的手机是12gb的内存,问下官方的demo演示用的是多大的内存。提供的apk是用的int4量化版本的模型吗。推荐的运行最低配置是多少,我现在的报错是:
MLCChat failed
Stack trace:
org.apache.tvm.Base$TVMError: TVMError: OpenCL build error for device=0x77c655bb58
Error: CL_OUT_OF_HOST_MEMORY
Pass
Stack trace:
File "/Users/acha/Desktop/mlc-MiniCPM/3rdparty/tvm/src/runtime/[library_module.cc](http://library_module.cc/)", line 78
at org.apache.tvm.Base.checkCall(Base.java:173)
at org.apache.tvm.Function.invoke(Function.java:130)
at com.modelbest.mlcllm.ChatModule.image(ChatModule.java:63)
at com.modelbest.minicpm.AppViewModel$ChatState$requestImage$1$1.invoke(AppViewModel.kt:629)
at com.modelbest.minicpm.AppViewModel$ChatState$requestImage$1$1.invoke(AppViewModel.kt:629)
at com.modelbest.minicpm.AppViewModel$ChatState.callBackend(AppViewModel.kt:515)
at com.modelbest.minicpm.AppViewModel$ChatState.requestImage$lambda$4(AppViewModel.kt:629)
at com.modelbest.minicpm.AppViewModel$ChatState.$r8$lambda$pENZ-KlflB_gdrlr6E2VgmxXkJA(Unknown Source:0)
at com.modelbest.minicpm.AppViewModel$ChatState$$ExternalSyntheticLambda2.run(Unknown Source:6)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:487)
at java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
at java.lang.Thread.run(Thread.java:1012)
Error message:
TVMError: OpenCL build error for device=0x77c655bb58
Error: CL_OUT_OF_HOST_MEMORY
Pass
Stack trace:
File "/Users/acha/Desktop/mlc-MiniCPM/3rdparty/tvm/src/runtime/[library_module.cc](http://library_module.cc/)", line 78
后面我尝试了本地编译,到这一步报错了:
python -m mlc_chat convert_weight --model-type ${MODEL_TYPE} ../dist/models/${MODEL_NAME}/ --quantization $QUANTIZATION -o ../dist/$MODEL_NAME/
/Users/river/Library/Python/3.9/lib/python/site-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
_torch_pytree._register_pytree_node(
[2024-02-06 01:14:00] INFO auto_config.py:115: Found model configuration: ../dist/models/MiniCPM-V/config.json
[2024-02-06 01:14:00] INFO auto_device.py:85: Not found device: cuda:0
[2024-02-06 01:14:01] INFO auto_device.py:85: Not found device: rocm:0
[2024-02-06 01:14:01] INFO auto_device.py:76: Found device: metal:0
[2024-02-06 01:14:01] INFO auto_device.py:85: Not found device: vulkan:0
[2024-02-06 01:14:02] INFO auto_device.py:85: Not found device: opencl:0
[2024-02-06 01:14:02] INFO auto_device.py:33: Using device: metal:0
[2024-02-06 01:14:02] INFO auto_weight.py:70: Finding weights in: ../dist/models/MiniCPM-V
[2024-02-06 01:14:02] INFO auto_weight.py:136: Not found Huggingface PyTorch
[2024-02-06 01:14:02] INFO auto_weight.py:143: Found source weight format: huggingface-safetensor. Source configuration: ../dist/models/MiniCPM-V/model.safetensors.index.json
[2024-02-06 01:14:02] INFO auto_weight.py:106: Using source weight configuration: ../dist/models/MiniCPM-V/model.safetensors.index.json. Use `--source` to override.
[2024-02-06 01:14:02] INFO auto_weight.py:110: Using source weight format: huggingface-safetensor. Use `--source-format` to override.
[2024-02-06 01:14:02] INFO auto_config.py:153: Found model type: minicpm_v. Use `--model-type` to override.
Weight conversion with arguments:
--config ../dist/models/MiniCPM-V/config.json
--quantization GroupQuantize(name='q4f16_1', kind='group-quant', group_size=32, quantize_dtype='int4', storage_dtype='uint32', model_dtype='float16', linear_weight_layout='NK', num_elem_per_storage=8, num_storage_per_group=4, max_int_value=7)
--model-type minicpm_v
--device metal:0
--source ../dist/models/MiniCPM-V/model.safetensors.index.json
--source-format huggingface-safetensor
--output ../dist/MiniCPM-V
[2024-02-06 01:14:02] INFO mistral_model.py:55: prefill_chunk_size defaults to sliding_window_size (4096)
Traceback (most recent call last):
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/Users/river/code/mlc-MiniCPM/python/mlc_chat/__main__.py", line 39, in <module>
main()
File "/Users/river/code/mlc-MiniCPM/python/mlc_chat/__main__.py", line 28, in main
cli.main(sys.argv[2:])
File "/Users/river/code/mlc-MiniCPM/python/mlc_chat/cli/convert_weight.py", line 87, in main
convert_weight(
File "/Users/river/code/mlc-MiniCPM/python/mlc_chat/interface/convert_weight.py", line 156, in convert_weight
_convert_args(args)
File "/Users/river/code/mlc-MiniCPM/python/mlc_chat/interface/convert_weight.py", line 76, in _convert_args
_, _named_params, _ = model.export_tvm( # type: ignore[misc]
File "/Users/river/Library/Python/3.9/lib/python/site-packages/tvm/relax/frontend/nn/core.py", line 479, in export_tvm
mod, params, ext_mods = Exporter(debug=debug).build(spec)
File "/Users/river/Library/Python/3.9/lib/python/site-packages/tvm/relax/frontend/nn/exporter.py", line 136, in build
outputs, inputs = _emit_method(self.builder, method_spec, params, effects)
File "/Users/river/Library/Python/3.9/lib/python/site-packages/tvm/relax/frontend/nn/exporter.py", line 277, in _emit_method
outputs = spec.method(*explicit_inputs)
File "/Users/river/code/mlc-MiniCPM/python/mlc_chat/model/mistral/mistral_model.py", line 700, in image
inputs = self.vpm(inputs)
File "/Users/river/Library/Python/3.9/lib/python/site-packages/tvm/relax/frontend/nn/core.py", line 427, in __call__
return self.forward(*args, **kwargs) # pylint: disable=no-member
File "/Users/river/Library/Python/3.9/lib/python/site-packages/tvm/relax/frontend/nn/subroutine.py", line 87, in new_forward
return old_forward(self, *args, **kwargs)
File "/Users/river/code/mlc-MiniCPM/python/mlc_chat/model/mistral/vit_model.py", line 195, in forward
hidden_states = self.patch_embed(inputs)
File "/Users/river/Library/Python/3.9/lib/python/site-packages/tvm/relax/frontend/nn/core.py", line 427, in __call__
return self.forward(*args, **kwargs) # pylint: disable=no-member
File "/Users/river/Library/Python/3.9/lib/python/site-packages/tvm/relax/frontend/nn/subroutine.py", line 87, in new_forward
return old_forward(self, *args, **kwargs)
File "/Users/river/code/mlc-MiniCPM/python/mlc_chat/model/mistral/vit_model.py", line 155, in forward
embed = self.proj(inputs)
File "/Users/river/Library/Python/3.9/lib/python/site-packages/tvm/relax/frontend/nn/core.py", line 427, in __call__
return self.forward(*args, **kwargs) # pylint: disable=no-member
File "/Users/river/Library/Python/3.9/lib/python/site-packages/tvm/relax/frontend/nn/subroutine.py", line 87, in new_forward
return old_forward(self, *args, **kwargs)
File "/Users/river/code/mlc-MiniCPM/python/mlc_chat/model/mistral/vit_model.py", line 141, in forward
x = x + self.bias.reshape([1, self.bias.shape[0], 1, 1])
File "/Users/river/Library/Python/3.9/lib/python/site-packages/tvm/relax/frontend/nn/_tensor_op.py", line 82, in reshape
return _op().reshape(self, shape)
File "/Users/river/Library/Python/3.9/lib/python/site-packages/tvm/relax/frontend/nn/op.py", line 644, in reshape
return wrap_nested(_op.reshape(x._expr, shape), name)
File "/Users/river/Library/Python/3.9/lib/python/site-packages/tvm/relax/op/manipulate.py", line 215, in reshape
return _ffi_api.reshape(x, shape) # type: ignore
File "tvm/_ffi/_cython/./packed_func.pxi", line 332, in tvm._ffi._cy3.core.PackedFuncBase.__call__
File "tvm/_ffi/_cython/./packed_func.pxi", line 263, in tvm._ffi._cy3.core.FuncCall
File "tvm/_ffi/_cython/./packed_func.pxi", line 252, in tvm._ffi._cy3.core.FuncCall3
File "tvm/_ffi/_cython/./base.pxi", line 182, in tvm._ffi._cy3.core.CHECK_CALL
File "/Users/river/Library/Python/3.9/lib/python/site-packages/tvm/_ffi/base.py", line 481, in raise_last_ffi_error
raise py_err
tvm._ffi.base.TVMError: Traceback (most recent call last):
File "/Users/catalyst/Workspace/mlc-ai-package-self-runner/_work/package/package/tvm/src/relax/op/tensor/[manipulate.cc](http://manipulate.cc/)", line 675
TVMError: Check failed: (_len != nullptr) is false: Reshape only expects the input new shape to be either an Expr or an Array of PrimExprs. However, the given new shape is [[1, 1152, 1, 1]]
[01:14:02] /Users/catalyst/Workspace/mlc-ai-package-self-runner/_work/package/package/tvm/src/relax/ir/[block_builder.cc:65](http://block_builder.cc:65/): Warning: BlockBuilder destroyed with remaining blocks!
我的手机配置是 oneplus 9R, 12GB 运行内存(没额外使用内存扩展),骁龙 870 八核。提供的 apk 是 int4 版本的。 我试了些同样是 oneplus 系列或 oppo 系列的新手机,可用运行内存更大,但是却不能运行,现在尚不明确具体原因。
链接的apk下载不了了吗?点积下载链接显示如下:
UserDisable
UserDisable 65C18EB7FC567C3437887155 openbmb.oss-cn-hongkong.aliyuncs.com 0003-00000801 https://api.aliyun.com/troubleshoot?q=0003-00000801
现在应该是可以下载了。
我的手机配置是 oneplus 9R, 12GB 运行内存(没额外使用内存扩展),骁龙 870 八核。提供的 apk 是 int4 版本的。 我试了些同样是 oneplus 系列或 oppo 系列的新手机,可用运行内存更大,但是却不能运行,现在尚不明确具体原因。
请问现在多模态的有进展吗?还有上面我贴的自己编译遇到的问题麻烦给看下: File "/Users/catalyst/Workspace/mlc-ai-package-self-runner/_work/package/package/tvm/src/relax/op/tensor/manipulate.cc", line 675 TVMError: Check failed: (_len != nullptr) is false: Reshape only expects the input new shape to be either an Expr or an Array of PrimExprs. However, the given new shape is [[1, 1152, 1, 1]] [01:14:02] /Users/catalyst/Workspace/mlc-ai-package-self-runner/_work/package/package/tvm/src/relax/ir/block_builder.cc:65: Warning: BlockBuilder destroyed with remaining blocks!
多模态现在还在研究。你这个 shape [[1, 1152, 1, 1]] 的问题应该和 #47 是一样的,是版本问题
Description / 描述
环境:
问题
使用这里提供的apk:https://github.com/OpenBMB/mlc-MiniCPM 安装后下载完模型,按照说明操作到了ready to chat后上传图片就没有反应了,没有出现预期的“image process is done” 输入文字会直接报错:![20240205-171906](https://github.com/OpenBMB/MiniCPM/assets/48130519/0f27972c-2b7c-401a-bd6c-556aff69bf92)
详细报错:
Case Explaination / 案例解释
No response