Closed kmn1024 closed 7 months ago
Issue seems similar to https://github.com/wangzhaode/llm-export/issues/8, but even after I upgrade to MNN=2.8.1 and repeat the steps, nothing changes...
用了./MNNDump2Json 才发现 phi2 release 是改了这些:
改了之后就没问题了。
是的,phi-2需要使用对称量化,这里有写:https://github.com/wangzhaode/llm-export/blob/7b442e4075cb909ba5803f0f3a0557e1ee83e57c/llm_export.py#L788C10-L788C10
本来在转换MNN时会使用asymmetric=False
的,但是在某次push时把phi-2的那个patch修改错了
谢大佬!
I carried out these steps on Linux desktop (i7-6700K, Ubuntu 22.04)
The conversion had some warnings but was successful:
The .mnn file sizes are different from the release however:
And when I test with cli_demo, the responses are very different:
A:
and
(
and- (
and- and (
A: 's theory of general relativity, which describes the force of gravity, is the most successful theory in the history of science. It has been tested and confirmed by countless experiments and observations, and it has been used to make predictions and discoveries in various fields, such...