Open 2213601279 opened 8 months ago
./benchmark -p /opt/Convert/flm/qwen-14b-chart-int4.flm -f ../example/benchmark/prompts/beijing.txt -b 1 Load (323 / 323) Warmup... finish. AVX: ON AVX2: ON AARCH64: OFF Neon FP16: OFF Neon DOT: OFF
卡住可能是qwen-14B-int4生成的结果停不下来, 可以考虑加入参数“-l 512” 限制输出长度为512 tokens。
./benchmark -p /opt/Convert/flm/qwen-14b-chart-int4.flm -f ../example/benchmark/prompts/beijing.txt -b 1 Load (323 / 323) Warmup... finish. AVX: ON AVX2: ON AARCH64: OFF Neon FP16: OFF Neon DOT: OFF