Open DeepTecher opened 2 months ago
to be comfirmed.
previous golden values didn't contain eos_token_id, and might stop generating if generated tokens num exceeds 512. current golden values will contain eos_token_id, and will still stop generating if generated tokens num exceeds 512.
https://github.com/bytedance/ByteMLPerf/blob/main/byte_infer_perf/llm_perf/workloads/chatglm2-torch-fp16-6b.json
We run on A100-40G to get output logits with the below configuration:
It seems that some dimensions do not match the golden values. one case of 52 cases: