Is there any accuracy loss when converting to flm model?

[empty2enrich]

I convert llama2-7b using fastllm_pytools.torch2flm, The inference result looks wrong, Also inconsistent with inference results using llama2-7b directly:

prompt: The president of the United States is

generate result:

### Instruction:
The president of the United States is

### Response:
The president of the United States

The president of the United States is

### Instruction:
The president of the United States is

### Response:
The president of the United States is

### Instruction:
The president of the United States is

### Response:
The president of the United States is

### Instruction:
The president of the United States is

### Response:
The president of the United States is

The president of the United States is

### Instruction:
The president of the United States is

### Response:
The president of the United States is

[TylunasLi]

you should pay atention to difference of the prompt building process between original python code and fastlllm code. for llama2-7b:

• python code is Llama.text_completion(),
• in fastllm, there's no directly callable method, but you can wrap a method to call fastllm_lib.launch_response_str_llm_model(). or you can config the following parts in config.json and re-convert the model.

  "pre_prompt": "",
  "user_role": "",
  "bot_role": "",
  "history_sep":  "",

for llama2-7b-chat-hf,

• python code is Llama.chat_completion()
• in fastllm, you should config the following parts in config.json and re-convert the model.

  "pre_prompt": "",
  "user_role": "[INST] ",
  "bot_role": " [/INST]",
  "history_sep":  " ",