Hi, Thanks for the great work. I'm just in general curious about whether there is a reason to use the Chinese version of '｜' and '▁'instead of the '|' , ‘_’ which is standard ASCII characters in eos_token and bos_token. ('<｜end▁of▁sentence｜>' and '<｜begin▁of▁sentence｜>' ). Is this for distinguishing deep seek model from English only LLM's like Llamma?
Hi, Thanks for the great work. I'm just in general curious about whether there is a reason to use the Chinese version of '｜' and '▁'instead of the '|' , ‘_’ which is standard ASCII characters in eos_token and bos_token. ('<｜end▁of▁sentence｜>' and '<｜begin▁of▁sentence｜>' ). Is this for distinguishing deep seek model from English only LLM's like Llamma?