ollama generate incorrect answer when it run glm-4-9b-chat model

When ollama create glm-4-9b-chat model and inference, it always gives a random and incorrect response from second round, like below:

The complete log of curl command is folded at here

``` # curl http://localhost:11434/api/chat -d '{ "model": "glm4", "messages": [ { "role": "user", "content": "介绍一下上海的著名景点" } ] }' {"model":"glm4","created_at":"2024-10-14T14:02:34.047484624Z","message":{"role":"assistant","content":"\""},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.083040067Z","message":{"role":"assistant","content":"东方"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.117628086Z","message":{"role":"assistant","content":"明珠"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.152355197Z","message":{"role":"assistant","content":"塔"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.186764637Z","message":{"role":"assistant","content":"”："},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.221193975Z","message":{"role":"assistant","content":"作为"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.255737188Z","message":{"role":"assistant","content":"上海的"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.290304743Z","message":{"role":"assistant","content":"标志性"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.324637153Z","message":{"role":"assistant","content":"建筑"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.359387512Z","message":{"role":"assistant","content":"，"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.394246012Z","message":{"role":"assistant","content":"它"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.429042753Z","message":{"role":"assistant","content":"融合"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.463491151Z","message":{"role":"assistant","content":"了"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.498027441Z","message":{"role":"assistant","content":"现代"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.532440349Z","message":{"role":"assistant","content":"城市"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.567029853Z","message":{"role":"assistant","content":"美"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.601535756Z","message":{"role":"assistant","content":"学的"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.636126708Z","message":{"role":"assistant","content":"精华"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.670677665Z","message":{"role":"assistant","content":"。\n\n"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.705137513Z","message":{"role":"assistant","content":"\""},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.739545318Z","message":{"role":"assistant","content":"外"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.774045714Z","message":{"role":"assistant","content":"滩"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.808468877Z","message":{"role":"assistant","content":"”："},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.842924158Z","message":{"role":"assistant","content":"这里"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.877376785Z","message":{"role":"assistant","content":"被誉为"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.91187573Z","message":{"role":"assistant","content":"“"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.946337116Z","message":{"role":"assistant","content":"万"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:34.980840121Z","message":{"role":"assistant","content":"国"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.015301191Z","message":{"role":"assistant","content":"建筑"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.049693979Z","message":{"role":"assistant","content":"博览"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.084161034Z","message":{"role":"assistant","content":"群"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.118586049Z","message":{"role":"assistant","content":"”，"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.153068224Z","message":{"role":"assistant","content":"是"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.187589385Z","message":{"role":"assistant","content":"上海"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.222024104Z","message":{"role":"assistant","content":"近代"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.256522112Z","message":{"role":"assistant","content":"历史的"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.291032609Z","message":{"role":"assistant","content":"见证"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.325573882Z","message":{"role":"assistant","content":"。\n\n"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.360055075Z","message":{"role":"assistant","content":"\""},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.394575517Z","message":{"role":"assistant","content":"豫"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.429031499Z","message":{"role":"assistant","content":"园"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.463573883Z","message":{"role":"assistant","content":"及"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.498206243Z","message":{"role":"assistant","content":"商城"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.532779767Z","message":{"role":"assistant","content":"”："},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.567530373Z","message":{"role":"assistant","content":"豫"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.601984238Z","message":{"role":"assistant","content":"园"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.636503015Z","message":{"role":"assistant","content":"始建于"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.671091827Z","message":{"role":"assistant","content":"明代"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.705748153Z","message":{"role":"assistant","content":"，"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.74030126Z","message":{"role":"assistant","content":"是中国"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.77489374Z","message":{"role":"assistant","content":"南方"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.809531759Z","message":{"role":"assistant","content":"园林"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.844077845Z","message":{"role":"assistant","content":"建筑的"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.878674749Z","message":{"role":"assistant","content":"代表"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.913122158Z","message":{"role":"assistant","content":"。"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.947790869Z","message":{"role":"assistant","content":"豫"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:35.982317741Z","message":{"role":"assistant","content":"园"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.017024209Z","message":{"role":"assistant","content":"商城"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.051508233Z","message":{"role":"assistant","content":"则"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.08601788Z","message":{"role":"assistant","content":"是一个"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.12060447Z","message":{"role":"assistant","content":"集"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.155157966Z","message":{"role":"assistant","content":"购物"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.189693696Z","message":{"role":"assistant","content":"、"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.224233101Z","message":{"role":"assistant","content":"餐饮"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.258778113Z","message":{"role":"assistant","content":"、"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.293408116Z","message":{"role":"assistant","content":"娱乐"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.327981817Z","message":{"role":"assistant","content":"为一体的"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.362561936Z","message":{"role":"assistant","content":"商业"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.397029822Z","message":{"role":"assistant","content":"区"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.431596919Z","message":{"role":"assistant","content":"。\n\n"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.466154337Z","message":{"role":"assistant","content":"\""},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.500694428Z","message":{"role":"assistant","content":"田"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.535199046Z","message":{"role":"assistant","content":"子"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.569816106Z","message":{"role":"assistant","content":"坊"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.604400786Z","message":{"role":"assistant","content":"”："},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.639096194Z","message":{"role":"assistant","content":"它"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.673737633Z","message":{"role":"assistant","content":"是一个"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.708318353Z","message":{"role":"assistant","content":"充满"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.742884497Z","message":{"role":"assistant","content":"艺术"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.777447721Z","message":{"role":"assistant","content":"气息"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.812018152Z","message":{"role":"assistant","content":"的"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.846584797Z","message":{"role":"assistant","content":"创意"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.881128177Z","message":{"role":"assistant","content":"产业"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.915641048Z","message":{"role":"assistant","content":"聚集"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.950315734Z","message":{"role":"assistant","content":"区"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:36.984804477Z","message":{"role":"assistant","content":"。\n\n"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.019346375Z","message":{"role":"assistant","content":"这些"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.053957696Z","message":{"role":"assistant","content":"著名"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.08845107Z","message":{"role":"assistant","content":"景点"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.123007248Z","message":{"role":"assistant","content":"不仅"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.157571509Z","message":{"role":"assistant","content":"展示了"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.192129579Z","message":{"role":"assistant","content":"上海的"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.226633234Z","message":{"role":"assistant","content":"悠久"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.261219964Z","message":{"role":"assistant","content":"历史"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.295772792Z","message":{"role":"assistant","content":"和"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.330232997Z","message":{"role":"assistant","content":"独特的"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.364854404Z","message":{"role":"assistant","content":"文化"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.399489432Z","message":{"role":"assistant","content":"底蕴"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.434011916Z","message":{"role":"assistant","content":"，"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.468651895Z","message":{"role":"assistant","content":"同时也"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.503276169Z","message":{"role":"assistant","content":"吸引了"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.537887437Z","message":{"role":"assistant","content":"世界"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.572501397Z","message":{"role":"assistant","content":"各地的"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.607259788Z","message":{"role":"assistant","content":"游客"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.641892547Z","message":{"role":"assistant","content":"前来"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.676499929Z","message":{"role":"assistant","content":"观光"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.71112524Z","message":{"role":"assistant","content":"游览"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.745736113Z","message":{"role":"assistant","content":"。"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:37.780498755Z","message":{"role":"assistant","content":""},"done_reason":"stop","done":true,"total_duration":9999226996,"load_duration":6054386923,"prompt_eval_count":10,"prompt_eval_duration":210537000,"eval_count":109,"eval_duration":3733003000} # curl http://localhost:11434/api/chat -d '{ "model": "glm4", "messages": [ { "role": "user", "content": "介绍一下你自己" } ] }' {"model":"glm4","created_at":"2024-10-14T14:02:52.654030893Z","message":{"role":"assistant","content":"\""},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:52.694561966Z","message":{"role":"assistant","content":"如果"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:52.72306927Z","message":{"role":"assistant","content":"电磁"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:52.757496305Z","message":{"role":"assistant","content":"动态"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:52.792244873Z","message":{"role":"assistant","content":"和"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:52.826882222Z","message":{"role":"assistant","content":" "},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:52.861382188Z","message":{"role":"assistant","content":"申"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:52.895900925Z","message":{"role":"assistant","content":"玖"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:52.930406102Z","message":{"role":"assistant","content":" At"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:52.965083325Z","message":{"role":"assistant","content":"-"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:52.999605626Z","message":{"role":"assistant","content":" ►"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.034098069Z","message":{"role":"assistant","content":"ite"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.06854197Z","message":{"role":"assistant","content":"Bo"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.103194578Z","message":{"role":"assistant","content":"le"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.1378153Z","message":{"role":"assistant","content":" ever"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.172248342Z","message":{"role":"assistant","content":"\u003c/"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.206666723Z","message":{"role":"assistant","content":"eu"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.241183897Z","message":{"role":"assistant","content":" \n"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.275647604Z","message":{"role":"assistant","content":" Итал"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.310238486Z","message":{"role":"assistant","content":"’on"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.344762065Z","message":{"role":"assistant","content":"ibo"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.379315067Z","message":{"role":"assistant","content":" is"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.413705128Z","message":{"role":"assistant","content":"myp"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.448166387Z","message":{"role":"assistant","content":"ot"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.48250088Z","message":{"role":"assistant","content":".isDefined"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.517030324Z","message":{"role":"assistant","content":" "},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.551511697Z","message":{"role":"assistant","content":"6"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.585989644Z","message":{"role":"assistant","content":"25"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.620472775Z","message":{"role":"assistant","content":" a"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.654947054Z","message":{"role":"assistant","content":" E"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.68938128Z","message":{"role":"assistant","content":"_"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.723968144Z","message":{"role":"assistant","content":"...\n"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.758586713Z","message":{"role":"assistant","content":"卢"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.792985133Z","message":{"role":"assistant","content":"煜"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.827508001Z","message":{"role":"assistant","content":" E"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.861950884Z","message":{"role":"assistant","content":" _"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.896446482Z","message":{"role":"assistant","content":"Y"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.93097066Z","message":{"role":"assistant","content":"K"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:53.965436419Z","message":{"role":"assistant","content":"字"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.000019662Z","message":{"role":"assistant","content":"001"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.034540788Z","message":{"role":"assistant","content":"草"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.069032218Z","message":{"role":"assistant","content":"cos"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.103580788Z","message":{"role":"assistant","content":"K"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.13808331Z","message":{"role":"assistant","content":"_s"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.172515329Z","message":{"role":"assistant","content":" mos"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.207046631Z","message":{"role":"assistant","content":"pi"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.241586674Z","message":{"role":"assistant","content":"qm"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.276187563Z","message":{"role":"assistant","content":" A"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.310726205Z","message":{"role":"assistant","content":":semicolon"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.345260325Z","message":{"role":"assistant","content":"_EDEFAULT"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.379758056Z","message":{"role":"assistant","content":"ully"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.414314672Z","message":{"role":"assistant","content":" I"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.448885458Z","message":{"role":"assistant","content":"atchet"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.483470421Z","message":{"role":"assistant","content":" （"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.518406906Z","message":{"role":"assistant","content":"大"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.553149503Z","message":{"role":"assistant","content":"_console"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.587833797Z","message":{"role":"assistant","content":"/\n"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.622409599Z","message":{"role":"assistant","content":"COD"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.656863626Z","message":{"role":"assistant","content":" Ghost"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.691364169Z","message":{"role":"assistant","content":")"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.725771183Z","message":{"role":"assistant","content":"Highest"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.760400666Z","message":{"role":"assistant","content":" Check"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.794754518Z","message":{"role":"assistant","content":","},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.89839946Z","message":{"role":"assistant","content":" �."},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.933003312Z","message":{"role":"assistant","content":" 用户"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:54.967530848Z","message":{"role":"assistant","content":":"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.001969948Z","message":{"role":"assistant","content":"�真"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.036494328Z","message":{"role":"assistant","content":"个"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.071003108Z","message":{"role":"assistant","content":"�c"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.105533176Z","message":{"role":"assistant","content":"搜"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.140175282Z","message":{"role":"assistant","content":"行"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.174770388Z","message":{"role":"assistant","content":"\u003e="},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.20928064Z","message":{"role":"assistant","content":"-"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.243859118Z","message":{"role":"assistant","content":"106"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.278338777Z","message":{"role":"assistant","content":"K"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.312866693Z","message":{"role":"assistant","content":"YO"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.347278353Z","message":{"role":"assistant","content":"OW"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.381757985Z","message":{"role":"assistant","content":"LL"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.416375562Z","message":{"role":"assistant","content":"100"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.450901017Z","message":{"role":"assistant","content":"_t"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.485449013Z","message":{"role":"assistant","content":".fb"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.519997854Z","message":{"role":"assistant","content":"Pay"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.554507412Z","message":{"role":"assistant","content":" audi"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.589001463Z","message":{"role":"assistant","content":"ocommerce"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.6235425Z","message":{"role":"assistant","content":"HEMA"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.658033266Z","message":{"role":"assistant","content":" списк"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.692470904Z","message":{"role":"assistant","content":"201"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.72705665Z","message":{"role":"assistant","content":"号"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.761593663Z","message":{"role":"assistant","content":"，"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.796110808Z","message":{"role":"assistant","content":" And"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.83070236Z","message":{"role":"assistant","content":"yalty"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.865214288Z","message":{"role":"assistant","content":"pbs"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.89982706Z","message":{"role":"assistant","content":"参"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.934387743Z","message":{"role":"assistant","content":"为"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:55.968842486Z","message":{"role":"assistant","content":"保护"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:56.003398197Z","message":{"role":"assistant","content":"�c"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:56.037931096Z","message":{"role":"assistant","content":" \n"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:56.072475939Z","message":{"role":"assistant","content":"Im"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:56.107027308Z","message":{"role":"assistant","content":"refer"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:56.141598037Z","message":{"role":"assistant","content":"-m"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:56.176067668Z","message":{"role":"assistant","content":"oi"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:56.210628073Z","message":{"role":"assistant","content":" 自"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:56.245100112Z","message":{"role":"assistant","content":"带的"},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:56.279576845Z","message":{"role":"assistant","content":"["},"done":false} {"model":"glm4","created_at":"2024-10-14T14:02:56.314205225Z","message":{"role":"assistant","content":""},"done_reason":"stop","done":true,"total_duration":4021640842,"load_duration":9962400,"prompt_eval_count":8,"prompt_eval_duration":309739000,"eval_count":107,"eval_duration":3660121000} ```

The modelfile is:

FROM /home/ollama/AI-models/LLM/ollama-model/glm-4-9b-chat.Q4_K_S.gguf
TEMPLATE """[gMASK]<sop>{{ if .System }}<|system|>
{{ .System }}{{ end }}{{ if .Prompt }}<|user|>
{{ .Prompt }}{{ end }}<|assistant|>
{{ .Response }}"""
PARAMETER stop "<|system|>"
PARAMETER stop "<|user|>"
PARAMETER stop "<|assistant|>"

These are the pip list in my container:

Package                       Version
----------------------------- --------------
accelerate                    0.33.0
bigdl-core-cpp                2.6.0b20240919
blinker                       1.4
certifi                       2024.8.30
charset-normalizer            3.4.0
cryptography                  3.4.8
dbus-python                   1.2.18
distro                        1.7.0
distro-info                   1.1+ubuntu0.2
einops                        0.8.0
filelock                      3.16.1
fsspec                        2024.9.0
gguf                          0.10.0
httplib2                      0.20.2
huggingface-hub               0.25.2
idna                          3.10
importlib-metadata            4.6.4
ipex-llm                      2.2.0b20240919
jeepney                       0.7.1
Jinja2                        3.1.4
keyring                       23.5.0
launchpadlib                  1.10.16
lazr.restfulclient            0.14.4
lazr.uri                      1.0.6
MarkupSafe                    3.0.1
more-itertools                8.10.0
mpmath                        1.3.0
networkx                      3.4.1
numpy                         1.26.4
nvidia-cublas-cu12            12.1.3.1
nvidia-cuda-cupti-cu12        12.1.105
nvidia-cuda-nvrtc-cu12        12.1.105
nvidia-cuda-runtime-cu12      12.1.105
nvidia-cudnn-cu12             8.9.2.26
nvidia-cufft-cu12             11.0.2.54
nvidia-curand-cu12            10.3.2.106
nvidia-cusolver-cu12          11.4.5.107
nvidia-cusparse-cu12          12.1.0.106
nvidia-nccl-cu12              2.19.3
nvidia-nvjitlink-cu12         12.6.77
nvidia-nvtx-cu12              12.1.105
oauthlib                      3.2.0
packaging                     24.1
pip                           24.2
protobuf                      4.25.5
psutil                        6.0.0
PyGObject                     3.42.1
PyJWT                         2.3.0
pyparsing                     2.4.7
python-apt                    2.4.0+ubuntu4
PyYAML                        6.0.2
regex                         2024.9.11
requests                      2.32.3
safetensors                   0.4.5
SecretStorage                 3.3.1
sentencepiece                 0.1.99
setuptools                    59.6.0
six                           1.16.0
ssh-import-id                 5.11
sympy                         1.13.3
tiktoken                      0.8.0
tokenizers                    0.15.2
torch                         2.2.0
tqdm                          4.66.5
transformers                  4.36.2
transformers-stream-generator 0.0.5
triton                        2.2.0
typing_extensions             4.12.2
unattended-upgrades           0.1
urllib3                       2.2.3
wadllib                       1.3.6
wheel                         0.37.1
zipp                          1.0.0

The start script of ollama is:

#!/bin/bash

llm_model_name=${LLM_MODEL_NAME}
model="/home/ollama/AI-models/ollama-model/${llm_model_name}"
served_model_name=${llm_model_name}

export PATH=$PATH:/opt/python-3.11.9/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/python-3.11.9/lib/
export OLLAMA_NUM_GPU=999
export ZES_ENABLE_SYSMAN=1
source /opt/intel/oneapi/setvars.sh
export SYCL_CACHE_PERSISTENT=1
# [optional] under most circumstances, the following environment variable may improve performance, but sometimes this may also cause performance degradation
export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
# [optional] if you want to run on single GPU, use below command to limit GPU may improve performance
export ONEAPI_DEVICE_SELECTOR=level_zero:0

# expose the service in 0.0.0.0
export OLLAMA_HOST=0.0.0.0

# init ollama first
if [ ! -d "/home/${USERNAME}/WorkSpace/ollama" ]; then
  mkdir -p "/home/${USERNAME}/WorkSpace/ollama"
fi

cd /home/${USERNAME}/WorkSpace/ollama
init-ollama

nohup ./ollama serve > ./ollama.log 2>&1 &

# Wait a few seconds to ensure that the serve command has started
echo "=====ollama serve starting====="
sleep 5
echo "=====ollama serve started!====="
echo "=====ollama model creating====="
# create the model
./ollama create glm4 -f /home/ollama/AI-models/ollama-model/glm4_modelfile
echo "=====ollama model created!====="
tail -f ollama.log

Related issues:

Is it because of the wrong version of ollama installed to run glm4, or the format of modelfile is incorrect, which may leads to the wrong answer?

intel-analytics / ipex-llm

ollama generate incorrect answer when it run glm-4-9b-chat model #12193