bug: Aya 23 8B Q4 get garbage response

[HamidrezaSafari]

Jan version

0.5.7

Describe the Bug

when i use Aya 23 8B Q4 get garbage response like this " )."EOS_TOKEN>!1$-036DK42'P#JRL<SEP><BOS_TOKEN>%/5ECO(T,N@[<CLS.V!XZF]SEOS_TOKEN>B4^G3#"$IYK6%E07<MASK_TOKEN>DH<EOP_TOKEN>,PT<PAD>2V'QC1<SEP><BOS_TOKEN>(UR/LFGSXJZM<UNK>-@>!O^[+IK4)N3T.<EOS_TOKENW,](EV6#2GQHM"-\LU%Y0BRD)^&4P/>5T3"

Steps to Reproduce

No response

Screenshots / Logs

No response

What is your OS?

• [X] MacOS
• [ ] Windows
• [ ] Linux

[dan-homebrew]

Transferring this to model repo @hahuyhoang411

[hahuyhoang411]

The current model is using model from bartowski. I will convert this with our own pipeline for further investigation. Track here: https://github.com/janhq/models/issues/54

[hahuyhoang411]

The problem is related to the underline engine Cortex. We will update Jan to fix the problem in the next update. Thank you for your report @HamidrezaSafari

Related issue: https://github.com/janhq/models/issues/54