AmineDiro / cria

OpenAI compatible API for serving LLAMA-2 model
MIT License
215 stars 13 forks source link

freeze tokenizers dependency to 0.13.3 #9

Closed bringitup closed 1 year ago

bringitup commented 1 year ago

Froze tokenizers dep to v0.13.3 to prevent build error on decode signature which change in v0.13.4 Build errors are coming from tokenizers -> llm-base -> cria :

error[E0308]: mismatched types
   --> llm/crates/llm-base/src/tokenizer/huggingface.rs:25:21
    |
25  |             .decode(vec![idx as u32], true)
    |              ------ ^^^^^^^^^^^^^^^^ expected `&[u32]`, found `Vec<u32>`
    |              |
    |              arguments to this method are incorrect
    |
    = note: expected reference `&[u32]`
                  found struct `Vec<u32>`
AmineDiro commented 1 year ago

Thanks for the PR. Normally, we only use ggml model so we wouldn't depend on external tokenizers 🤔

bringitup commented 1 year ago

This error is raised at build time, from fresh clone you could easily replicate with

cargo b --release

Sure this should and will be quickly fixed by llm-base direct dependency, so this PR will not be necessary in future

AmineDiro commented 1 year ago

I can't replicate this issue using both cargo b --release and cargo b --release --features {metal, cublas}... 🤔

bringitup commented 1 year ago

I can still replicate it using fresh cria repo The issue has been already shared in tokenizers repo : https://github.com/huggingface/tokenizers/issues/1323