search

gabrielmbmb / candle-holder

A Rust crate offering similar functionality to the Python transformers package using Candle.
Apache License 2.0
13 stars 0 forks source link

why not use the candle Bert models and Tokenizer::from_file ? #2

Closed jondot closed 3 months ago

jondot commented 7 months ago

Hi, reading the code I noticed something odd, it seems that the Bert model is re-implemented vs the candle models, and Tokenizer could have been used as-is to load (instead of manual loading from vocab). In addition in terms of timeline seems this project comes after these were present in candle.

Curious to see what I'm missing, is there anything here to note?

gabrielmbmb commented 6 months ago

Hi @jondot,

at first I tried using candle-transformers models, but there are many structs that are not public, therefore cannot be imported from candle-holder. Also, to implement PreTrainedModel trait from candle-holder was easy to do this way.

I'm using Tokenizer::from_file at first method to build the tokenizer for the model, but there are sometimes that a model from the Hub was uploaded a long time ago, when https://github.com/huggingface/tokenizers was not used, and as a second method I build the Tokenizer using the vocab.