Public token <-> id conversion + Unknown Tokens

[pcuenca]

I used an API similar to transformers. To do (in new PRs, potentially):

• Some tests
• Use unknown token id
• Handle special tokens

cc @ZachNagengast, feel free to review!

[pcuenca]

@ZachNagengast I further checked the Python code and brought over the relevant changes:

• The unknown token and unknown token id are now read from the tokenizer configuration.
• They can be nil. This happens, for example, in the Falcon tokenizer.

Tests now pass for the various tokenizers we are considering, and comparing against the well-known Python unknown token ids.