Update tokenizer.py

[ginnm]

The tokenize function can return index now. It would be helpful for developing https://huggingface.co/docs/tokenizers/api/pre-tokenizers for Java.