def detokenize(self, tokens: list):
"""
Returns a list of tokens for the text <- wrong description
:param text: text to be tokenized <- wrong description
:return: A string representing the text extracted from the tokens
"""
return pp.llama_tokens_to_str(self._ctx, tokens)
This function was introduced in a PR, so probably the developer was doing copy/paste from the tokenize function.
I will try to fix that in the next release.