Closed MaxAkbar closed 1 month ago
Trying to debug the ModuleNotFoundError for triton, while then, just wanted to suggest the import statement in line 1 is wrong.
It should be :
from mistral_inference.transformer import Transformer
Edit: Regarding ModuleNotfoundError, Windows does not have support for triton and thus you are getting this error, but xformers runs without it too just triton based optimizations will not take place.
@bhargavyagnik Thank you good sir that was it. Now I do get the error, but it still generates the response. I will go ahead and close this ticket.
python simplechat.py
A matching Triton is not available, some optimizations will not be enabled
Traceback (most recent call last):
File "C:\Users\\****\anaconda3\envs\mistral\Lib\site-packages\xformers\__init__.py", line 57, in _is_triton_available
import triton # noqa
^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'triton'
Machine Learning (ML) is a subset of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed.
In simpler terms, Machine Learning is a way of teaching computers to learn by themselves, allowing them to make decisions or predictions based on data. This is
EDIT: By the way that script was from the huggingface for Mistralai model sheet.
Yeah, I think the model sheet has yet to be updated. PR for the change has not been reviewed.
Python -VV
Pip Freeze
Reproduction Steps
from mistral_common.tokens.tokenizers.mistral import MistralTokenizer from mistral_common.protocol.instruct.messages import UserMessage from mistral_common.protocol.instruct.request import ChatCompletionRequest
tokenizer = MistralTokenizer.from_file(f"{mistral_models_path}/tokenizer.model.v3") model = Transformer.from_folder(mistral_models_path)
completion_request = ChatCompletionRequest(messages=[UserMessage(content="Explain Machine Learning to me in a nutshell.")])
tokens = tokenizer.encode_chat_completion(completion_request).tokens
outtokens, = generate([tokens], model, max_tokens=64, temperature=0.0, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id) result = tokenizer.instruct_tokenizer.tokenizer.decode(out_tokens[0])
print(result)