Maybe insight into closed issue 46?

[TradingPlacesResearch]

Reference: https://github.com/openai/tiktoken/issues/46

M1 Mac macOS 13.3.1 python 3.9 and 3.10

Trying to install Microsoft guidance, which uses titoken as a dependency. First I got this:

ERROR: Could not find a version that satisfies the requirement tiktoken>=0.3 (from guidance) (from versions: none)
ERROR: No matching distribution found for tiktoken>=0.3 (from guidance)

So I tried pip install tiktoken and this error

ERROR: Could not find a version that satisfies the requirement tiktoken (from versions: none)
ERROR: No matching distribution found for tiktoken

Just going through the issues I thought I would try and force python 3.9 with python3.9 -m pip install tiktoken. That worked, and subsequently so did python3.9 -m pip install guidance

Anyway, my guess is those closed issues were also ARM Macs, and this is a specific issue with having 3.10 installed. Thanks.

[hauntsaninja]

You're probably not using the Python you think you're using, which is why pip install tiktoken fails, while python3.9 -m pip install tiktoken works. I'd figure out what Python that pip is pointing to, but yeah, in general python3.9 -m pip install tiktoken or python3.10 -m pip install tiktoken is going to be harder to mess up.

This issue is not going to be related to ARM or Mac

[arianpasquali]

Same error as described by @TradingPlacesResearch trying to install tiktoken to use with guidance. This time ubuntu.