Can't install tiktoken==0.4.0 or tiktoken==0.5.1in Python 3.12

[JackObid]

If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.

  To update pip, run:

      pip install --upgrade pip

  and then retry package installation.

  If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for tiktoken Failed to build tiktoken ERROR: Could not build wheels for tiktoken, which is required to install pyproject.toml-based projects

[JackObid]

This eventual worked RUN pip install --upgrade pip && pip install setuptools==69.0.1 RUN pip install tiktoken==0.5.2

[hauntsaninja]

Versions of tiktoken before 0.5.2 did not build Python3.12 wheels, see https://github.com/openai/tiktoken/blob/main/CHANGELOG.md#v052 If you want to use older versions with Python 3.12, you'll need to compile from sdist, which will require a rust compiler as the error message above is telling you.