Issues with documentation and failure to tokenize

The documentation in the article has the following issues.

The following will not work: optimum-cli export openvino --model meta-llama/Llama-3.2-3B-Instruct --task text-generation-with-past --weight-format int4 --group-size 64 --ratio 1.0 --sym --awq --scale-estimation --dataset 'wikitext2' --all-layers llama-3.2-3b-instruct-INT4

Solution: 'wikitext2' must be wrapped in double quotes "wikitext2"

After about 30 minutes installation (Windows) it decides to check the versions and then fails. There is no OpenVINO 2024.5 so the commands provided will not work.

If you downgrade you then get the following: openvino-genai 2024.5.0.0.dev20241024 requires openvino_tokenizers~=2024.5.0.0.dev, but you have openvino-tokenizers 2024.4.1.0.dev20240926 which is incompatible.

Solution: pip install openvino==2024.4.0 openvino-tokenizers==2024.4.0.0 openvino-genai==2024.4.0

Then things go a little more smoothly, until you notice this little gem on the last line: Exporting tokenizers to OpenVINO is not supported for tokenizers version > 0.19 and openvino version <= 2024.4. Please downgrade to tokenizers version <= 0.19 to export tokenizers to OpenVINO.

So you think OK, no biggy, just downgrade:

Installing collected packages: tokenizers
  Attempting uninstall: tokenizers
    Found existing installation: tokenizers 0.20.1
    Uninstalling tokenizers-0.20.1:
      Successfully uninstalled tokenizers-0.20.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
transformers 4.45.2 requires tokenizers<0.21,>=0.20, but you have tokenizers 0.19.0 which is incompatible.
Successfully installed tokenizers-0.19.0

After deleting the venv and starting again:

(ov-env) C:\LLMs\openvino.genai>pip list
Package             Version
------------------- ----------------------
numpy               2.1.2
openvino            2024.5.0.dev20241024
openvino-genai      2024.5.0.0.dev20241024
openvino-telemetry  2024.1.0
openvino-tokenizers 2024.5.0.0.dev20241024
packaging           24.1
pip                 24.2

All good, after optimum-cli command:

OpenVINO and OpenVINO Tokenizers versions are not binary compatible.
OpenVINO version:            2024.4.0-16579
OpenVINO Tokenizers version: 2024.5.0.0
First 3 numbers should be the same. Update OpenVINO Tokenizers to compatible version. It is recommended to use the same day builds for pre-release version. To install both OpenVINO and OpenVINO Tokenizers release version perform:
pip install --force-reinstall openvino openvino-tokenizers
To update both OpenVINO and OpenVINO Tokenizers to the latest pre-release version perform:
pip install --pre -U openvino openvino-tokenizers --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
Tokenizer won't be converted.

Try to get it back to 2024.5.0.dev20241024: pip install --force-reinstall openvino==2024.5.0.dev20241024

ERROR: Could not find a version that satisfies the requirement openvino==2024.5.0.dev20241024 (from versions: 2024.1.0, 2024.2.0, 2024.3.0, 2024.4.0, 2024.4.1.dev20240926)
ERROR: No matching distribution found for openvino==2024.5.0.dev20241024

Yet pip list:

openvino                  2024.5.0.dev20241024
openvino-genai            2024.5.0.0.dev20241024
openvino-telemetry        2024.1.0
openvino-tokenizers       2024.5.0.0.dev20241024

openvinotoolkit / openvino.genai

Issues with documentation and failure to tokenize #1069