openvinotoolkit / openvino.genai

Run Generative AI models using native OpenVINO C++ API
Apache License 2.0
99 stars 136 forks source link

cannot generate models for cpp sample. #635

Closed xczhai closed 1 month ago

xczhai commented 1 month ago

When run optimum-cli export openvino --trust-remote-code --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 TinyLlama-1.1B-Chat-v1.0 locally, it reports like:

Traceback (most recent call last):
  File "//workspace/llm_dev/venv/llm/bin/optimum-cli", line 8, in <module>
    sys.exit(main())
  File "//workspace/llm_dev/venv/llm/lib/python3.10/site-packages/optimum/commands/optimum_cli.py", line 163, in main
    service.run()
  File "//workspace/llm_dev/venv/llm/lib/python3.10/site-packages/optimum/commands/export/openvino.py", line 310, in run
    task = infer_task(self.args.task, self.args.model)
  File "//workspace/llm_dev/venv/llm/lib/python3.10/site-packages/optimum/exporters/openvino/__main__.py", line 51, in infer_task
    task = TasksManager.infer_task_from_model(model_name_or_path)
  File "//workspace/llm_dev/venv/llm/lib/python3.10/site-packages/optimum/exporters/tasks.py", line 1668, in infer_task_from_model
    task = cls._infer_task_from_model_name_or_path(model, subfolder=subfolder, revision=revision)
  File "//workspace/llm_dev/venv/llm/lib/python3.10/site-packages/optimum/exporters/tasks.py", line 1581, in _infer_task_from_model_name_or_path
    raise RuntimeError(
RuntimeError: Cannot infer the task from a local directory yet, please specify the task manually (masked-im, question-answering, image-to-image, zero-shot-object-detection, feature-extraction, object-detection, automatic-speech-recognition, text-generation, token-classification, audio-xvector, stable-diffusion-xl, text-classification, depth-estimation, audio-frame-classification, image-classification, mask-generation, image-to-text, text-to-audio, stable-diffusion, image-segmentation, fill-mask, multiple-choice, sentence-similarity, audio-classification, text2text-generation, zero-shot-image-classification, conversational, semantic-segmentation).
Wovchena commented 1 month ago

You have TinyLlama folder which contains incomplete model. Remove or rename it

xczhai commented 1 month ago

I checked the SHA256. They are same.

Wovchena commented 1 month ago

How did you check SHA256 of a folder? :)

xczhai commented 1 month ago
xczhai commented 1 month ago