search

microsoft / Olive

Olive: Simplify ML Model Finetuning, Conversion, Quantization, and Optimization for CPUs, GPUs and NPUs.
https://microsoft.github.io/Olive/
MIT License
1.56k stars 164 forks source link

getting error while running llama2/bert on GPU #1279

Open himanshushukla12 opened 2 months ago

himanshushukla12 commented 2 months ago

when I run the command:

! python3 -m olive.workflows.run --config config_gpu.json

I'm getting the issue:

/home/z004x2xz/WorkAssignedByMatt/Olive/venvOlive3.11/lib/python3.11/site-packages/olive/cli/launcher.py:56: FutureWarning: Running `python -m olive.workflows.run` is deprecated and might be removed in the future. Please use `olive run` or `python -m olive run` instead.
  warn(
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/z004x2xz/WorkAssignedByMatt/Olive/venvOlive3.11/lib/python3.11/site-packages/olive/workflows/run/__main__.py", line 10, in <module>
    legacy_call("olive.workflows.run", "run", *sys.argv[1:])
  File "/home/z004x2xz/WorkAssignedByMatt/Olive/venvOlive3.11/lib/python3.11/site-packages/olive/cli/launcher.py", line 64, in legacy_call
    main(raw_args)
  File "/home/z004x2xz/WorkAssignedByMatt/Olive/venvOlive3.11/lib/python3.11/site-packages/olive/cli/launcher.py", line 44, in main
    service.run()
  File "/home/z004x2xz/WorkAssignedByMatt/Olive/venvOlive3.11/lib/python3.11/site-packages/olive/cli/run.py", line 42, in run
    olive_run(**var_args)
  File "/home/z004x2xz/WorkAssignedByMatt/Olive/venvOlive3.11/lib/python3.11/site-packages/olive/workflows/run/run.py", line 284, in run
    run_config = RunConfig.parse_file_or_obj(run_config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/z004x2xz/WorkAssignedByMatt/Olive/venvOlive3.11/lib/python3.11/site-packages/olive/common/config_utils.py", line 127, in parse_file_or_obj
    return cls.parse_file(file_path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/z004x2xz/WorkAssignedByMatt/Olive/venvOlive3.11/lib/python3.11/site-packages/pydantic/v1/main.py", line 569, in parse_file
    return cls.parse_obj(obj)
           ^^^^^^^^^^^^^^^^^^
  File "/home/z004x2xz/WorkAssignedByMatt/Olive/venvOlive3.11/lib/python3.11/site-packages/pydantic/v1/main.py", line 526, in parse_obj
    return cls(**obj)
           ^^^^^^^^^^
  File "/home/z004x2xz/WorkAssignedByMatt/Olive/venvOlive3.11/lib/python3.11/site-packages/pydantic/v1/main.py", line 339, in __init__
    values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/z004x2xz/WorkAssignedByMatt/Olive/venvOlive3.11/lib/python3.11/site-packages/pydantic/v1/main.py", line 1074, in validate_model
    v_, errors_ = field.validate(value, values, loc=field.alias, cls=cls_)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/z004x2xz/WorkAssignedByMatt/Olive/venvOlive3.11/lib/python3.11/site-packages/pydantic/v1/fields.py", line 864, in validate
    v, errors = self._apply_validators(v, values, loc, cls, self.pre_validators)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/z004x2xz/WorkAssignedByMatt/Olive/venvOlive3.11/lib/python3.11/site-packages/pydantic/v1/fields.py", line 1154, in _apply_validators
    v = validator(cls, v, values, self, self.model_config)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/z004x2xz/WorkAssignedByMatt/Olive/venvOlive3.11/lib/python3.11/site-packages/pydantic/v1/class_validators.py", line 306, in <lambda>
    return lambda cls, v, values, field, config: validator(cls, v, values=values)
                                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/z004x2xz/WorkAssignedByMatt/Olive/venvOlive3.11/lib/python3.11/site-packages/olive/workflows/run/config.py", line 161, in insert_aml_client
    input_model_path = v["config"].get("model_path")
                       ~^^^^^^^^^^
KeyError: 'config'

OS Version: Description: Ubuntu 22.04.3 LTS Release: 22.04 Codename: jammy Olive Version: ONNXRuntime Version: Version: 1.18.1

I'm using the notebook given in the examples

MayuraRam commented 2 months ago

I am getting a similar error when I run the following: python -m olive.workflows.run --config whisper_cpu_int8.json --setup

MayuraRam commented 2 months ago

I was able to avoid the KeyError: 'config' issue by installing Olive from source - as noted here - https://github.com/microsoft/Olive/blob/main/examples/README.md

MayuraRam commented 2 months ago

But now I get the following errors when I try to look at the quantized model that was created: [32856:0806/092327.441:ERROR:cache_util_win.cc(20)] Unable to move the cache: Access is denied. (0x5) [32856:0806/092327.441:ERROR:disk_cache.cc(208)] Unable to create cache

xiaoyu-work commented 2 months ago

@himanshushukla12 can you please install Olive from source with main branch and try again?

xiaoyu-work commented 2 months ago

@MayuraRam can you please open a new issue and paste the full logs of your error?

prasad-pr-20 commented 3 weeks ago

Hi All, Any fix wrt to this issue. Even the example files are failing here.