huggingface / optimum-intel

🤗 Optimum Intel: Accelerate inference with Intel optimization tools
https://huggingface.co/docs/optimum/main/en/intel/index
Apache License 2.0
355 stars 99 forks source link

Fix TemporaryDirectory error when exporting models on Windows #749

Closed helena-intel closed 4 weeks ago

helena-intel commented 1 month ago

On Windows, exporting models results in a PermissionError because Python tries to delete the TemporaryDirectory containing OpenVINO model files when they were still in used. The model export works, but the error is confusing, it makes it difficult to debug issues on Windows, and the temporary directories are never deleted. This PR fixes that with the same method that is already used for stable diffusion models.

In [2]: model = OVModelForCausalLM.from_pretrained("hf-internal-testing/tiny-random-MistralForCausalLM", export=True)
c:\users\helena\venvs\optimum_312_env\Lib\site-packages\huggingface_hub\file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
Framework not specified. Using pt to export the model.
c:\users\helena\venvs\optimum_312_env\Lib\site-packages\huggingface_hub\file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
Using framework PyTorch: 2.3.0+cpu
Overriding 1 configuration item(s)
        - use_cache -> True
c:\users\helena\venvs\optimum_312_env\Lib\site-packages\transformers\modeling_utils.py:4481: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
  warnings.warn(
c:\users\helena\venvs\optimum_312_env\Lib\site-packages\transformers\modeling_attn_mask_utils.py:114: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if (input_shape[-1] > 1 or self.sliding_window is not None) and self.is_causal:
c:\users\helena\venvs\optimum_312_env\Lib\site-packages\optimum\exporters\onnx\model_patcher.py:300: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if past_key_values_length > 0:
c:\users\helena\venvs\optimum_312_env\Lib\site-packages\transformers\models\mistral\modeling_mistral.py:119: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if seq_len > self.max_seq_len_cached:
c:\users\helena\venvs\optimum_312_env\Lib\site-packages\transformers\models\mistral\modeling_mistral.py:662: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attention_mask.size() != (bsz, 1, q_len, kv_seq_len):
Compiling the model to CPU ...
Exception ignored in: <finalize object at 0x174ceb52cc0; dead>
Traceback (most recent call last):
  File "C:\Users\helena\AppData\Local\Programs\Python\Python312\Lib\weakref.py", line 590, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\helena\AppData\Local\Programs\Python\Python312\Lib\tempfile.py", line 935, in _cleanup
    cls._rmtree(name, ignore_errors=ignore_errors)
  File "C:\Users\helena\AppData\Local\Programs\Python\Python312\Lib\tempfile.py", line 930, in _rmtree
    _shutil.rmtree(name, onexc=onexc)
  File "C:\Users\helena\AppData\Local\Programs\Python\Python312\Lib\shutil.py", line 820, in rmtree
    return _rmtree_unsafe(path, onexc)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\helena\AppData\Local\Programs\Python\Python312\Lib\shutil.py", line 648, in _rmtree_unsafe
    onexc(os.unlink, fullname, err)
  File "C:\Users\helena\AppData\Local\Programs\Python\Python312\Lib\tempfile.py", line 905, in onexc
    _os.unlink(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\helena\\AppData\\Local\\Temp\\tmppf_5t6hr\\openvino_model.bin'
HuggingFaceDocBuilderDev commented 1 month ago

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.