search

intel / intel-extension-for-pytorch

A Python package for extending the official PyTorch that can easily obtain performance on Intel platform
Apache License 2.0
1.58k stars 244 forks source link

'<' not supported between instances of 'function' and 'function' when import ipex #316

Open cyita opened 1 year ago

cyita commented 1 year ago

Describe the bug

In the python console, I can import intel_extension_for_pytorch as ipex without error, but after executing some code, import intel_extension_for_pytorch as ipex will raise the error '<' not supported between instances of 'function' and 'function':

Error Message:

Traceback (most recent call last):
  File "/root/yina/dev/client-stable-diffusion/invokeai/backend/invoke_ai_web_server.py", line 1476, in generate_images
    self.generate.prompt2image(
  File "/root/yina/dev/client-stable-diffusion/ldm/generate.py", line 484, in prompt2image
    model = self.set_model(self.model_name, nano_device=nano, controlnet=controlnet, preprocessers=preprocessers, lora_info=lora_info)
  File "/root/yina/dev/client-stable-diffusion/ldm/generate.py", line 1034, in set_model
    self.set_nano_optim(nano_device=nano_device, controlnet=controlnet, preprocessers=preprocessers, lora_info=lora_info)
  File "/root/yina/dev/client-stable-diffusion/ldm/generate.py", line 1136, in set_nano_optim
    self.model.vae = optimize_vae(self.model.vae, **optim_args)
  File "/root/yina/dev/client-stable-diffusion/ldm/invoke/optimize/nano_optimize.py", line 199, in optimize_vae
    nano_vae_decoder = nano_optimize_model(vae.decoder, input_sample, accelerator=accelerator, ipex=ipex, precision=precision,
  File "/root/yina/dev/client-stable-diffusion/ldm/invoke/optimize/nano_optimize.py", line 375, in nano_optimize_model
    optimized_model = InferenceOptimizer.quantize(model,
  File "/opt/intel/oneapi/intelpython/latest/envs/yina-diffusers/lib/python3.9/site-packages/bigdl/nano/pytorch/inference/optimizer.py", line 790, in quantize
    return PytorchIPEXJITBF16Model(model, input_sample=input_sample,
  File "/opt/intel/oneapi/intelpython/latest/envs/yina-diffusers/lib/python3.9/site-packages/bigdl/nano/deps/ipex/ipex_api.py", line 104, in PytorchIPEXJITBF16Model
    return PytorchIPEXJITBF16Model(model, input_sample=input_sample, use_ipex=use_ipex,
  File "/opt/intel/oneapi/intelpython/latest/envs/yina-diffusers/lib/python3.9/site-packages/bigdl/nano/deps/ipex/ipex_inference_bf16_model.py", line 74, in __init__
    PytorchIPEXJITModel.__init__(self, model, input_sample=input_sample, use_ipex=use_ipex,
  File "/opt/intel/oneapi/intelpython/latest/envs/yina-diffusers/lib/python3.9/site-packages/bigdl/nano/deps/ipex/ipex_inference_model.py", line 119, in __init__
    self.model = ipex_optimize(self.model, dtype=dtype, inplace=inplace,
  File "/opt/intel/oneapi/intelpython/latest/envs/yina-diffusers/lib/python3.9/site-packages/bigdl/nano/deps/ipex/ipex_api.py", line 24, in ipex_optimize
    import intel_extension_for_pytorch as ipex
  File "/opt/intel/oneapi/intelpython/latest/envs/yina-diffusers/lib/python3.9/site-packages/intel_extension_for_pytorch/__init__.py", line 34, in <module>
    import intel_extension_for_pytorch.xpu
  File "/opt/intel/oneapi/intelpython/latest/envs/yina-diffusers/lib/python3.9/site-packages/intel_extension_for_pytorch/xpu/__init__.py", line 415, in <module>
    serialization.register_package(30, _xpu_tag, _xpu_deserialize)
  File "/opt/intel/oneapi/intelpython/latest/envs/yina-diffusers/lib/python3.9/site-packages/torch/serialization.py", line 93, in register_package
    _package_registry.sort()
TypeError: '<' not supported between instances of 'function' and 'function'

Versions

Collecting environment information... PyTorch version: 1.13.1+cu117 PyTorch CXX11 ABI: No IPEX version: 1.13.100 IPEX commit: ef12c70 Build type: release

OS: Rocky Linux 8.7 (Green Obsidian) (x86_64) GCC version: (GCC) 8.5.0 20210514 (Red Hat 8.5.0-16) Clang version: N/A IGC version: 2023.0.0 (2023.0.0.20221201) CMake version: version 3.26.1 Libc version: glibc-2.28

Python version: 3.9.15 (main, Nov 11 2022, 13:58:57) [GCC 11.2.0] (64-bit runtime) Python platform: Linux-6.2.7-1.el8.elrepo.x86_64-x86_64-with-glibc2.28 Is XPU available: False DPCPP runtime version: 2023.0.0 MKL version: 2023.0.0 GPU models and configuration:

Intel OpenCL ICD version: N/A Level Zero version: N/A

CPU: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 192 On-line CPU(s) list: 0-191 Thread(s) per core: 2 Core(s) per socket: 48 Socket(s): 2 NUMA node(s): 2 Vendor ID: GenuineIntel BIOS Vendor ID: Intel(R) Corporation CPU family: 6 Model: 143 Model name: Intel (R) Xeon (R) CPU Max 9468 BIOS Model name: Intel (R) Xeon (R) CPU Max 9468 Stepping: 8 CPU MHz: 2420.441 CPU max MHz: 3500.0000 CPU min MHz: 800.0000 BogoMIPS: 4200.00 Virtualization: VT-x L1d cache: 48K L1i cache: 32K L2 cache: 2048K L3 cache: 107520K NUMA node0 CPU(s): 0-47,96-143 NUMA node1 CPU(s): 48-95,144-191 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cat_l2 cdp_l3 invpcid_single intel_ppin cdp_l2 ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local split_lock_detect avx_vnni avx512_bf16 wbnoinvd dtherm ida arat pln pts hfi avx512vbmi umip pku ospke waitpkg avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq la57 rdpid bus_lock_detect cldemote movdiri movdir64b enqcmd fsrm md_clear serialize tsxldtrk pconfig arch_lbr ibt amx_bf16 avx512_fp16 amx_tile amx_int8 flush_l1d arch_capabilities

Versions of relevant libraries: [pip3] clip-anytorch==2.5.2 [pip3] intel-extension-for-pytorch==1.13.100 [pip3] mypy-extensions==1.0.0 [pip3] numpy==1.23.1 [pip3] pytorch-lightning==1.6.4 [pip3] torch==1.13.1 [pip3] torch-fidelity==0.3.0 [pip3] torchdiffeq==0.2.3 [pip3] torchmetrics==0.11.0 [pip3] torchsde==0.2.5 [pip3] torchvision==0.14.1 [conda] clip-anytorch 2.5.2 pypi_0 pypi [conda] intel-extension-for-pytorch 1.13.100 pypi_0 pypi [conda] numpy 1.23.1 pypi_0 pypi [conda] pytorch-lightning 1.6.4 pypi_0 pypi [conda] torch 1.13.1 pypi_0 pypi [conda] torch-fidelity 0.3.0 pypi_0 pypi [conda] torchdiffeq 0.2.3 pypi_0 pypi [conda] torchmetrics 0.11.0 pypi_0 pypi [conda] torchsde 0.2.5 pypi_0 pypi [conda] torchvision 0.14.1 pypi_0 pypi

jgong5 commented 1 year ago

I guess this is because this particular line of code (serialization.register_package(30, _xpu_tag, _xpu_deserialize)) is executed twice which means that this xpu/__init__.py is evaluated twice. Does it mean "ipex" is imported twice? Why would it happen?

Note that inside register_package, the new package will be appended into the _package_registry without checking the duplication while the tagger and deserializer are functors that cannot be compared by sort() when priority is the same (called twice with the same priority).

def register_package(priority, tagger, deserializer):
    queue_elem = (priority, tagger, deserializer)
    _package_registry.append(queue_elem)
    _package_registry.sort()
cyita commented 1 year ago

Yes, this error occurs when I try to import IPEX for the second time. Due to the complexity of the application, It's difficult to avoid repeated imports of IPEX. As a workaround, we should import intel_extension_for_pytorch.xpu at the beginning of the application or comment it in IPEX. I think neither of these approaches is a good solution.

jgong5 commented 1 year ago

Yes, this error occurs when I try to import IPEX for the second time. Due to the complexity of the application, It's difficult to avoid repeated imports of IPEX. As a workaround, we should import intel_extension_for_pytorch.xpu at the beginning of the application or comment it in IPEX. I think neither of these approaches is a good solution.

But I don't think Python only invokes a module init once if it is imported twice?