search

jtydhr88 / ComfyUI-Unique3D

ComfyUI Unique3D is custom nodes that running AiuniAI/Unique3D into ComfyUI
Apache License 2.0
192 stars 13 forks source link

CUDA error: no kernel image is available #27

Open Obambaaaa opened 2 months ago

Obambaaaa commented 2 months ago 
E:\Desktop\ComfyUI_3d>.\python_embeded\python.exe -s ComfyUI\main.py --windows-standalone-build
[START] Security scan
[DONE] Security scan
## ComfyUI-Manager: installing dependencies done.
** ComfyUI startup time: 2024-08-21 19:04:11.628411
** Platform: Windows
** Python version: 3.11.9 (tags/v3.11.9:de54cf5, Apr  2 2024, 10:12:12) [MSC v.1938 64 bit (AMD64)]
** Python executable: E:\Desktop\ComfyUI_3d\python_embeded\python.exe
** ComfyUI Path: E:\Desktop\ComfyUI_3d\ComfyUI
** Log path: E:\Desktop\ComfyUI_3d\comfyui.log

Prestartup times for custom nodes:
   8.1 seconds: E:\Desktop\ComfyUI_3d\ComfyUI\custom_nodes\ComfyUI-Manager

Total VRAM 8191 MB, total RAM 16335 MB
pytorch version: 2.3.0+cu121
xformers version: 0.0.26.post1
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3070 : cudaMallocAsync
Using xformers cross attention
[Prompt Server] web root: E:\Desktop\ComfyUI_3d\ComfyUI\web
Warn!: xFormers is available (Attention)
Warn!: E:\Desktop\ComfyUI_3d\python_embeded\Lib\site-packages\torch\utils\cpp_extension.py:1967: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation.
If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
  warnings.warn(

### Loading: ComfyUI-Manager (V2.50)
### ComfyUI Revision: 2514 [75b9b55b] *DETACHED | Released on '2024-08-10'
python_executable_path:  E:\Desktop\ComfyUI_3d\python_embeded\python.exe
python_embeded_path:  E:\Desktop\ComfyUI_3d\python_embeded
Warn!: Traceback (most recent call last):
  File "E:\Desktop\ComfyUI_3d\ComfyUI\nodes.py", line 1941, in load_custom_node
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 936, in exec_module
  File "<frozen importlib._bootstrap_external>", line 1073, in get_code
  File "<frozen importlib._bootstrap_external>", line 1130, in get_data
FileNotFoundError: [Errno 2] No such file or directory: 'E:\\Desktop\\ComfyUI_3d\\ComfyUI\\custom_nodes\\Unique3D\\__init__.py'

Warn!: Cannot import E:\Desktop\ComfyUI_3d\ComfyUI\custom_nodes\Unique3D module for custom nodes: [Errno 2] No such file or directory: 'E:\\Desktop\\ComfyUI_3d\\ComfyUI\\custom_nodes\\Unique3D\\__init__.py'

Import times for custom nodes:
   0.0 seconds: E:\Desktop\ComfyUI_3d\ComfyUI\custom_nodes\ComfyUI-Unique3D
   0.0 seconds (IMPORT FAILED): E:\Desktop\ComfyUI_3d\ComfyUI\custom_nodes\Unique3D
   0.1 seconds: E:\Desktop\ComfyUI_3d\ComfyUI\custom_nodes\ComfyUI-InstantMesh
   0.1 seconds: E:\Desktop\ComfyUI_3d\ComfyUI\custom_nodes\websocket_image_save.py
   1.0 seconds: E:\Desktop\ComfyUI_3d\ComfyUI\custom_nodes\ComfyUI-Manager
  58.7 seconds: E:\Desktop\ComfyUI_3d\ComfyUI\custom_nodes\ComfyUI-3D-Pack

Starting server

To see the GUI go to: http://127.0.0.1:8188
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json
FETCH DATA from: E:\Desktop\ComfyUI_3d\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json [DONE]
got prompt
2024-08-21 19:06:26.6042688 [E:onnxruntime:Default, provider_bridge_ort.cc:1978 onnxruntime::TryGetProviderInfo_TensorRT] D:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1637 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "E:\Desktop\ComfyUI_3d\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_providers_tensorrt.dll"

*************** EP Error ***************
EP Error D:\a\_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:490 onnxruntime::python::RegisterTensorRTPluginsAsCustomOps Please install TensorRT libraries as mentioned in the GPU requirements page, make sure they're in the PATH or LD_LIBRARY_PATH, and that your GPU is supported.
 when using [('TensorrtExecutionProvider', {'device_id': 0, 'trt_max_workspace_size': 8589934592, 'trt_fp16_enable': True, 'trt_engine_cache_enable': True}), ('CUDAExecutionProvider', {'device_id': 0, 'arena_extend_strategy': 'kSameAsRequested', 'gpu_mem_limit': 8589934592, 'cudnn_conv_algo_search': 'HEURISTIC'})]
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.
****************************************
2024-08-21 19:06:27.9912705 [E:onnxruntime:Default, provider_bridge_ort.cc:1992 onnxruntime::TryGetProviderInfo_CUDA] D:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1637 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "E:\Desktop\ComfyUI_3d\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

PyTorch version 2.3.0+cu121 available.
Warning! extra parameter in cli is not verified, may cause erros.
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 5/5 [00:01<00:00,  3.24it/s]
You have disabled the safety checker for <class 'custum_3d_diffusion.custum_pipeline.unifield_pipeline_img2mvimg.StableDiffusionImage2MVCustomPipeline'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
RGB image not RGBA! still remove bg!
Warn!: E:\Desktop\ComfyUI_3d\python_embeded\Lib\site-packages\transformers\models\clip\modeling_clip.py:480: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:455.)
  attn_output = torch.nn.functional.scaled_dot_product_attention(

  0%|                                                                                           | 0/30 [00:00<?, ?it/s]Warning! condition_latents is not None, but self_attn_ref is not enabled! This warning will only be raised once.
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:03<00:00,  7.85it/s]
Warn!: E:\Desktop\ComfyUI_3d\python_embeded\Lib\site-packages\huggingface_hub\file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(

Warn!: E:\Desktop\ComfyUI_3d\python_embeded\Lib\site-packages\huggingface_hub\file_download.py:159: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\Utente\.cache\huggingface\hub\models--runwayml--stable-diffusion-v1-5. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the `HF_HUB_DISABLE_SYMLINKS_WARNING` environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations.
To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to see activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development
  warnings.warn(message)

Fetching 14 files: 100%|███████████████████████████████████████████████████████████████| 14/14 [11:21<00:00, 48.67s/it]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 6/6 [00:14<00:00,  2.38s/it]
Pipelines loaded with `dtype=torch.float16` cannot run with `cpu` device. It is not recommended to move them to `cpu` as running them will fail. Please make sure to use an accelerator to run the pipeline in inference, due to the lack of support for`float16` operations on this device in PyTorch. Please, remove the `torch_dtype=torch.float16` argument, or use another device for inference.
Pipelines loaded with `dtype=torch.float16` cannot run with `cpu` device. It is not recommended to move them to `cpu` as running them will fail. Please make sure to use an accelerator to run the pipeline in inference, due to the lack of support for`float16` operations on this device in PyTorch. Please, remove the `torch_dtype=torch.float16` argument, or use another device for inference.
Pipelines loaded with `dtype=torch.float16` cannot run with `cpu` device. It is not recommended to move them to `cpu` as running them will fail. Please make sure to use an accelerator to run the pipeline in inference, due to the lack of support for`float16` operations on this device in PyTorch. Please, remove the `torch_dtype=torch.float16` argument, or use another device for inference.
Loading pipeline components...: 100%|██████████████████████████████████████████████████| 6/6 [00:00<00:00, 5990.44it/s]
Warn!: E:\Desktop\ComfyUI_3d\python_embeded\Lib\site-packages\huggingface_hub\file_download.py:159: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\Utente\.cache\huggingface\hub\models--h94--IP-Adapter. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the `HF_HUB_DISABLE_SYMLINKS_WARNING` environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations.
To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to see activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development
  warnings.warn(message)

Warning! extra parameter in cli is not verified, may cause erros.
Warn!: E:\Desktop\ComfyUI_3d\python_embeded\Lib\site-packages\huggingface_hub\file_download.py:159: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\Utente\.cache\huggingface\hub\models--lambdalabs--sd-image-variations-diffusers. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the `HF_HUB_DISABLE_SYMLINKS_WARNING` environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations.
To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to see activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development
  warnings.warn(message)

Fetching 6 files: 100%|███████████████████████████████████████████| 6/6 [00:01<00:00,  5.32it/s]
Loading pipeline components...: 100%|█████████████████████████████| 5/5 [00:01<00:00,  2.85it/s]
You have disabled the safety checker for <class 'custum_3d_diffusion.custum_pipeline.unifield_pipeline_img2img.StableDiffusionImageCustomPipeline'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
100%|███████████████████████████████████████████████████████████| 10/10 [08:29<00:00, 50.91s/it]
2024-08-21 20:12:42.6720227 [E:onnxruntime:Default, provider_bridge_ort.cc:1978 onnxruntime::TryGetProviderInfo_TensorRT] D:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1637 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "E:\Desktop\ComfyUI_3d\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_providers_tensorrt.dll"

*************** EP Error ***************
EP Error D:\a\_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:490 onnxruntime::python::RegisterTensorRTPluginsAsCustomOps Please install TensorRT libraries as mentioned in the GPU requirements page, make sure they're in the PATH or LD_LIBRARY_PATH, and that your GPU is supported.
 when using [('TensorrtExecutionProvider', {'device_id': 0, 'trt_max_workspace_size': 8589934592, 'trt_fp16_enable': True, 'trt_engine_cache_enable': True}), ('CUDAExecutionProvider', {'device_id': 0, 'arena_extend_strategy': 'kSameAsRequested', 'gpu_mem_limit': 8589934592, 'cudnn_conv_algo_search': 'HEURISTIC'})]
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.
****************************************
2024-08-21 20:12:49.2380864 [E:onnxruntime:Default, provider_bridge_ort.cc:1992 onnxruntime::TryGetProviderInfo_CUDA] D:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1637 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "E:\Desktop\ComfyUI_3d\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

100%|████████████████████████████████████████████████████████| 30/30 [1:15:48<00:00, 151.60s/it]
!!! Exception during processing!!! CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Exception raised from c10_cuda_check_implementation at ..\c10\cuda\CUDAException.cpp:43 (most recent call first):
00007FFB8618366200007FFB86183600 c10.dll!c10::Error::Error [<unknown file> @ <unknown line number>]
00007FFB8618318A00007FFB86183130 c10.dll!c10::detail::torchCheckFail [<unknown file> @ <unknown line number>]
00007FFBBC95F93F00007FFBBC95F640 c10_cuda.dll!c10::cuda::c10_cuda_check_implementation [<unknown file> @ <unknown line number>]
00007FFA67460391 <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFA6740BE07 <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFA6741A36E <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFA6741A3F4 <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFA67407F9F <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFB858F82BE00007FFB858F8060 python311.dll!PyObject_MakeTpCall [<unknown file> @ <unknown line number>]
00007FFB858FBDDF00007FFB858FBB50 python311.dll!PyObject_Vectorcall [<unknown file> @ <unknown line number>]
00007FFB858FD42300007FFB858FCC20 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFB858FAE6400007FFB858FACC0 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFB859A507300007FFB859A5018 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFB609D026100007FFB609BCEA0 torch_python.dll!THPPointer<THPStorage>::THPPointer<THPStorage> [<unknown file> @ <unknown line number>]
00007FFB859275F500007FFB85927540 python311.dll!PyArg_Parse_SizeT [<unknown file> @ <unknown line number>]
00007FFB859A50C400007FFB859A5018 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFB859A4D4000007FFB859A48EC python311.dll!PyObject_CallObject [<unknown file> @ <unknown line number>]
00007FFB85901F7F00007FFB858FCC20 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFB858FAE6400007FFB858FACC0 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFB85945C2500007FFB85943DC4 python311.dll!PyIter_Send [<unknown file> @ <unknown line number>]
00007FFB859A4C6A00007FFB859A48EC python311.dll!PyObject_CallObject [<unknown file> @ <unknown line number>]
00007FFB85901F7F00007FFB858FCC20 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFB858FAE6400007FFB858FACC0 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFB859A4CF300007FFB859A48EC python311.dll!PyObject_CallObject [<unknown file> @ <unknown line number>]
00007FFB85901F7F00007FFB858FCC20 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFB858FAE6400007FFB858FACC0 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFB85945C7F00007FFB85943DC4 python311.dll!PyIter_Send [<unknown file> @ <unknown line number>]
00007FFB859A507300007FFB859A5018 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFB85962AA000007FFB859626EC python311.dll!PyInterpreterState_Delete [<unknown file> @ <unknown line number>]
00007FFB85962A0600007FFB859626EC python311.dll!PyInterpreterState_Delete [<unknown file> @ <unknown line number>]
00007FFBE5F71BB200007FFBE5F71B20 ucrtbase.dll!configthreadlocale [<unknown file> @ <unknown line number>]
00007FFBE72B737400007FFBE72B7360 KERNEL32.DLL!BaseThreadInitThunk [<unknown file> @ <unknown line number>]
00007FFBE87FCC9100007FFBE87FCC70 ntdll.dll!RtlUserThreadStart [<unknown file> @ <unknown line number>]

Traceback (most recent call last):
  File "E:\Desktop\ComfyUI_3d\ComfyUI\execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Desktop\ComfyUI_3d\ComfyUI\execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Desktop\ComfyUI_3d\ComfyUI\execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Desktop\ComfyUI_3d\ComfyUI\custom_nodes\ComfyUI-Unique3D\run.py", line 145, in run
    new_meshes, img_list, rm_normals = geo_reconstruct(
                                       ^^^^^^^^^^^^^^^^
  File "E:\Desktop\ComfyUI_3d\ComfyUI\custom_nodes\ComfyUI-Unique3D\scripts\multiview_inference.py", line 90, in geo_reconstruct
    _ = multiview_color_projection(meshes, rgb_pils, resolution=512, device="cuda", complete_unseen=False, confidence_threshold=0.1)    # just check for validation, may throw error
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Desktop\ComfyUI_3d\ComfyUI/custom_nodes/ComfyUI-Unique3D\scripts\project_mesh.py", line 283, in multiview_color_projection
    ret = project_color(meshes, camera, image, eps=eps, resolution=resolution, device=device, use_alpha=use_alpha)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Desktop\ComfyUI_3d\ComfyUI/custom_nodes/ComfyUI-Unique3D\scripts\project_mesh.py", line 134, in project_color
    faces_normals = meshes.faces_normals_packed()[unique_faces]
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Desktop\ComfyUI_3d\python_embeded\Lib\site-packages\pytorch3d\structures\meshes.py", line 800, in faces_normals_packed
    self._compute_face_areas_normals()
  File "E:\Desktop\ComfyUI_3d\python_embeded\Lib\site-packages\pytorch3d\structures\meshes.py", line 868, in _compute_face_areas_normals
    face_areas, face_normals = mesh_face_areas_normals(verts_packed, faces_packed)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Desktop\ComfyUI_3d\python_embeded\Lib\site-packages\torch\autograd\function.py", line 598, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Desktop\ComfyUI_3d\python_embeded\Lib\site-packages\pytorch3d\ops\mesh_face_areas_normals.py", line 48, in forward
    areas, normals = _C.face_areas_normals_forward(verts, faces)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Exception raised from c10_cuda_check_implementation at ..\c10\cuda\CUDAException.cpp:43 (most recent call first):
00007FFB8618366200007FFB86183600 c10.dll!c10::Error::Error [<unknown file> @ <unknown line number>]
00007FFB8618318A00007FFB86183130 c10.dll!c10::detail::torchCheckFail [<unknown file> @ <unknown line number>]
00007FFBBC95F93F00007FFBBC95F640 c10_cuda.dll!c10::cuda::c10_cuda_check_implementation [<unknown file> @ <unknown line number>]
00007FFA67460391 <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFA6740BE07 <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFA6741A36E <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFA6741A3F4 <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFA67407F9F <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFB858F82BE00007FFB858F8060 python311.dll!PyObject_MakeTpCall [<unknown file> @ <unknown line number>]
00007FFB858FBDDF00007FFB858FBB50 python311.dll!PyObject_Vectorcall [<unknown file> @ <unknown line number>]
00007FFB858FD42300007FFB858FCC20 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFB858FAE6400007FFB858FACC0 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFB859A507300007FFB859A5018 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFB609D026100007FFB609BCEA0 torch_python.dll!THPPointer<THPStorage>::THPPointer<THPStorage> [<unknown file> @ <unknown line number>]
00007FFB859275F500007FFB85927540 python311.dll!PyArg_Parse_SizeT [<unknown file> @ <unknown line number>]
00007FFB859A50C400007FFB859A5018 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFB859A4D4000007FFB859A48EC python311.dll!PyObject_CallObject [<unknown file> @ <unknown line number>]
00007FFB85901F7F00007FFB858FCC20 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFB858FAE6400007FFB858FACC0 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFB85945C2500007FFB85943DC4 python311.dll!PyIter_Send [<unknown file> @ <unknown line number>]
00007FFB859A4C6A00007FFB859A48EC python311.dll!PyObject_CallObject [<unknown file> @ <unknown line number>]
00007FFB85901F7F00007FFB858FCC20 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFB858FAE6400007FFB858FACC0 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFB859A4CF300007FFB859A48EC python311.dll!PyObject_CallObject [<unknown file> @ <unknown line number>]
00007FFB85901F7F00007FFB858FCC20 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFB858FAE6400007FFB858FACC0 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFB85945C7F00007FFB85943DC4 python311.dll!PyIter_Send [<unknown file> @ <unknown line number>]
00007FFB859A507300007FFB859A5018 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFB85962AA000007FFB859626EC python311.dll!PyInterpreterState_Delete [<unknown file> @ <unknown line number>]
00007FFB85962A0600007FFB859626EC python311.dll!PyInterpreterState_Delete [<unknown file> @ <unknown line number>]
00007FFBE5F71BB200007FFBE5F71B20 ucrtbase.dll!configthreadlocale [<unknown file> @ <unknown line number>]
00007FFBE72B737400007FFBE72B7360 KERNEL32.DLL!BaseThreadInitThunk [<unknown file> @ <unknown line number>]
00007FFBE87FCC9100007FFBE87FCC70 ntdll.dll!RtlUserThreadStart [<unknown file> @ <unknown line number>]

Prompt executed in 9197.32 seconds
jtydhr88 commented 2 months ago

Hi looks like it caused an issue with Cuda and pytorch3d installation, and to be honest j never meet this issue on my local