MrForExample / ComfyUI-3D-Pack

An extensive node suite that enables ComfyUI to process 3D inputs (Mesh & UV Texture, etc) using cutting edge algorithms (3DGS, NeRF, etc.)
MIT License
2.21k stars 222 forks source link

Server stop while running this wf #128

Open MaraScott opened 6 months ago

MaraScott commented 6 months ago

https://github.com/MrForExample/ComfyUI-3D-Pack/blob/main/_Example_Workflows/TripoSR_to_Mesh.json

when running this workflow, I have the comfyui server which stop on that node

image

the error :

[Comfy3D] [TripoSR] Running TripoSR...
CUDA kernel failed : no kernel image is available for execution on the device
void prescan_small(int *, int *, int, int, CUstream_st *) at L:126 in C:\Users\reall\Softwares\Miniconda3\envs\Wonder3D\_Projects\torchmcubes\cxx\pscan.cu

I have no cuda on my GC

neocao123 commented 5 months ago

Same issue

SergioKingOne commented 4 months ago

Same here, I do have CUDA...

prairiefawkes commented 1 month ago

Same issue but it throws when I'm running Triplane Gaussian Transformers node. This is probably my 15th time trying to install and run 3D-Pack. I'm installing into a Miniconda environment. I believe I have all the correct versions.

Here's startup:

C:\Users\pfawkes\ai_programs\comfyui3d>call conda activate comfyui3d
[START] Security scan
[DONE] Security scan
## ComfyUI-Manager: installing dependencies done.
** ComfyUI startup time: 2024-08-07 00:52:38.992399
** Platform: Windows
** Python version: 3.11.9 | packaged by Anaconda, Inc. | (main, Apr 19 2024, 16:40:41) [MSC v.1916 64 bit (AMD64)]
** Python executable: C:\Users\pfawkes\miniconda3\envs\comfyui3d\python.exe
** ComfyUI Path: C:\Users\pfawkes\ai_programs\comfyui3d
** Log path: C:\Users\pfawkes\ai_programs\comfyui3d\comfyui.log

Prestartup times for custom nodes:
   1.6 seconds: C:\Users\pfawkes\ai_programs\comfyui3d\custom_nodes\ComfyUI-Manager

Total VRAM 24576 MB, total RAM 65461 MB
pytorch version: 2.3.0+cu121
xformers version: 0.0.26.post1
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3090 : cudaMallocAsync
Using xformers cross attention
****** User settings have been changed to be stored on the server instead of browser storage. ******
****** For multi-user setups add the --multi-user CLI argument to enable multiple user profiles. ******
[Prompt Server] web root: C:\Users\pfawkes\ai_programs\comfyui3d\web
Adding extra search path configs C:\Users\pfawkes\ai_programs\auto1111\webui\models/Stable-diffusion
Adding extra search path vae C:\Users\pfawkes\ai_programs\auto1111\webui\models/VAE
Adding extra search path upscale_models C:\Users\pfawkes\ai_programs\auto1111\webui\models/ESRGAN
Adding extra search path upscale_models C:\Users\pfawkes\ai_programs\auto1111\webui\models/RealESRGAN
Adding extra search path upscale_models C:\Users\pfawkes\ai_programs\auto1111\webui\models/SwinIR
Adding extra search path embeddings C:\Users\pfawkes\ai_programs\auto1111\webui\embeddings
Adding extra search path hypernetworks C:\Users\pfawkes\ai_programs\auto1111\webui\models/hypernetworks
Adding extra search path controlnet C:\Users\pfawkes\ai_programs\auto1111\webui\models/ControlNet
Adding extra search path checkpoints C:\Users\pfawkes\ai_programs\ai_resources\checkpoints
Adding extra search path loras C:\Users\pfawkes\ai_programs\ai_resources\loras
Warn!: xFormers is available (Attention)
Warn!: C:\Users\pfawkes\miniconda3\envs\comfyui3d\Lib\site-packages\torch\utils\cpp_extension.py:1967: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation.
If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
  warnings.warn(

### Loading: ComfyUI-Manager (V2.48.5)
### ComfyUI Revision: 2481 [2a02546e] | Released on '2024-08-06'

Import times for custom nodes:
   0.0 seconds: C:\Users\pfawkes\ai_programs\comfyui3d\custom_nodes\websocket_image_save.py
   0.7 seconds: C:\Users\pfawkes\ai_programs\comfyui3d\custom_nodes\ComfyUI-Manager
  79.4 seconds: C:\Users\pfawkes\ai_programs\comfyui3d\custom_nodes\ComfyUI-3D-Pack

Starting server

My investigations and ClaudeAI suggest that the error stems from "This error typically occurs when there's a mismatch between the CUDA version your PyTorch was compiled with and the CUDA version installed on your system, or when the CUDA drivers are not properly installed or configured."

Here's Error:


[Comfy3D] [WARNING] [Load_Triplane_Gaussian_Transformers] can't find checkpoint C:\Users\pfawkes\ai_programs\comfyui3d\custom_nodes\ComfyUI-3D-Pack\Checkpoints\TriplaneGaussian\model_lvis_rel.ckpt, will download it from repo VAST-AI/TriplaneGaussian instead
TriplaneGaussian.models.tokenizers.image
TriplaneGaussian.models.networks
TriplaneGaussian.models.tokenizers.triplane
TriplaneGaussian.models.transformers
TriplaneGaussian.models.networks
TriplaneGaussian.models.renderer
TriplaneGaussian.models.pointclouds.simplepoint
TriplaneGaussian.models.tokenizers.image
TriplaneGaussian.models.networks
TriplaneGaussian.models.tokenizers.point
TriplaneGaussian.models.transformers
TriplaneGaussian.models.networks
TriplaneGaussian.models.snowflake.model_spdpp
TriplaneGaussian.models.pointclouds.pointnet
[Comfy3D] [Load_Triplane_Gaussian_Transformers] loaded model ckpt from C:\Users\pfawkes\ai_programs\comfyui3d\custom_nodes\ComfyUI-3D-Pack\Checkpoints\TriplaneGaussian\model_lvis_rel.ckpt
Warn!: C:\Users\pfawkes\ai_programs\comfyui3d\custom_nodes\ComfyUI-3D-Pack\Gen_3D_Modules\TriplaneGaussian\data.py:91: UserWarning: Using torch.cross without specifying the dim arg is deprecated.
Please either pass the dim explicitly or simply use torch.linalg.cross.
The default value of dim will change to agree with that of linalg.cross in a future release. (Triggered internally at ..\aten\src\ATen\native\Cross.cpp:66.)
  right: Float[Tensor, "B 3"] = F.normalize(torch.cross(lookat, up), dim=-1)

Warn!: C:\Users\pfawkes\miniconda3\envs\comfyui3d\Lib\site-packages\diffusers\models\attention_processor.py:1584: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:455.)
  hidden_states = F.scaled_dot_product_attention(

!!! Exception during processing!!! CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Exception raised from c10_cuda_check_implementation at ..\c10\cuda\CUDAException.cpp:43 (most recent call first):
00007FFDD4BC366200007FFDD4BC3600 c10.dll!c10::Error::Error [<unknown file> @ <unknown line number>]
00007FFDD4BC318A00007FFDD4BC3130 c10.dll!c10::detail::torchCheckFail [<unknown file> @ <unknown line number>]
00007FFE1CDDF93F00007FFE1CDDF640 c10_cuda.dll!c10::cuda::c10_cuda_check_implementation [<unknown file> @ <unknown line number>]
00007FFC54397D08 <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFC5417EF8C <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFC5417F418 <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFC5418D169 <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFC5418D1F4 <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFC54177F9F <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFD9B8A42CC00007FFD9B8A3550 python311.dll!PyCFunction_GetFlags [<unknown file> @ <unknown line number>]
00007FFD9B85F67800007FFD9B85F5D0 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFD9B95D88400007FFD9B95D320 python311.dll!PyEval_GetFuncDesc [<unknown file> @ <unknown line number>]
00007FFD9B95903F00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B95BD0E00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B85F76D00007FFD9B85F730 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFD9B85F50E00007FFD9B85F420 python311.dll!PyVectorcall_Function [<unknown file> @ <unknown line number>]
00007FFD9B85F61F00007FFD9B85F5D0 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFD64E0026100007FFD64DECEA0 torch_python.dll!THPPointer<THPStorage>::THPPointer<THPStorage> [<unknown file> @ <unknown line number>]
00007FFD9B8A430600007FFD9B8A3550 python311.dll!PyCFunction_GetFlags [<unknown file> @ <unknown line number>]
00007FFD9B85F67800007FFD9B85F5D0 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFD9B95D88400007FFD9B95D320 python311.dll!PyEval_GetFuncDesc [<unknown file> @ <unknown line number>]
00007FFD9B95903F00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B95BD0E00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B85F76D00007FFD9B85F730 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFD9B86154900007FFD9B861190 python311.dll!PyCell_Set [<unknown file> @ <unknown line number>]
00007FFD9B861B5100007FFD9B861960 python311.dll!PyMethod_Self [<unknown file> @ <unknown line number>]
00007FFD9B85F50E00007FFD9B85F420 python311.dll!PyVectorcall_Function [<unknown file> @ <unknown line number>]
00007FFD9B85F61F00007FFD9B85F5D0 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFD9B95D7BD00007FFD9B95D320 python311.dll!PyEval_GetFuncDesc [<unknown file> @ <unknown line number>]
00007FFD9B95903F00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B95BD0E00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B85F76D00007FFD9B85F730 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFD9B86154900007FFD9B861190 python311.dll!PyCell_Set [<unknown file> @ <unknown line number>]
00007FFD9B861B5100007FFD9B861960 python311.dll!PyMethod_Self [<unknown file> @ <unknown line number>]
00007FFD9B85F50E00007FFD9B85F420 python311.dll!PyVectorcall_Function [<unknown file> @ <unknown line number>]
00007FFD9B85F61F00007FFD9B85F5D0 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFD9B95D7BD00007FFD9B95D320 python311.dll!PyEval_GetFuncDesc [<unknown file> @ <unknown line number>]
00007FFD9B95903F00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B95BD0E00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B85F76D00007FFD9B85F730 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFD9B85F25400007FFD9B85F180 python311.dll!PyObject_FastCallDictTstate [<unknown file> @ <unknown line number>]
00007FFD9B85F9F200007FFD9B85F950 python311.dll!PyObject_Call_Prepend [<unknown file> @ <unknown line number>]
00007FFD9B8CAAF400007FFD9B8C72D0 python311.dll!PyType_Ready [<unknown file> @ <unknown line number>]
00007FFD9B85F3D100007FFD9B85F2B0 python311.dll!PyObject_MakeTpCall [<unknown file> @ <unknown line number>]
00007FFD9B85F59100007FFD9B85F570 python311.dll!PyObject_Vectorcall [<unknown file> @ <unknown line number>]
00007FFD9B957F1A00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B95BD0E00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B85F76D00007FFD9B85F730 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFD9B86154900007FFD9B861190 python311.dll!PyCell_Set [<unknown file> @ <unknown line number>]
00007FFD9B861B5100007FFD9B861960 python311.dll!PyMethod_Self [<unknown file> @ <unknown line number>]
00007FFD9B85F50E00007FFD9B85F420 python311.dll!PyVectorcall_Function [<unknown file> @ <unknown line number>]
00007FFD9B85F61F00007FFD9B85F5D0 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFD9B95D7BD00007FFD9B95D320 python311.dll!PyEval_GetFuncDesc [<unknown file> @ <unknown line number>]
00007FFD9B95903F00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B95BD0E00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B85F76D00007FFD9B85F730 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFD9B86154900007FFD9B861190 python311.dll!PyCell_Set [<unknown file> @ <unknown line number>]
00007FFD9B861B5100007FFD9B861960 python311.dll!PyMethod_Self [<unknown file> @ <unknown line number>]
00007FFD9B85F50E00007FFD9B85F420 python311.dll!PyVectorcall_Function [<unknown file> @ <unknown line number>]
00007FFD9B85F61F00007FFD9B85F5D0 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFD9B95D7BD00007FFD9B95D320 python311.dll!PyEval_GetFuncDesc [<unknown file> @ <unknown line number>]
00007FFD9B95903F00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B95BD0E00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B85F76D00007FFD9B85F730 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]

Traceback (most recent call last):
  File "C:\Users\pfawkes\ai_programs\comfyui3d\execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\ai_programs\comfyui3d\execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\ai_programs\comfyui3d\execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\ai_programs\comfyui3d\custom_nodes\ComfyUI-3D-Pack\nodes.py", line 1416, in run_TGS
    gs_ply.extend(tgs_model(batch))
                  ^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\miniconda3\envs\comfyui3d\Lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\miniconda3\envs\comfyui3d\Lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\ai_programs\comfyui3d\custom_nodes\ComfyUI-3D-Pack\Gen_3D_Modules\TriplaneGaussian\triplane_gaussian_transformers.py", line 145, in forward
    out = self._forward(batch)
          ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\ai_programs\comfyui3d\custom_nodes\ComfyUI-3D-Pack\Gen_3D_Modules\TriplaneGaussian\triplane_gaussian_transformers.py", line 95, in _forward
    out = self.pointcloud_generator(batch)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\miniconda3\envs\comfyui3d\Lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\miniconda3\envs\comfyui3d\Lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\ai_programs\comfyui3d\custom_nodes\ComfyUI-3D-Pack\Gen_3D_Modules\TriplaneGaussian\models\pointclouds\simplepoint.py", line 103, in forward
    up_results = self.pointcloud_upsampling(upsampling_input)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\miniconda3\envs\comfyui3d\Lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\miniconda3\envs\comfyui3d\Lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\ai_programs\comfyui3d\custom_nodes\ComfyUI-3D-Pack\Gen_3D_Modules\TriplaneGaussian\models\snowflake\model_spdpp.py", line 187, in forward
    results = self.decoder(x)
              ^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\miniconda3\envs\comfyui3d\Lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\miniconda3\envs\comfyui3d\Lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\ai_programs\comfyui3d\custom_nodes\ComfyUI-3D-Pack\Gen_3D_Modules\TriplaneGaussian\models\snowflake\model_spdpp.py", line 152, in forward
    up_cond = self.calculate_image_token(points, feat_cond, x)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\ai_programs\comfyui3d\custom_nodes\ComfyUI-3D-Pack\Gen_3D_Modules\TriplaneGaussian\models\snowflake\model_spdpp.py", line 119, in calculate_image_token
    local_features_proj = points_projection(
                          ^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\ai_programs\comfyui3d\custom_nodes\ComfyUI-3D-Pack\Gen_3D_Modules\TriplaneGaussian\models\snowflake\model_spdpp.py", line 64, in points_projection
    fragments = rasterize(Pointclouds(points))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\miniconda3\envs\comfyui3d\Lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\miniconda3\envs\comfyui3d\Lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\miniconda3\envs\comfyui3d\Lib\site-packages\pytorch3d\renderer\points\rasterizer.py", line 161, in forward
    idx, zbuf, dists2 = rasterize_points(
                        ^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\miniconda3\envs\comfyui3d\Lib\site-packages\pytorch3d\renderer\points\rasterize_points.py", line 134, in rasterize_points
    return _RasterizePoints.apply(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\miniconda3\envs\comfyui3d\Lib\site-packages\torch\autograd\function.py", line 598, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pfawkes\miniconda3\envs\comfyui3d\Lib\site-packages\pytorch3d\renderer\points\rasterize_points.py", line 214, in forward
    idx, zbuf, dists = _C.rasterize_points(*args)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Exception raised from c10_cuda_check_implementation at ..\c10\cuda\CUDAException.cpp:43 (most recent call first):
00007FFDD4BC366200007FFDD4BC3600 c10.dll!c10::Error::Error [<unknown file> @ <unknown line number>]
00007FFDD4BC318A00007FFDD4BC3130 c10.dll!c10::detail::torchCheckFail [<unknown file> @ <unknown line number>]
00007FFE1CDDF93F00007FFE1CDDF640 c10_cuda.dll!c10::cuda::c10_cuda_check_implementation [<unknown file> @ <unknown line number>]
00007FFC54397D08 <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFC5417EF8C <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFC5417F418 <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFC5418D169 <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFC5418D1F4 <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFC54177F9F <unknown symbol address> _C.cp311-win_amd64.pyd!<unknown symbol> [<unknown file> @ <unknown line number>]
00007FFD9B8A42CC00007FFD9B8A3550 python311.dll!PyCFunction_GetFlags [<unknown file> @ <unknown line number>]
00007FFD9B85F67800007FFD9B85F5D0 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFD9B95D88400007FFD9B95D320 python311.dll!PyEval_GetFuncDesc [<unknown file> @ <unknown line number>]
00007FFD9B95903F00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B95BD0E00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B85F76D00007FFD9B85F730 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFD9B85F50E00007FFD9B85F420 python311.dll!PyVectorcall_Function [<unknown file> @ <unknown line number>]
00007FFD9B85F61F00007FFD9B85F5D0 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFD64E0026100007FFD64DECEA0 torch_python.dll!THPPointer<THPStorage>::THPPointer<THPStorage> [<unknown file> @ <unknown line number>]
00007FFD9B8A430600007FFD9B8A3550 python311.dll!PyCFunction_GetFlags [<unknown file> @ <unknown line number>]
00007FFD9B85F67800007FFD9B85F5D0 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFD9B95D88400007FFD9B95D320 python311.dll!PyEval_GetFuncDesc [<unknown file> @ <unknown line number>]
00007FFD9B95903F00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B95BD0E00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B85F76D00007FFD9B85F730 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFD9B86154900007FFD9B861190 python311.dll!PyCell_Set [<unknown file> @ <unknown line number>]
00007FFD9B861B5100007FFD9B861960 python311.dll!PyMethod_Self [<unknown file> @ <unknown line number>]
00007FFD9B85F50E00007FFD9B85F420 python311.dll!PyVectorcall_Function [<unknown file> @ <unknown line number>]
00007FFD9B85F61F00007FFD9B85F5D0 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFD9B95D7BD00007FFD9B95D320 python311.dll!PyEval_GetFuncDesc [<unknown file> @ <unknown line number>]
00007FFD9B95903F00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B95BD0E00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B85F76D00007FFD9B85F730 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFD9B86154900007FFD9B861190 python311.dll!PyCell_Set [<unknown file> @ <unknown line number>]
00007FFD9B861B5100007FFD9B861960 python311.dll!PyMethod_Self [<unknown file> @ <unknown line number>]
00007FFD9B85F50E00007FFD9B85F420 python311.dll!PyVectorcall_Function [<unknown file> @ <unknown line number>]
00007FFD9B85F61F00007FFD9B85F5D0 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFD9B95D7BD00007FFD9B95D320 python311.dll!PyEval_GetFuncDesc [<unknown file> @ <unknown line number>]
00007FFD9B95903F00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B95BD0E00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B85F76D00007FFD9B85F730 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFD9B85F25400007FFD9B85F180 python311.dll!PyObject_FastCallDictTstate [<unknown file> @ <unknown line number>]
00007FFD9B85F9F200007FFD9B85F950 python311.dll!PyObject_Call_Prepend [<unknown file> @ <unknown line number>]
00007FFD9B8CAAF400007FFD9B8C72D0 python311.dll!PyType_Ready [<unknown file> @ <unknown line number>]
00007FFD9B85F3D100007FFD9B85F2B0 python311.dll!PyObject_MakeTpCall [<unknown file> @ <unknown line number>]
00007FFD9B85F59100007FFD9B85F570 python311.dll!PyObject_Vectorcall [<unknown file> @ <unknown line number>]
00007FFD9B957F1A00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B95BD0E00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B85F76D00007FFD9B85F730 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFD9B86154900007FFD9B861190 python311.dll!PyCell_Set [<unknown file> @ <unknown line number>]
00007FFD9B861B5100007FFD9B861960 python311.dll!PyMethod_Self [<unknown file> @ <unknown line number>]
00007FFD9B85F50E00007FFD9B85F420 python311.dll!PyVectorcall_Function [<unknown file> @ <unknown line number>]
00007FFD9B85F61F00007FFD9B85F5D0 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFD9B95D7BD00007FFD9B95D320 python311.dll!PyEval_GetFuncDesc [<unknown file> @ <unknown line number>]
00007FFD9B95903F00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B95BD0E00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B85F76D00007FFD9B85F730 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]
00007FFD9B86154900007FFD9B861190 python311.dll!PyCell_Set [<unknown file> @ <unknown line number>]
00007FFD9B861B5100007FFD9B861960 python311.dll!PyMethod_Self [<unknown file> @ <unknown line number>]
00007FFD9B85F50E00007FFD9B85F420 python311.dll!PyVectorcall_Function [<unknown file> @ <unknown line number>]
00007FFD9B85F61F00007FFD9B85F5D0 python311.dll!PyObject_Call [<unknown file> @ <unknown line number>]
00007FFD9B95D7BD00007FFD9B95D320 python311.dll!PyEval_GetFuncDesc [<unknown file> @ <unknown line number>]
00007FFD9B95903F00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B95BD0E00007FFD9B953780 python311.dll!PyEval_EvalFrameDefault [<unknown file> @ <unknown line number>]
00007FFD9B85F76D00007FFD9B85F730 python311.dll!PyFunction_Vectorcall [<unknown file> @ <unknown line number>]