Open Rosetta-Leong opened 4 months ago
I met the same error when using the latest code, could you please check the latest version?
rank6: Traceback (most recent call last):
rank6: File "main.py", line 299, in
rank6: File "main.py", line 146, in main
rank6: File "/usr/local/lib/python3.8/dist-packages/accelerate/accelerator.py", line 2001, in backward
rank6: File "/usr/local/lib/python3.8/dist-packages/torch/_tensor.py", line 525, in backward
rank6: File "/usr/local/lib/python3.8/dist-packages/torch/autograd/init.py", line 267, in backward
rank6: File "/usr/local/lib/python3.8/dist-packages/torch/autograd/graph.py", line 744, in _engine_run_backward
rank6: return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
rank6: RuntimeError: CUDA error: an illegal memory access was encountered
rank6: CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
rank6: For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
rank6: Compile with TORCH_USE_CUDA_DSA
to enable device-side assertions.
When I use the latest version of the rasterizer(commit: 2917773), I encounter the following problems: There seems to be an illegal memory access problem with the rasterizer implementation causing CUDA to report an error. But strangely, I was able to use the older version of the rasterizer(commit: ade8f02) and it worked fine, which confused me.
I would like to ask what major changes were made in the latest two commits? Do I need to make any corresponding changes in usage? I would greatly appreciate it if you could provide some information.
P.S. This is my code for using the rasterizer,
Or could you please share with me the previous workable version?
When I use the latest version of the rasterizer(commit: 2917773), I encounter the following problems: There seems to be an illegal memory access problem with the rasterizer implementation causing CUDA to report an error. But strangely, I was able to use the older version of the rasterizer(commit: ade8f02) and it worked fine, which confused me. I would like to ask what major changes were made in the latest two commits? Do I need to make any corresponding changes in usage? I would greatly appreciate it if you could provide some information. P.S. This is my code for using the rasterizer,
Or could you please share with me the previous workable version?
commit: ade8f02
When I use the latest version of the rasterizer(commit: 2917773), I encounter the following problems: There seems to be an illegal memory access problem with the rasterizer implementation causing CUDA to report an error. But strangely, I was able to use the older version of the rasterizer(commit: ade8f02) and it worked fine, which confused me. I would like to ask what major changes were made in the latest two commits? Do I need to make any corresponding changes in usage? I would greatly appreciate it if you could provide some information. P.S. This is my code for using the rasterizer,
Or could you please share with me the previous workable version?
commit: ade8f02
I have met the same problem. Does this previous version work correctly?
When I use the latest version of the rasterizer(commit: 2917773c24af4decc298e511c3bf65c5397689fd), I encounter the following problems: There seems to be an illegal memory access problem with the rasterizer implementation causing CUDA to report an error. But strangely, I was able to use the older version of the rasterizer(commit: ade8f028d7895e37d337e5313bc0952cc2c331dd) and it worked fine, which confused me.
I would like to ask what major changes were made in the latest two commits? Do I need to make any corresponding changes in usage? I would greatly appreciate it if you could provide some information.
P.S. This is my code for using the rasterizer,