search

langmanbusi / Semantic-Aware-Low-Light-Image-Enhancement

Semantic-Aware LLIE. CVPR 2023
MIT License
189 stars 7 forks source link

我总是遇到以下报错： #20

Closed zheyicode closed 2 months ago

zheyicode commented 5 months ago

conf: LOLv2-L-SKF, heat: 0, name: 35, PSNR: 28.867, SSIM: 0.835, LPIPS: 0.235, LRC PSNR: 12.450

Intel MKL ERROR: Parameter 6 was incorrect on entry to SLASWP.

Intel MKL ERROR: Parameter 6 was incorrect on entry to SLASWP.

Intel MKL ERROR: Parameter 6 was incorrect on entry to SLASWP.

Intel MKL ERROR: Parameter 6 was incorrect on entry to SLASWP.

Intel MKL ERROR: Parameter 6 was incorrect on entry to SLASWP. CUBLAS error: memory mapping error (11) in magma_sgetrf_gpu_expert at /opt/conda/conda-bld/magma-cuda111_1605822518874/work/src/sgetrf_gpu.cpp:295 CUDA runtime error: an illegal memory access was encountered (700) in magma_sgetrf_gpu_expert at /opt/conda/conda-bld/magma-cuda111_1605822518874/work/src/sgetrf_gpu.cpp:333 CUDA runtime error: an illegal memory access was encountered (700) in magma_queue_destroy_internal at /opt/conda/conda-bld/magma-cuda111_1605822518874/work/interface_cuda/interface.cpp:944 CUDA runtime error: an illegal memory access was encountered (700) in magma_queue_destroy_internal at /opt/conda/conda-bld/magma-cuda111_1605822518874/work/interface_cuda/interface.cpp:945 CUDA runtime error: an illegal memory access was encountered (700) in magma_queue_destroy_internal at /opt/conda/conda-bld/magma-cuda111_1605822518874/work/interface_cuda/interface.cpp:946 CUDA runtime error: an illegal memory access was encountered (700) in magma_sgetrf_gpu_expert at /opt/conda/conda-bld/magma-cuda111_1605822518874/work/src/sgetrf_gpu.cpp:334 CUDA runtime error: an illegal memory access was encountered (700) in magma_queue_destroy_internal at /opt/conda/conda-bld/magma-cuda111_1605822518874/work/interface_cuda/interface.cpp:944 CUDA runtime error: an illegal memory access was encountered (700) in magma_queue_destroy_internal at /opt/conda/conda-bld/magma-cuda111_1605822518874/work/interface_cuda/interface.cpp:945 CUDA runtime error: an illegal memory access was encountered (700) in magma_queue_destroy_internal at /opt/conda/conda-bld/magma-cuda111_1605822518874/work/interface_cuda/interface.cpp:946 CUDA runtime error: an illegal memory access was encountered (700) in magma_sgetrf_gpu_expert at /opt/conda/conda-bld/magma-cuda111_1605822518874/work/src/sgetrf_gpu.cpp:336 CUDA runtime error: an illegal memory access was encountered (700) in magma_sgetrf_gpu_expert at /opt/conda/conda-bld/magma-cuda111_1605822518874/work/src/sgetrf_gpu.cpp:342 CUDA error: an illegal memory access was encountered Traceback (most recent call last): File "/home/mac/NewNet/lowlight/Semantic-Aware-Low-Light-Image-Enhancement-main/LLFlow_SKF/code/models/modules/Permutations.py", line 27, in get_weight dlogdet = torch.slogdet(self.weight)[1] * pixels RuntimeError: CUDA error: an illegal memory access was encountered

zheyicode commented 5 months ago

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "test.py", line 199, in main() File "test.py", line 154, in main sr_t = model.get_sr(lq=lr_t.cuda(), heat=None, seg_map=seg_map, seg_ft=seg_ft) File "/home/mac/NewNet/lowlight/Semantic-Aware-Low-Light-Image-Enhancement-main/LLFlow_SKF/code/models/LLFlow_model.py", line 354, in get_sr return self.get_sr_with_z(lq, heat, seed, z, epses, seg_map, seg_ft)[0] File "/home/mac/NewNet/lowlight/Semantic-Aware-Low-Light-Image-Enhancement-main/LLFlow_SKF/code/models/LLFlow_model.py", line 377, in get_sr_with_z sr, logdet = self.netG(lr=lq, z=z, eps_std=heat, reverse=True, epses=epses, seg_map=seg_map, seg_fea=seg_fea) File "/home/mac/miniconda3/envs/lowlightLLF/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, kwargs) File "/home/mac/miniconda3/envs/lowlightLLF/lib/python3.7/site-packages/torch/cuda/amp/autocast_mode.py", line 138, in decorate_autocast return func(*args, *kwargs) File "/home/mac/NewNet/lowlight/Semantic-Aware-Low-Light-Image-Enhancement-main/LLFlow_SKF/code/models/modules/LLFlow_pc_small_arch.py", line 139, in forward add_gt_noise=add_gt_noise, seg_map=seg_map, seg_fea=seg_fea) File "/home/mac/NewNet/lowlight/Semantic-Aware-Low-Light-Image-Enhancement-main/LLFlow_SKF/code/models/modules/LLFlow_pc_small_arch.py", line 273, in reverse_flow logdet=logdet) File "/home/mac/miniconda3/envs/lowlightLLF/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(input, kwargs) File "/home/mac/NewNet/lowlight/Semantic-Aware-Low-Light-Image-Enhancement-main/LLFlow_SKF/code/models/modules/FlowUpsamplerNet.py", line 204, in forward sr, logdet = self.decode(rrdbResults, z, eps_std, epses=epses_copy, logdet=logdet, y_onehot=y_onehot) File "/home/mac/NewNet/lowlight/Semantic-Aware-Low-Light-Image-Enhancement-main/LLFlow_SKF/code/models/modules/FlowUpsamplerNet.py", line 293, in decode fl_fea, logdet = layer(fl_fea, logdet=logdet, reverse=True, rrdbResults=level_conditionals[level]) File "/home/mac/miniconda3/envs/lowlightLLF/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, *kwargs) File "/home/mac/NewNet/lowlight/Semantic-Aware-Low-Light-Image-Enhancement-main/LLFlow_SKF/code/models/modules/FlowStep.py", line 73, in forward return self.reverse_flow(input, logdet, rrdbResults) File "/home/mac/NewNet/lowlight/Semantic-Aware-Low-Light-Image-Enhancement-main/LLFlow_SKF/code/models/modules/FlowStep.py", line 111, in reverse_flow self, z, logdet, True) File "/home/mac/NewNet/lowlight/Semantic-Aware-Low-Light-Image-Enhancement-main/LLFlow_SKF/code/models/modules/FlowStep.py", line 22, in "invconv": lambda obj, z, logdet, rev: obj.invconv(z, logdet, rev), File "/home/mac/miniconda3/envs/lowlightLLF/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(input, *kwargs) File "/home/mac/NewNet/lowlight/Semantic-Aware-Low-Light-Image-Enhancement-main/LLFlow_SKF/code/models/modules/Permutations.py", line 49, in forward weight, dlogdet = self.get_weight(input, reverse) File "/home/mac/NewNet/lowlight/Semantic-Aware-Low-Light-Image-Enhancement-main/LLFlow_SKF/code/models/modules/Permutations.py", line 32, in get_weight self.weight + (self.weight.mean() torch.randn(self.w_shape).to(input.device) 0.001))[ RuntimeError: CUDA error: an illegal memory access was encountered

zheyicode commented 5 months ago

我的试验环境是3090ti的显卡，并不清楚为啥会报这个错....试过减少batchsize和指定特定GPU了（torch.cuda.set_device(1)）

langmanbusi commented 5 months ago

呃，我也没见过这个报错啊。。我是在A100上跑的，环境也是LLFlow本身的环境。test的时候我记得显存占用也不大，要不你再试试吧