RuntimeError: CUDA error: out of memory
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
Interrogate Request!
Network: [interrogator_network: "BLIP"
]
File size: 20132
Unloading interrogator blip2-opt-6.7b
Exception processing item!
Exception string: 'CUDA error: out of memory
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
'
Traceback (most recent call last):
File "E:\Downloads\BDTM_V2.0.3_NetCore60_AllInclude\interrogator_rpc\main.py", line 142, in InterrogateImage
tag_ret = interrogate_image(network_conf.interrogator_network, image_obj, skip_online=request.skip_internet_requests)
File "E:\Downloads\BDTM_V2.0.3_NetCore60_AllInclude\interrogator_rpc\main.py", line 57, in interrogate_image
interg.stop()
File "E:\Downloads\BDTM_V2.0.3_NetCore60_AllInclude\interrogator_rpc\ext_kohya\captioning.py", line 55, in stop
self.interrogator.unload()
File "E:\Downloads\BDTM_V2.0.3_NetCore60_AllInclude\interrogator_rpc\ext_kohya\interrogators\blip2_captioning.py", line 30, in unload
devices.torch_gc()
File "E:\Downloads\BDTM_V2.0.3_NetCore60_AllInclude\interrogator_rpc\ext_kohya\devices.py", line 67, in torch_gc
torch.cuda.empty_cache()
File "C:\Users\George\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\cuda\memory.py", line 159, in empty_cache
torch._C._cuda_emptyCache()
RuntimeError: CUDA error: out of memory
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
RuntimeError: CUDA error: out of memory CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with
TORCH_USE_CUDA_DSAto enable device-side assertions.Interrogate Request! Network: [interrogator_network: "BLIP" ] File size: 20132 Unloading interrogator blip2-opt-6.7b Exception processing item! Exception string: 'CUDA error: out of memory CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with
TORCH_USE_CUDA_DSAto enable device-side assertions. ' Traceback (most recent call last): File "E:\Downloads\BDTM_V2.0.3_NetCore60_AllInclude\interrogator_rpc\main.py", line 142, in InterrogateImage tag_ret = interrogate_image(network_conf.interrogator_network, image_obj, skip_online=request.skip_internet_requests) File "E:\Downloads\BDTM_V2.0.3_NetCore60_AllInclude\interrogator_rpc\main.py", line 57, in interrogate_image interg.stop() File "E:\Downloads\BDTM_V2.0.3_NetCore60_AllInclude\interrogator_rpc\ext_kohya\captioning.py", line 55, in stop self.interrogator.unload() File "E:\Downloads\BDTM_V2.0.3_NetCore60_AllInclude\interrogator_rpc\ext_kohya\interrogators\blip2_captioning.py", line 30, in unload devices.torch_gc() File "E:\Downloads\BDTM_V2.0.3_NetCore60_AllInclude\interrogator_rpc\ext_kohya\devices.py", line 67, in torch_gc torch.cuda.empty_cache() File "C:\Users\George\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\cuda\memory.py", line 159, in empty_cache torch._C._cuda_emptyCache() RuntimeError: CUDA error: out of memory CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile withTORCH_USE_CUDA_DSAto enable device-side assertions.