deepfakes / faceswap

Deepfakes Software For All
https://www.faceswap.dev
GNU General Public License v3.0
52.05k stars 13.18k forks source link

Profiling failure on CUDNN engine 1#TC: RESOURCE_EXHAUSTED: Out of memory while trying to allocate 95420720 bytes #1354

Closed NavdeepD2 closed 11 months ago

NavdeepD2 commented 11 months ago

When doing Train, facing following error:

No algorithm worked! Error messages: Profiling failure on CUDNN engine 1#TC: RESOURCE_EXHAUSTED: Out of memory while trying to allocate 95420720 bytes. Profiling failure on CUDNN engine 1: RESOURCE_EXHAUSTED: Out of memory while trying to allocate 95420432 bytes. Profiling failure on CUDNN engine 0#TC: RESOURCE_EXHAUSTED: Out of memory while trying to allocate 16777216 bytes. Profiling failure on CUDNN engine 0: RESOURCE_EXHAUSTED: Out of memory while trying to allocate 16777216 bytes. [[{{node gradient_tape/original/encoder/upscale_512_0_conv2d_conv2d/Conv2D_1/Conv2DBackpropInput}}]] [Op:__inference_train_function_7782] 10/15/2023 03:11:20 CRITICAL An unexpected crash has occurred. Crash report written to 'C:\Users\navde\faceswap\crash_report.2023.10.15.031116815608.log'. You MUST provide this file if seeking assistance. Please verify you are running the latest version of faceswap before reporting Process exited.

My system: OS Name Microsoft Windows 11 Home Single Language Version 10.0.22621 Build 22621 Processor 12th Gen Intel(R) Core(TM) i5-12500H Physical Memory (RAM) 16.0 GB GPU NVIDIA GeForce RTX 3050 Laptop GPU

torzdf commented 11 months ago

This is an out of memory error, as clearly stated in the error report. Get a bigger GPU or use a smaller model, or follow one of these remedial steps:

https://forum.faceswap.dev/app.php/faqpage?sid=c6e8313d10e2337cd3f53f966ec88875#f3r9

Either way, this is not a bug.