Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0) CUDNN_STATUS_BAD_PARAM

[bigdwg71]

Getting the above failure when using video2x.exe to upscale a movie.

Any help would be appreciated.

error_log_20200103-130854.17500.txt

rtx-2080-ti.json.txt

c:\Users\Chris\Downloads\video2x.v1.0.2>video2x.exe -i d:\MakeMKV\DVD\Speed.mkv -o d:\MakeMKV\DVD\Speed1080.mkv -c C:\Users\Chris\Downloads\video2x.v1.0.2\config\rtx-2080-ti.json

#    # # #    # #      #    # #     #  #  #
#    # # #    # #####  #    #       #   ##
#    # # #    # #      #    #  #####    ##
 #  #  # #    # #      #    # #        #  #
  ##   # #####  ######  ####  ####### #    #

Reading config file in "C:\Users\Chris\Downloads\video2x.v1.0.2\config\rtx-2080-ti.json" Retrieving video metadata Command: ffprobe -of json -show_streams -show_format "d:\MakeMKV\DVD\Speed.mkv" Creating new workspace directory: temp\3b11b8e0-2e50-11ea-945a-2fc02fa70f7f Creating directory under workspace to store extracted frames: temp\3b11b8e0-2e50-11ea-945a-2fc02fa70f7f\extracted Creating directory under workspace to store upscaled frames: temp\3b11b8e0-2e50-11ea-945a-2fc02fa70f7f\upscaled Extracting frames from source video Command: ffmpeg -i d:\MakeMKV\DVD\Speed.mkv -y -threads 6 temp\3b11b8e0-2e50-11ea-945a-2fc02fa70f7f\extracted\frame%d.png Extracted 208377 frames from "d:\MakeMKV\DVD\Speed.mkv" (Size: 67.39 GB, Time: 0.41 hours, approx. 25 minutes) Upscaling the extracted frames from 720x480 to 1920x1440 Splitting up the frames into smaller groups, each to be processed by its own waifu2x thread [Thread1] 208377 (100.0%) frames - cuDNN mode Command: C:\Users\Chris\Downloads\waifu2x-caffe\waifu2x-caffe-cui.exe -i temp\3b11b8e0-2e50-11ea-945a-2fc02fa70f7f\extracted\thread1 -o temp\3b11b8e0-2e50-11ea-945a-2fc02fa70f7f\upscaled -w 1920 -h 1440 --model_dir C:\Users\Chris\Downloads\waifu2x-caffe\models\cunet -p cudnn --gpu 0 -b 32 -c 128 -e png -m scale -n 2 -t 0

\C:\snapshot\video2x\node_modules\q\q.js:155 throw e; ^

Error: Command failed: C:\Users\Chris\Downloads\waifu2x-caffe\waifu2x-caffe-cui.exe -i temp\3b11b8e0-2e50-11ea-945a-2fc02fa70f7f\extracted\thread1 -o temp\3b11b8e0-2e50-11ea-945a-2fc02fa70f7f\upscaled -w 1920 -h 1440 --model_dir C:\Users\Chris\Downloads\waifu2x-caffe\models\cunet -p cudnn --gpu 0 -b 32 -c 128 -e png -m scale -n 2 -t 0 Could not create log file: File exists COULD NOT CREATE LOGFILE '20200103-130854.17500'! F0103 13:08:54.012122 13152 cudnn_conv_layer.cu:28] Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0) CUDNN_STATUS_BAD_PARAM Check failure stack trace:

at ChildProcess.exithandler (child_process.js:281:12)
at emitTwo (events.js:126:13)
at ChildProcess.emit (events.js:214:7)
at maybeClose (internal/child_process.js:915:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)

c:\Users\Chris\Downloads\video2x.v1.0.2>

[kwsou]

Hi there,

I don't see anything obvious that could be wrong. Could you check if the waifu2x-caffe GUI works with CUDNN: Run waifu2x-caffe.exe and click on the Check cuDNN button to ensure that waifu2x can use the cuDNN library.

If that works, you can try to simulate what it was trying to do on the command line: C:\Users\Chris\Downloads\waifu2x-caffe\waifu2x-caffe-cui.exe -i temp\3b11b8e0-2e50-11ea-945a-2fc02fa70f7f\extracted\thread1 -o temp\3b11b8e0-2e50-11ea-945a-2fc02fa70f7f\upscaled -w 1920 -h 1440 --model_dir C:\Users\Chris\Downloads\waifu2x-caffe\models\cunet -p cudnn --gpu 0 -b 32 -c 128 -e png -m scale -n 2 -t 0

You can use temp\3b11b8e0-2e50-11ea-945a-2fc02fa70f7f\extracted\thread1 as input folder and temp\3b11b8e0-2e50-11ea-945a-2fc02fa70f7f\test as output folder. The other options you can leave as-is.

Let me know how that goes, thanks.

[bigdwg71]

@kwsou Thanks for responding!

Tested the CUDNN library and that says "cuDNN can be used" so I think that's fine.

Running the command directly as you suggests, produces the same error:

c:\Users\Chris\Downloads\waifu2x-caffe>waifu2x-caffe-cui.exe -i .\temp\a71c56f0-2e5d-11ea-84ca-1f69fc29fce4\extracted\thread1 -o .\temp\a71c56f0-2e5d-11ea-84ca-1f69fc29fce4\upscaled -w 1920 -h 1440 --model_dir C:\Users\Chris\Downloads\waifu2x-caffe\models\cunet -p cudnn --gpu 0 -b 32 -c 128 -e png -m scale -n 2 -t 0 Could not create log file: File exists COULD NOT CREATE LOGFILE '20200104-132634.16640'! F0104 13:26:34.076897 1276 cudnn_conv_layer.cu:28] Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0) CUDNN_STATUS_BAD_PARAM Check failure stack trace:

[bigdwg71]

Also, adding some detail. I am using CUDNN 7.6.5 library with a 2070 on Windows 10. I have not installed the CUDA toolkit, but I didn't see that in your instructions, so hopefully that's just not implied.

[kwsou]

Ah, here's an idea: can you run that same command again? But this time monitor your GPU memory usage. Does it run for a little bit, use up all of the memory available, and then crash with that error?

[bigdwg71]

It does seem like it starts to ramp up GPU memory usage. The 2070 has 8GB. and I see the memory get to 7260MB (using GPU-Z to monitor it) and then it crashes out. If it is memory related, is there a way to reduce the memory footprint? Are there settings that may affect this?

[kwsou]

Ah, so that's the problem. My GPU has 11GB of memory and most likely you are using the same config as that and running out of memory. You need to lower the -b and -c parameters to make it work under 8GB.

Here are my vague descriptions of what they do (you'd need to ask waifu-caffe developer for further details):

-b <int>,  --batch_size <int>
  input batch size

-c <int>,  --crop_size <int>
  input image split size

They correspond to these options in the GUI:

You can change these values in the config json file you are using (rtx-2080-ti.json), under the CUDNN_PRESET category. At the very minimum, b=1 and c=64 should work, but you wouldn't be fully utilizing the GPU.

If you do get it working, can you let me know what values you ended up using (or creating a PR to add rtx-2070.json)?

Thanks!

[jChuckran]

Hello! I was running into the same issue using a 2070 SUPER 8GB.

I found success using

    "CUDNN_PRESET": [
        {
            "flag": "-p",
            "value": "cudnn"
        },
        {
            "flag": "--gpu",
            "value": 0
        },
        {
            "flag": "-b",
            "value": 32
        },
        {
            "flag": "-c",
            "value": 64
        }
    ],