styler00dollar / VSGAN-tensorrt-docker

Using VapourSynth with super resolution and interpolation models and speeding them up with TensorRT.
BSD 3-Clause "New" or "Revised" License
286 stars 30 forks source link

AttributeError: 'IFNet' object has no attribute 'block0' #64

Closed yuvraj108c closed 9 months ago

yuvraj108c commented 9 months ago

Trying to run RIFE inference pytorch, but getting this error: (btw, i am trying to create a custom node for comfyui inside the docker image, upscaling trt, rife trt works for now)

vspipe -c y4m inference.py - | ffmpeg -i pipe: example.mkv -y

ffmpeg version N-113674-g75697836b1 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 13.2.1 (GCC) 20230801
  configuration: --extra-cflags='-fopenmp -lcrypto -lz -ldl -static-libgcc -I/opt/cuda/include' --extra-cxxflags='-fopenmp -lcrypto -lz -ldl -static-libgcc' --extra-ldflags='-fopenmp -lcrypto -lz -ldl -static-libgcc -L/opt/cuda/lib64' --extra-libs='-lstdc++ -lcrypto -lz -ldl -static-libgcc' --pkg-config-flags=--static --toolchain=hardened --disable-debug --disable-shared --disable-doc --disable-ffplay --enable-static --enable-gpl --enable-gray --enable-nonfree --enable-openssl --enable-iconv --enable-libxml2 --enable-libmp3lame --enable-libfdk-aac --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvpx --enable-libx264 --enable-libx265 --enable-libwebp --enable-libspeex --enable-libaom --enable-libvidstab --enable-libkvazaar --enable-libfreetype --enable-fontconfig --enable-libfribidi --enable-libass --enable-libsoxr --enable-libopenjpeg --enable-libdav1d --enable-librav1e --enable-libsrt --enable-libsvtav1 --enable-libdavs2 --enable-libxavs2 --enable-libvmaf --enable-cuda-nvcc --enable-vapoursynth --enable-libopenh264 --enable-optimizations --enable-cuda-llvm --enable-nvdec --enable-nvenc --enable-cuvid --enable-cuda --enable-pthreads --enable-runtime-cpudetect --enable-lto
  libavutil      58. 39.100 / 58. 39.100
  libavcodec     60. 39.101 / 60. 39.101
  libavformat    60. 21.100 / 60. 21.100
  libavdevice    60.  4.100 / 60.  4.100
  libavfilter     9. 17.100 /  9. 17.100
  libswscale      7.  6.100 /  7.  6.100
  libswresample   4. 13.100 /  4. 13.100
  libpostproc    57.  4.100 / 57.  4.100
Error: Failed to retrieve frame 1 with error: 
Traceback (most recent call last):
  File "src/cython/vapoursynth.pyx", line 3062, in vapoursynth.publicFunction
  File "src/cython/vapoursynth.pyx", line 3064, in vapoursynth.publicFunction
  File "src/cython/vapoursynth.pyx", line 697, in vapoursynth.FuncData.__call__
  File "/ComfyUI/custom_nodes/ComfyUI-Vsgan/src/vfi_inference.py", line 63, in execute
    middle = model_inference.execute(I0, I1, (n % multi) / multi)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ComfyUI/custom_nodes/ComfyUI-Vsgan/src/rife.py", line 59, in execute
    middle = self.model(
             ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ComfyUI/custom_nodes/ComfyUI-Vsgan/src/rife_arch.py", line 407, in forward
    block = [self.block0, self.block1, self.block2, self.block3]
             ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1688, in __getattr__
    raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'IFNet' object has no attribute 'block0'

Output 4 frames in 1.06 seconds (3.79 fps)
Input #0, yuv4mpegpipe, from 'pipe:':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p(progressive), 822x480, 48 fps, 48 tbr, 48 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
[vost#0:0/libx264 @ 0x556a98965640] No filtered frames for output stream, trying to initialize anyway.
[libx264 @ 0x556a98963fc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 AVX512
[libx264 @ 0x556a98963fc0] profile High, level 3.1, 4:2:0, 8-bit
[libx264 @ 0x556a98963fc0] 264 - core 164 r3173 4815cca - H.264/MPEG-4 AVC codec - Copyleft 2003-2024 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=7 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'example.mkv':
  Metadata:
    encoder         : Lavf60.21.100
  Stream #0:0: Video: h264 (H264 / 0x34363248), yuv420p(progressive), 822x480, q=2-31, 48 fps, 1k tbn
      Metadata:
        encoder         : Lavc60.39.101 libx264
      Side data:
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
[out#0/matroska @ 0x556a98964b00] video:0KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown
[out#0/matroska @ 0x556a98964b00] Output file is empty, nothing was encoded(check -ss / -t / -frames parameters if used)
frame=    0 fps=0.0 q=0.0 Lsize=       1KiB time=N/A bitrate=N/A speed=N/A
styler00dollar commented 9 months ago

Sounds like user error. I think you tried to input an invalid architecture version. My current code works. Look up how rife.py specifies architectures.

def inference_clip(video_path="", clip=None):
    clip = core.bs.VideoSource(source=video_path)

    clip = vs.core.resize.Bicubic(clip, format=vs.RGBS, matrix_in_s="709")

    model_inference = RIFE(
       scale=1, fastmode=True, ensemble=False, model_version="rife46", fp16=True
    )

    clip = vfi_inference(
      model_inference=model_inference, clip=clip, multi=2, metric_thresh=0.999
    )

    clip = vs.core.resize.Bicubic(clip, format=vs.YUV420P8, matrix_s="709")
    return clip
styler00dollar commented 9 months ago

As a sidenote, there already is a comfy extention which used vsgan code which is called ComfyUI-Frame-Interpolation.