search

ClipsAI / clipsai

Clips AI is an open-source Python library that automatically converts long videos into clips.
https://www.clipsai.com
MIT License
132 stars 22 forks source link

free(): double free detected in tcache 2 in resize() #4

Closed vlad196 closed 4 months ago

vlad196 commented 5 months ago

When i try resize() video with Nvidia driver, i have (Even if i try use CPU):

[nltk_data] Downloading package punkt to /home/vlad/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
/home/vlad/bin/clipsai-project/.venv/lib/python3.11/site-packages/pyannote/audio/core/io.py:43: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
  torchaudio.set_audio_backend("soundfile")
INFO:pyscenedetect:Downscale factor set to 5, effective resolution: 256 x 144
INFO:pyscenedetect:Detecting scenes...
free(): double free detected in tcache 2
[1]    19517 IOT instruction (core dumped)  python crop.py

And code is exit.

My code:

import configparser, os, os.path

from clipsai import resize, AudioVideoFile, MediaEditor

config = configparser.ConfigParser()
config.read('settings.ini')
os.environ['CUDA_VISIBLE_DEVICES'] = config['Default']['CUDA_VISIBLE_DEVICES']

SourceVideo = config['Default']['SourceVideo']
OutputVideo = config['Default']['OutputVideo']
ResizeVideo = config['Default']['ResizeVideo']
PyannoteToken = config['Default']['PyannoteToken']
Name = config['Default']['Name']
media_editor = MediaEditor()

def resize(PyannoteToken, OutputVideo, num_clip, Name, ResizeVideo):
    crop = resize(
        video_file_path = f'{OutputVideo}{Name}-clip{num_clip}.mp4',
        pyannote_auth_token = PyannoteToken,
        aspect_ratio=(9, 16),
    )

    media_file = AudioVideoFile(f'{OutputVideo}{Name}-clip{num_clip}.mp4')

    resized_video_file = media_editor.resize_video(
        original_video_file=media_file,
        resized_video_file_path=f'{ResizeVideo}{Name}-clip{num_clip}(crop).mp4',  # doesn't exist yet
        width=crop.crop_width,
        height=crop.crop_height,
        segments=crop.to_dict()["segments"],
    )
if __name__ == '__main__':
    num_clip = 0
    for clip in range(1, len(os.listdir(OutputVideo))):
        num_clip += 1
        resize(PyannoteToken, OutputVideo, num_clip, Name, ResizeVideo) # this

set device parameter in resize() function (cuda or cpu) nothing change set CUDA_VISIBLE_DEVICES environment nothing change. (0, 1, " ", etc) (for Transcriber() environment is working) For Transcriber() CUDA work pretty well

If i change driver to nouveau, i have normal output and video successfully resize:

INFO:pyscenedetect:Downscale factor set to 5, effective resolution: 256 x 144
INFO:pyscenedetect:Detecting scenes...
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1713724103.863680    2544 gl_context_egl.cc:85] Successfully initialized EGL. Major : 1 Minor: 5
I0000 00:00:1713724103.865078    2850 gl_context.cc:357] GL version: 3.2 (OpenGL ES 3.2 Mesa 24.1.0-devel (git-65f5f1559f)), renderer: NV166
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
ffmpeg version n6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13.2.1 (GCC) 20230801
  configuration: --prefix=/usr --disable-debug ...(etc)

nvidia-smi:

Sun Apr 21 20:13:21 2024       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.76                 Driver Version: 550.76         CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 2070        Off |   00000000:0A:00.0  On |                  N/A |
|  0%   43C    P8             18W /  175W |     431MiB /   8192MiB |      1%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A       865      G   /usr/lib/Xorg                                  15MiB |
|    0   N/A  N/A       950      G   /usr/bin/kwalletd6                              1MiB |
|    0   N/A  N/A       977      G   /usr/bin/kwin_wayland                          78MiB |
|    0   N/A  N/A      1061      G   /usr/bin/Xwayland                               2MiB |
|    0   N/A  N/A      1102      G   /usr/bin/ksmserver                              1MiB |
|    0   N/A  N/A      1104      G   /usr/bin/kded6                                  1MiB |
|    0   N/A  N/A      1134      G   /usr/bin/plasmashell                           65MiB |
|    0   N/A  N/A      1168      G   /usr/lib/kactivitymanagerd                      1MiB |
|    0   N/A  N/A      1175      G   ...b/polkit-kde-authentication-agent-1          1MiB |
|    0   N/A  N/A      1176      G   /usr/lib/org_kde_powerdevil                     1MiB |
|    0   N/A  N/A      1177      G   /usr/lib/xdg-desktop-portal-kde                 1MiB |
|    0   N/A  N/A      1394      G   /usr/lib/kdeconnectd                            1MiB |
|    0   N/A  N/A      1443      G   /usr/bin/kclockd                                1MiB |
|    0   N/A  N/A      1453      G   /usr/bin/usbguard-qt                            1MiB |
|    0   N/A  N/A      1463      G   /usr/bin/kaccess                                1MiB |
|    0   N/A  N/A      1468      G   /usr/lib/DiscoverNotifier                       1MiB |
|    0   N/A  N/A      1469      G   /usr/bin/kalendarac                             1MiB |
|    0   N/A  N/A      1621      G   /usr/bin/konsole                                1MiB |
|    0   N/A  N/A      1772      G   /usr/bin/dolphin                                1MiB |
|    0   N/A  N/A      3015      G   ...apHandler --variations-seed-version        124MiB |
|    0   N/A  N/A     16983      G   /usr/bin/kate                                   1MiB |
|    0   N/A  N/A     18631      G   ...erProcess --variations-seed-version         28MiB |
|    0   N/A  N/A     20157      G   /usr/lib/kf6/kioworker                          1MiB |
+-----------------------------------------------------------------------------------------+

glxinfo -B:

name of display: :1
display: :1  screen: 0
direct rendering: Yes
Memory info (GL_NVX_gpu_memory_info):
    Dedicated video memory: 8192 MB
    Total available memory: 8192 MB
    Currently available dedicated video memory: 7492 MB
OpenGL vendor string: NVIDIA Corporation
OpenGL renderer string: NVIDIA GeForce RTX 2070/PCIe/SSE2
OpenGL core profile version string: 4.6.0 NVIDIA 550.76
OpenGL core profile shading language version string: 4.60 NVIDIA
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 4.6.0 NVIDIA 550.76
OpenGL shading language version string: 4.60 NVIDIA
OpenGL context flags: (none)
OpenGL profile mask: (none)

OpenGL ES profile version string: OpenGL ES 3.2 NVIDIA 550.76
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20

pip list:

Package                  Version
------------------------ -----------
absl-py                  2.1.0
aiohttp                  3.9.5
aiosignal                1.3.1
alembic                  1.13.1
antlr4-python3-runtime   4.9.3
asteroid-filterbanks     0.4.0
attrs                    23.2.0
audioread                3.0.1
av                       11.0.0
certifi                  2024.2.2
cffi                     1.16.0
charset-normalizer       3.3.2
click                    8.1.7
clipsai                  0.2.1
coloredlogs              15.0.1
colorlog                 6.8.2
contourpy                1.2.1
ctranslate2              4.1.0
cycler                   0.12.1
decorator                5.1.1
docopt                   0.6.2
einops                   0.7.0
facenet-pytorch          2.5.3
faster-whisper           1.0.0
ffmpeg-python            0.2.0
filelock                 3.13.4
flatbuffers              24.3.25
fonttools                4.51.0
frozenlist               1.4.1
fsspec                   2024.3.1
future                   1.0.0
greenlet                 3.0.3
huggingface-hub          0.22.2
humanfriendly            10.0
HyperPyYAML              1.2.2
idna                     3.7
iniconfig                2.0.0
jax                      0.4.26
jaxlib                   0.4.26
Jinja2                   3.1.3
joblib                   1.4.0
julius                   0.2.7
kiwisolver               1.4.5
lazy_loader              0.4
librosa                  0.10.1
lightning                2.2.2
lightning-utilities      0.11.2
llvmlite                 0.42.0
Mako                     1.3.3
markdown-it-py           3.0.0
MarkupSafe               2.1.5
matplotlib               3.8.4
mdurl                    0.1.2
mediapipe                0.10.11
ml-dtypes                0.4.0
mpmath                   1.3.0
msgpack                  1.0.8
multidict                6.0.5
networkx                 3.3
nltk                     3.8.1
numba                    0.59.1
numpy                    1.26.4
nvidia-cublas-cu12       12.1.3.1
nvidia-cuda-cupti-cu12   12.1.105
nvidia-cuda-nvrtc-cu12   12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12        8.9.2.26
nvidia-cufft-cu12        11.0.2.54
nvidia-curand-cu12       10.3.2.106
nvidia-cusolver-cu12     11.4.5.107
nvidia-cusparse-cu12     12.1.0.106
nvidia-nccl-cu12         2.19.3
nvidia-nvjitlink-cu12    12.4.127
nvidia-nvtx-cu12         12.1.105
omegaconf                2.3.0
onnxruntime              1.17.3
opencv-contrib-python    4.9.0.80
opencv-python            4.9.0.80
opt-einsum               3.3.0
optuna                   3.6.1
packaging                24.0
pandas                   2.2.2
pillow                   10.3.0
pip                      24.0
platformdirs             4.2.0
pluggy                   1.4.0
pooch                    1.8.1
primePy                  1.3
protobuf                 3.20.3
psutil                   5.9.8
pyannote.audio           3.1.1
pyannote.core            5.0.0
pyannote.database        5.1.0
pyannote.metrics         3.2.1
pyannote.pipeline        3.0.1
pycparser                2.22
Pygments                 2.17.2
pynvml                   11.5.0
pyparsing                3.1.2
pytest                   8.1.1
python-dateutil          2.9.0.post0
python-magic             0.4.27
pytorch-lightning        2.2.2
pytorch-metric-learning  2.5.0
pytz                     2024.1
PyYAML                   6.0.1
regex                    2024.4.16
requests                 2.31.0
rich                     13.7.1
ruamel.yaml              0.18.6
ruamel.yaml.clib         0.2.8
safetensors              0.4.3
scenedetect              0.6.2
scikit-learn             1.4.2
scipy                    1.13.0
semver                   3.0.2
sentence-transformers    2.7.0
sentencepiece            0.2.0
setuptools               65.5.0
shellingham              1.5.4
six                      1.16.0
sortedcontainers         2.4.0
sounddevice              0.4.6
soundfile                0.12.1
soxr                     0.3.7
speechbrain              1.0.0
SQLAlchemy               2.0.29
sympy                    1.12
tabulate                 0.9.0
tensorboardX             2.6.2.2
threadpoolctl            3.4.0
tokenizers               0.15.2
torch                    2.2.2
torch-audiomentations    0.11.1
torch-pitch-shift        1.2.4
torchaudio               2.2.2
torchmetrics             1.3.2
torchvision              0.17.2
tqdm                     4.66.2
transformers             4.39.3
triton                   2.2.0
typer                    0.12.3
typing_extensions        4.11.0
tzdata                   2024.1
urllib3                  2.2.1
whisperx                 3.1.1
yarl                     1.9.4
vlad196 commented 4 months ago

This is an error from mediapipe. The error comes from here: https://github.com/ClipsAI/clipsai/blob/main/clipsai/resize/resizer.py#L76