Video detection problems

[myrainbowandsky]

Ubuntu 24.04 LTS (GNU/Linux 6.8.0-36-generic x86_64) R version 4.3.3 (2024-02-29) -- "Angel Food Cake" Copyright (C) 2024 The R Foundation for Statistical Computing Platform: x86_64-pc-linux-gnu (64-bit)

The text and image detection work fine. But the video detection has following problems:

For a local video:

> video_url <- "./test.mp4"
> # Array of emotion labels
emotions <- c("excitement", "happiness", "pride", "anger", "fear", "sadness", "neutral")

# Run FER on `nframes` of the video
result <- video_scores(video_url, classes = emotions,
                    nframes = 10, save_video = TRUE,
                    save_frames = TRUE, video_name = 'boris-johnson',
                    start = 10, end = 120)
Error occurred while getting video stream. Retrying ...
Error occurred while getting video stream. Retrying ...
Error occurred while getting video stream. Retrying ...
Error in py_call_impl(callable, call_args$unnamed, call_args$named) :
  ValueError: Failed to get video stream after 3 attempts.
Run `reticulate::py_last_error()` for details.
> reticulate::py_last_error()

── Python Exception Message ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Traceback (most recent call last):
  File "/home/lawrencexu/R/x86_64-pc-linux-gnu-library/4.3/transforEmotion/python/video.py", line 44, in yt_analyze
    raise ValueError("Failed to get video stream after 3 attempts.")
ValueError: Failed to get video stream after 3 attempts.

── R Traceback ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
    ▆
 1. └─transforEmotion::video_scores(...)
 2.   └─reticulate::py$yt_analyze(...)
 3.     └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
See `reticulate::py_last_error()$r_trace$full_call` for more details.
> reticulate::py_last_error()$r_trace$full_call
[[1]]
video_scores(video_url, classes = emotions, nframes = 10, save_video = TRUE,
    save_frames = TRUE, video_name = "boris-johnson", start = 10,
    end = 120)

[[2]]
reticulate::py$yt_analyze(url = video, nframes = nframes, labels = classes,
    side = face_selection, start = start, end = end, uniform = uniform,
    ff = ffreq, frame_dir = save_dir, video_name = video_name)

[[3]]
py_call_impl(callable, call_args$unnamed, call_args$named)

For Youtube:

> # Video URL or local filepath
video_url <- "https://www.youtube.com/watch?v=hdYNcv-chgY&ab_channel=Conservatives"

# Array of emotion labels
emotions <- c("excitement", "happiness", "pride", "anger", "fear", "sadness", "neutral")

# Run FER on `nframes` of the video
result <- video_scores(video_url, classes = emotions,
                    nframes = 10, save_video = TRUE,
                    save_frames = TRUE, video_name = 'boris-johnson',
                    start = 10, end = 120)
2024-08-02 17:28:57.349907: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2024-08-02 17:28:57.395829: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-08-02 17:28:57.395945: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-08-02 17:28:57.395990: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-08-02 17:28:57.406264: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-08-02 17:28:58.361863: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Error occurred while getting video stream. Retrying ...
Error occurred while getting video stream. Retrying ...
Error occurred while getting video stream. Retrying ...
Error in py_call_impl(callable, call_args$unnamed, call_args$named) :
  ValueError: Failed to get video stream after 3 attempts.
Run `reticulate::py_last_error()` for details.

[atomashevic]

Hi,

For YouTube video the issue is with pytube library and we need to find the replacement, because it seems YT is blocking this type of requests.

For local video, I can't seem to figure out the issue because Error occurred while getting video stream. Retrying ... message is only shown if video_url contains youtube link, if you are using video_url <- "./test.mp4" then I don't know how this can happen at all.

Thanks for reporting this, I'll try to sort out the local file issue first.

[atomashevic]

The issue with Youtube video downloads has been resolved we moved to pyutbefix library and it works without issues.

Install the latest github version of the package and everything will be OK

devtools::install_github("atomashevic/transforEmotion")

Regarding the local video, I wasn't able to reproduce the issue, plese check your code once again. Local files are working well, there is no way for that error to reproduce unless video path contains "youtu.be" in it.

I you will be able to use the package normally now.