--match-filter doesn't work fine

mbnoimi commented 1 year ago

What operating system are you using?

Linux Mint 21.2

What version of Tartube are you using?

v2.4.412

If your bug report is "I can't download this video", please provide a link to the video

/home/laptop/.local/bin/yt-dlp --newline -i --match-filter "duration<600 \& !is_live" --age-limit 21 -r 200K -R 99 --hls-prefer-native --cookies /home/laptop/Downloads/youtube-dl/cookies.txt --write-sub --write-auto-sub --prefer-ffmpeg --break-on-existing --no-cookies --write-desktop-link --sleep-requests 3 --sleep-subtitles 3 --reject-title "Stream|live" --output "/home/laptop/Downloads/youtube-dl/downloads/Temporary Videos/%(title)s.%(ext)s" https://www.youtube.com/watch?v=O9nejN42ZFY
[youtube] Extracting URL: https://www.youtube.com/watch?v=O9nejN42ZFY
[youtube] O9nejN42ZFY: Downloading webpage
[youtube] Sleeping 3.0 seconds ...
[youtube] O9nejN42ZFY: Downloading ios player API JSON
[youtube] Sleeping 3.0 seconds ...
[youtube] O9nejN42ZFY: Downloading android player API JSON
[youtube] Sleeping 3.0 seconds ...
[youtube] O9nejN42ZFY: Downloading m3u8 information
[info] O9nejN42ZFY: Downloading subtitles: en
[info] O9nejN42ZFY: Downloading 1 format(s): 22
ERROR: '<' not supported between instances of 'int' and 'str'

IMPORTANT: I can download same video using the same arguments using CLI

mbnoimi commented 1 year ago

As you can see below CLI result is fine:

$ /home/laptop/.local/bin/yt-dlp --newline -i --match-filter "duration<600" --age-limit 21 -r 200K -R 99 --hls-prefer-native --cookies /home/laptop/Downloads/youtube-dl/cookies.txt --write-sub --write-auto-sub --prefer-ffmpeg --break-on-existing --no-cookies --write-desktop-link --sleep-requests 3 --sleep-subtitles 3 --reject-title "Stream|live" --output "/home/laptop/Downloads/youtube-dl/downloads/Temporary Videos/%(title)s.%(ext)s" https://www.youtube.com/watch?v=O9nejN42ZFY
[youtube] Extracting URL: https://www.youtube.com/watch?v=O9nejN42ZFY
[youtube] O9nejN42ZFY: Downloading webpage
[youtube] Sleeping 3.0 seconds ...
[youtube] O9nejN42ZFY: Downloading ios player API JSON
[youtube] Sleeping 3.0 seconds ...
[youtube] O9nejN42ZFY: Downloading android player API JSON
[youtube] Sleeping 3.0 seconds ...
[youtube] O9nejN42ZFY: Downloading m3u8 information
[info] O9nejN42ZFY: Downloading subtitles: en
[download] talking about Godot Engine for 12 hours does not pass filter (duration<600), skipping ..

axcore commented 1 year ago

The two examples are not the same. One uses --match-filter "duration<600" and the other uses --match-filter "duration<600 \& !is_live"

$ yt-dlp --match-filter "duration<600 \& !is_live" https://www.youtube.com/watch?v=O9nejN42ZFY
bash: !is_live: event not found

$ yt-dlp --match-filter "duration<600" https://www.youtube.com/watch?v=O9nejN42ZFY
[youtube] Extracting URL: https://www.youtube.com/watch?v=O9nejN42ZFY
[youtube] O9nejN42ZFY: Downloading webpage
WARNING: [youtube] unable to extract initial player response; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
[youtube] O9nejN42ZFY: Downloading ios player API JSON
[youtube] O9nejN42ZFY: Downloading android player API JSON
[youtube] O9nejN42ZFY: Downloading iframe API JS
[youtube] O9nejN42ZFY: Downloading player 16f9263d
[youtube] O9nejN42ZFY: Downloading web player API JSON
[youtube] O9nejN42ZFY: Downloading m3u8 information
WARNING: [youtube] unable to extract yt initial data; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
WARNING: [youtube] Incomplete data received in embedded initial data; re-fetching using API.
[youtube] O9nejN42ZFY: Downloading initial data API JSON
[download] talking about Godot Engine for 12 hours does not pass filter (duration<600), skipping ..

axcore commented 1 year ago

In bash, it is apparently necessary to use \!is_live.

$yt-dlp --match-filter "duration<600 \& \!is_live" https://www.youtube.com/watch?v=O9nejN42ZFY
[youtube] Extracting URL: https://www.youtube.com/watch?v=O9nejN42ZFY
[youtube] O9nejN42ZFY: Downloading webpage
WARNING: [youtube] unable to extract initial player response; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
[youtube] O9nejN42ZFY: Downloading ios player API JSON
[youtube] O9nejN42ZFY: Downloading android player API JSON
[youtube] O9nejN42ZFY: Downloading iframe API JS
[youtube] O9nejN42ZFY: Downloading player 16f9263d
[youtube] O9nejN42ZFY: Downloading web player API JSON
[youtube] O9nejN42ZFY: Downloading m3u8 information
WARNING: [youtube] unable to extract yt initial data; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
WARNING: [youtube] Incomplete data received in embedded initial data; re-fetching using API.
[youtube] O9nejN42ZFY: Downloading initial data API JSON
[info] O9nejN42ZFY: Downloading 1 format(s): 137+251
ERROR: '<' not supported between instances of 'int' and 'str'

mbnoimi commented 1 year ago

The two examples are not the same. One uses --match-filter "duration<600" and the other uses --match-filter "duration<600 \& !is_live"

For unknown reason Tartube generates !is_live. I don't want to download any streams but I couldn't understand why it generates invalid token!

axcore commented 1 year ago

For unknown reason Tartube generates !is_live

The reason is, you have disabled downloading livestreams.

Edit > System preferences... > Operations > Livestreams Disable Do not check/download any livestream

mbnoimi commented 1 year ago

The reason is, you have disabled downloading livestreams.

But I don't want to download livestreams

axcore commented 1 year ago

It seems to be a yt-dlp issue. Because you are a loyal customer, I opened a new issue at the yt-dlp page.

axcore commented 1 year ago

Does this work for you?

--match-filter "duration<600 & !is_live"

match-filter

mbnoimi commented 1 year ago

Does this work for you?

--match-filter "duration<600 & !is_live"

It didn't through any error but yt-dlp actually ignore it! This video length is 12 hrs while the filter is 600 sec!

mbnoimi commented 1 year ago

I run this command using Tartube:

/home/admino/.local/bin/yt-dlp --newline -i --match-filter !is_live --age-limit 21 -R 99 --hls-prefer-native --cookies /home/admino/Downloads/youtube-dl/cookies.txt --write-sub --write-auto-sub --prefer-ffmpeg --break-on-existing --no-cookies --write-desktop-link --sleep-requests 3 --sleep-subtitles 3 --reject-title "برومو|ترويجي|Stream|مباشر|live|ترويج|الترويجي|الترويج|تشويقي|أحدث العروض|أفضل العروض|العروض الجديدة|عروض شهر|عروض ناشونال|عرض أول|عرض جديد|تعرف على أبرز|عرض حلقة|موعد العرض|التونة العنيدة|الكوارث الجوية|الكوارث الجويّة|موسم جديد|الإعلان الرسمي|ابتداءً من|برنامج جديد|ابتداء من|شاهدوا الآن" --output "/home/admino/Downloads/youtube-dl/downloads/Unsorted Videos/%(title)s.%(ext)s" --match-filter "duration<600 & !is_live" https://www.youtube.com/watch?v=O9nejN42ZFY

axcore commented 1 year ago

--match-filter "duration>600 & !is_live"

That downloads only videos which are at least 600 seconds long, and are not live. Is that what you wanted?

mbnoimi commented 1 year ago

--match-filter "duration>600 & !is_live"

That downloads only videos which are at least 600 seconds long, and are not live. Is that what you wanted?

Oops, I forgot >. Anyway, Tartube needs a bug fixing for this issue.

axcore commented 1 year ago

Yes, it does.

axcore commented 8 months ago

I am 99.9% sure that I updated the Tartube code, some months, to better handle --match-filter "duration>600 & !is_live". But, I can't find anything about it in my notes.

It should be working better in Tartube v2.5.0, but feel free to re-open this issue if not.

axcore / tartube