pystardust / ytfzf

A posix script to find and watch youtube videos from the terminal. (Without API)
GNU General Public License v3.0
3.79k stars 343 forks source link

Always get 420p video, no matter what I do #462

Closed slarrain closed 2 years ago

slarrain commented 2 years ago

Like the title says, no matter what I do, I always get a 640x360 youtube video.

I have tried:

  1. Editing the config file to video_pref="bestvideo*+bestaudio/best".
  2. Specifying --video-pref=best.
  3. Selecting a higher format with -f.

But I always end up with this.

 (+) Video --vid=1 (*) (h264 640x360 29.970fps)
 (+) Audio --aid=1 (*) (aac 2ch 44100Hz)
     Subs  --sid=1 --slang=English (auto-generated) 'unknown_video' (webvtt) (external)
AO: [pulse] 44100Hz stereo 2ch float
VO: [gpu] 640x360 yuv420p

I tried looking for something harcoded, but I couldn't find anything.

Any ideas?

By the way, thank you for this really awesome application. I love it. The only feature that is missing, in my opinion, is being able to access the Recommended youtube playlist for my user. Thanks!

Euro20179 commented 2 years ago

Try running ytfzf -a -If --loop=0 --video-pref=best search query, it should output best.

If that says -I is invalid or something, try updating to v2.0

git clone https://github.com/pystardust/ytfzf
cd ytfzf
git checkout v2.0
sudo make install man

What shell are you using? It really shouldn't matter, but you could try running bash ytfzf search query.

It could be something in your mpv config, but I find that unlikely.

Are you using yt-dlp or youtube-dl (i'd recommend using yt-dlp if you're using youtube-dl)

I tried looking for something harcoded, but I couldn't find anything.

The only thing that could be considered hard coded is that, if video_pref is unset or "", it defaults to best, but that's not happening here.

The only feature that is missing, in my opinion, is being able to access the Recommended youtube playlist for my user

I've implemented this before, and it was extremely inconsistent so I didn't go through with it.

slarrain commented 2 years ago

Thank you so much for your quick answer. I think you may have found the issue.

Try running ytfzf -a -If --loop=0 --video-pref=best search query, it should output best.

It does output best.

What shell are you using? It really shouldn't matter, but you could try running bash ytfzf search query.

I'm using zsh but running it with bash made no difference.

It could be something in your mpv config, but I find that unlikely.

I don't even have an mpv config.

Are you using yt-dlp or youtube-dl (i'd recommend using yt-dlp if you're using youtube-dl)

I couldn't find where to specify which one to use. But I did a pip install --upgrade youtube-dl and indeed my resolution went up from 420p to 720p. Wow! So I think that tell us that it was using youtube-dl instead of yt-dlp. How or where can I change that? ytdl_path=/usr/local/bin/yt-dlp on the config.sh file did not to anything.

I've implemented this before, and it was extremely inconsistent so I didn't go through with it.

If you ever plan on re-adding this, I'd love to help test and debug. I'm a YouTube's Recommended videos heavy user.

Euro20179 commented 2 years ago

It does output best.

It might be that that video you're watching has a max quality of 420p?

I'm using zsh but running it with bash made no difference.

As I expected.

I don't even have an mpv config.

Also as I expected.

I couldn't find where to specify which one to use. But I did a pip install --upgrade youtube-dl and indeed my resolution went > up from 420p to 720p. Wow! So I think that tell us that it was using youtube-dl instead of yt-dlp.

Ytfzf (in the 2.0 version) uses yt-dlp by default, if yt-dlp is not installed it falls back to youtube-dl. If you have both you could run ytfzf --ytdl-path=youtube-dl ....

However the fact that installing youtube-dl changed your resolution is very interesting, and this whole issue you're having could be with yt-dlp/youtube-dl

How or where can I change that? ytdl_path=/usr/local/bin/yt-dlp on the config.sh file did not to anything.

This is also odd, make sure conf.sh has execute permissions, and make sure you're on 2.0, (ytfzf --version), other than that I'll look into it in a bit, and get back to you

Euro20179 commented 2 years ago

If you ever plan on re-adding this, I'd love to help test and debug. I'm a YouTube's Recommended videos heavy user.

Well I implemented it as an addon, you put this script in ~/.config/ytfzf/scrapers/recommended, make sure recommended has execute permissions, then you'll need to get your cookies on youtube, to do this:

  1. open youtube
  2. sign in
  3. press ctrl shift i
  4. go to network tab
  5. refresh the page
  6. click the get request for file /
  7. find request headers
  8. copy the Cookie header
  9. put in ~/.config/ytfzf/cookie (may change if i add this to actual script) (you can also fiddle around with which cookies are, and are not necessary)
  10. run ytfzf -crecommended

Do note that watching videos will not update your recommendations.

script:

#!/bin/sh
# needs search_query as $*
## Scrape data and store video information in videos_data ( and thumbnails )

scrape_recommended () {
    search=$1
    output_json_file=$2
    _tmp_html="${session_temp_dir}/yt-search.html"
    _tmp_json="${session_temp_dir}/yt-search.json"

    #sp is the urlquery youtube uses for sorting videos
    #only runs if --filter-id or --sp was unspecified
    #youtube puts in %253d one ore more times in the filter id, it doesn't seem useful, so we are removing it if it's in the filter
    sp=${sp%%%*}

    printf "%s\n" "Scraping Youtube..."

    read -r cookie < "$HOME/.config/ytfzf/cookie"
    cookie="${cookie#Cookie: }"
    curl -s -f -b "$HOME/.config/ytfzf/cookie" "https://www.youtube.com" \
            -H "User-Agent: $4" \
            -H 'Accept-Language: en-US,en;q=0.9' \
            -H "Cookie: $cookie" \
            --compressed > "$_tmp_html" || exit "$?"
    sed -n '/var *ytInitialData/,$p' < "$_tmp_html" |
       tr -d '\n' |
       sed -E ' s_^.*var ytInitialData ?=__ ; s_;</script>.*__ ;' > "$_tmp_json"

    #gets a list of videos
    {
    jq '[ .contents|
    ..|.videoRenderer? |
    select(. !=null) |
        {
            url: "'"${yt_video_link_domain}"'/watch?v=\(.videoId)",
            title: .title.runs[0].text,
            channel: .longBylineText.runs[0].text,
            duration:.lengthText.simpleText,
            views: .shortViewCountText.simpleText,
            date: .publishedTimeText.simpleText,
            description: .detailedMetadataSnippets[0].snippetText.runs[0].text,
            ID: .videoId,
            thumbs: .thumbnail.thumbnails[0].url
        }
    ]' < "$_tmp_json" 

     jq '[ .contents|
    ..|.playlistRenderer? |
    select(. !=null) |
        {
            url: "'"${yt_video_link_domain}"'/playlist?list=\(.videoId)",
            title: "[Playlist] \(.title.simpleText)",
            channel: .longBylineText.runs[0].text,
            duration: "\(.videoCount) videos",
            views: "playlist",
            date: "playlist",
            ID: .playlistId,
            thumbs: .thumbnails[0].thumbnails[0].url,
            action: "scrape type=invidious-playlist search='"${yt_video_link_domain}"'/playlist?list=\(.playlistId)"
        }
    ]' <"$_tmp_json"
    } >> "$output_json_file"
}
slarrain commented 2 years ago

It might be that that video you're watching has a max quality of 420p?

Nope. I triple checked that.

This is also odd, make sure conf.sh has execute permissions, and make sure you're on 2.0, (ytfzf --version), other than that I'll look into it in a bit, and get back to you

It has execute permissions and it is reading from conf.sh, because I put maxres to the thumbnails and if I comment it out, it looks pretty bad. Version was 2.0. Now I'm on 2.1.rc-4, but it is the same.

Ytfzf (in the 2.0 version) uses yt-dlp by default, if yt-dlp is not installed it falls back to youtube-dl. If you have both you could run ytfzf --ytdl-path=youtube-dl ....

However the fact that installing youtube-dl changed your resolution is very interesting, and this whole issue you're having could be with yt-dlp/youtube-dl

To me this is definitely it. Because I had youtube-dl installed and I got 420p. Then when I --upgrade it, I got 720p. And now that I uninstall it, it doesn't work (even though I have yt-dlp installed and working). So ytfzf for some reason is not using yt-dlp.

this is what I was getting with ytfzfafter uninstalling youtube-dl:

[ytdl_hook]
[ytdl_hook] youtube-dl failed: not found or not enough permissions
Failed to recognize file format.

At first, I thought it might have to be related to this line on ytfzf:

[ -z "$ytdl_path" ] && { dep_check "yt-dlp" && ytdl_path="yt-dlp" || ytdl_path="youtube-dl"; }

because it was the only one that even mentioned youtube-dl. However, if I tried to play a youtube video with mpv, I got the exact same error:

➜  ytfzf git:(master) mpv https://www.youtube.com/watch\?v\=AttUjeGFHic
[ytdl_hook]
[ytdl_hook] youtube-dl failed: not found or not enough permissions
Failed to recognize file format.

That got me thinking. Maybe it is not and yt-dlp/youtube-dl problem. Maybe is an mpv problem. Furthermore, if I added the -d flag for downloading the video, it did so with no problem (and no youtube-dl installed, hence using yt-dlp).
So, what did I do? I upgraded mpv to the latest version and Voila!, ytfzf is working great with 1080p videos being played on mpv.

So first, it is good to know that if someone else runs into this problem in the future, it is not related to ytfzf but to mpv. And second, so sorry @Euro20179 for taking away your time with a problem that didn't had to do with your package but with the mpv app not being on its latest version.

Euro20179 commented 2 years ago

So first, it is good to know that if someone else runs into this problem in the future, it is not related to ytfzf but to mpv

TBH, I probably should've thought about this myself, cause ytfzf only uses youtube-dl with the -d and -f options.

so sorry @Euro20179 for taking away your time with a problem that didn't had to do with your package but with the mpv app not being on its latest version.

No problem, I'm bored most of the time.

slarrain commented 2 years ago

Well I implemented it as an addon, you put this script in ~/.config/ytfzf/scrapers/recommended, make sure recommended has execute permissions, then you'll need to get your cookies on youtube, to do this:

1. open youtube

2. sign in

3. press ctrl shift i

4. go to network tab

5. refresh the page

6. click the get request for file `/`

I did not find a / file. But I used the cookies from https://www.youtube.com/api/stats/watchtime and they seem to be working.

7. find request headers

8. copy the Cookie header

9. put in `~/.config/ytfzf/cookie` (may change if i add this to actual script) (you can also fiddle around with which cookies are, and are not necessary)

10. run `ytfzf -crecommended`

Do note that watching videos will not update your recommendations.

This is awesome. It works great. It asks me to do a search though, but apparently it doesn't affect the videos that are outputted. And the videos seem to be from my home recommendations. This is so awesome. Thank you so much.

Euro20179 commented 2 years ago

This is awesome. It works great. It asks me to do a search though

Yes there is a variable to fix that in conf.sh

scrape_search_exclude=" p S SI T invidious-popular H history youtube-subscriptions youtube-trending recommended "

I'm planning on making a variable called custom_scrape_search_exclude so that you don't have to type out all the scrapes you don't want to search for. When I do that all youll have to have is

custom_scrape_search_exclude="recommended other-scrape -other-scrape"

EDIT:

I did not find a / file.

It's whichever request is for youtube.com which for me displayed as /