Open cschwem2er opened 7 years ago
First, you are welcome! I want tuber to be as good as it can be so your feedback is valuable! Thank you.
403 error is an 'access forbidden' error. Generally YT restricts access to captions to videos the person owns, which is a bit of a shame but it likely follows copyright law. More on YT errors here: https://developers.google.com/youtube/v3/docs/errors
Let me investigate a bit more and get back but that is my quick answer.
Changing the 'scope' of the authorization may help also.
EDIT 2: I tried to download the captions for about 600 videos and got captions for about 60 of them. It seems that there's an option for the video uploaders to allow third-party contributions to captions (https://stackoverflow.com/questions/30653865/downloading-captions-always-returns-a-403).
So, seems like there's no quick fix. If anybody comes up with another solution, I'd be very interested.
--
EDIT: The below worked for a while, but now I'm getting 403 errors as well. I can't tell if I'm hitting a rate limit or something else.
--
I've been working on this as well. I was able to request the captions without an auth error but got hex back. Here's what I'm doing that seems to work reasonably well. There's still some mess in the output text, but that can be cleaned up.
get_captions(id="____") #Note that this is the caption id returned by list_caption_tracks, not the video id
captions = as.character(captions) captions = paste(captions, sep="", collapse="")
captions = sapply(seq(1, nchar(cpations), by=2), function(x) substr(captions, x, x+1)) captions = rawToChar(as.raw(strtoi(captions, 16L)))
It seems that you can only get captions for your own videos with the get_captions
function. This appears to be due to a change made by YouTube. Previously you could use the API to get captions for videos that had third-party contributions enabled (see the StackOverflow discussion linked by @danielcarter above). However, the option of enabling those has been removed by YouTube. Also see #78.
As an alternative, you can use the get_caption
function from the youtubecaption
package (https://github.com/jooyoungseo/youtubecaption).
@jobreu thanks for this!
if the R package you mention that uses a python package underneath the hood works, do you not think we can get it to work here?
Possibly so. The youtubecaption
package uses the youtube-transcript-api
Python library under the hood. It does not require setting up or providing YouTube API credentials. I haven't had a chance to look at the code for the Python library, so I don't know whether it uses the YouTube API (and if so with what credentials) or if it employs a screen scraping approach
thanks for this @jobreu! looks like the python package is scraping the captions? the kind of thing plausibly that can google probably doesn't endorse.
Yeah, it appears to scrape the comments. It's working for now but that may change if the package gets more (or too) popular or if the YouTube website changes.
Thanks!
A general-purpose scraping solution is apify. I recommend using that for scraping jobs.
@jobreu can you do a PR on the readme explaining the current functionality and alternatives as lots of people want captions. Thank you sir!
Done :-) I have also taken the liberty of removing the reference to the Freebase API in the README as that one has been deprecated.
Hi,
first of all thank you so much for creating this package. Until now I relied on the excellent Youtube Data Tool and its awesome to finally have a proper way of working with youtube data in R. While your latest change, the possibility to get all comments for a video, works very well for me, I have trouble with catching captions:
First, the language parameter does not seem to work as intended, as non-english results get returned as well. Second, pass one of the id's to
list_caption_tracks
does not work:Do you have any idea what's going on here?
Edit: Is it possible that I need more permissions in addition to all youtube and the freebase api?