fent / node-ytdl-core

YouTube video downloader in javascript.
MIT License
4.48k stars 793 forks source link

Update needed? Error 403 #18

Closed ghost closed 9 years ago

ghost commented 9 years ago

Hello there! I've been using ytdl for quite some time now and everything was working really good, but since some days now ytdl is unable to get most of the videos I give with an error 4 always, I think that now it cannot download videos by Vevo and before it was able to, also what I've noticed is that I get always this error:

GET http://r8---sn-8qu-t0az.googlevideo.com/videoplayback?sver=3&pl=23&key=yt5&…=jiGfyyV6KAo&initcwndbps=1272500&ratebypass=yes&ipbits=0&expire=1421387690 403 (Forbidden)

This is my code:

ytdl.getInfo(res.data.items[ self.currentIndex ].player["default"], { downloadURL: true }, function(err, data) {
          console.log(self.currentIndex);

          if (err) {
            return console.log(err);

          } else {
            self.loadMusic(data);

And as I say before it was all working perfectly, now it goes up to the (currentIndex) almost the 3th or 4th before giving a working video, and those videos are not good, most of the time they are jokes, covers or dancing lessons, and what im getting from the api (at least the first one, is the original music video, which now doesnt work).

Thank you for your help guys!

ghost commented 9 years ago

Hello again, I have found a response about my problem in another message:

Q: What is the 403 forbidden error? How to fix it? A: 403 is usually because youtube.com forbids the download of the video. Download YouTube can work around some of them. But as YouTube is consistently changing their protection algorithm, We need to keep up with pushing new releases too.

I still have the latest release, so not sure if you guys knew about this problem, Thank you.

fent commented 9 years ago

I can investigate this weekend.

ghost commented 9 years ago

Thanks @fent

icaliman commented 9 years ago

Same issue here with a lot of videos. youtube-dl seems to work.

Thank you!

ghost commented 9 years ago

Thanks @icaliman i'm gonna try youtube-dl for now to see if it works for me too :smile:


Update: for me that option doenst work either, I get an ENOENT error so not the same error as here but still not working :/

icaliman commented 9 years ago

I think ENOENT is not related to youtube-dl, maybe you try to save the video in a non existent folder.

Have you tried the example from the readme?

var fs = require('fs');
var youtubedl = require('youtube-dl');
var video = youtubedl('http://www.youtube.com/watch?v=90AiXO1pAiA',
  // Optional arguments passed to youtube-dl.
  ['--max-quality=18'],
  // Additional options can be given for calling `child_process.execFile()`.
  { cwd: __dirname });

// Will be called when the download starts.
video.on('info', function(info) {
  console.log('Download started');
  console.log('filename: ' + info.filename);
  console.log('size: ' + info.size);
});

video.pipe(fs.createWriteStream('myvideo.mp4'));
ghost commented 9 years ago

But what I wanted was only to get the info with this:

var youtubedl = require('youtube-dl');
var url = 'http://www.youtube.com/watch?v=WKsjaOqDXgg';
// Optional arguments passed to youtube-dl.
var options = [];
youtubedl.getInfo(url, options, function(err, info) {
  if (err) throw err;

  console.log('id:', info.id);
  console.log('title:', info.title);
  console.log('url:', info.url);
  console.log('thumbnail:', info.thumbnail);
  console.log('description:', info.description);
  console.log('filename:', info.filename);
  console.log('itag:', info.itag);
  console.log('resolution:', info.resolution);
});

I dont need to download the video as what I do with that is only play the video. Thank you.

avihaymenahem commented 9 years ago

@Cedriking Happened to me also, there is any fix for that?

ghost commented 9 years ago

We are still waiting for it :smile: hope to have it soon :sob: .

avihaymenahem commented 9 years ago

So https://github.com/Atraci/Atraci is down :(

ghost commented 9 years ago

Any update on this issue? :disappointed:

andrewrk commented 9 years ago

@Cedriking you could give it a shot yourself. Here are some relevant articles:

http://filippo.io/add-support-for-a-new-video-site-to-youtube-dl/ http://filippo.io/scraping-a-video-out-of-a-more-stubborn-site/

Otherwise we'll all have to wait patiently for an update to this fantastic node module.

alexhulbert commented 9 years ago

Having the same problem myself. I wonder what could've happened?

fent commented 9 years ago

https://github.com/fent/node-ytdl-core/commit/366f63dd62a6c8634acee1de58a14eba73364895

might need a better regexp to represent a javascript variable

http://es5.github.io/#x7.6

avihaymenahem commented 9 years ago

Still doesnt work @fent your fix didnt resolved the problem

ghost commented 9 years ago

After your fix @fent Kiwi is back online and working fine! Thank you a lot for your hard work and help with this issue!

fent commented 9 years ago

@avihaymenahem do you have an example video that doesn't work?

avihaymenahem commented 9 years ago

I have a bunch of them: (Actually every item i try giving 403 error) e-ORhEE9VVg OPf0YbXqDm0 lp-EO5I60KA qDc_5zpBj7s LBr7kECsjcQ

(Trying to .getInfo with downloadurl set to true)

fent commented 9 years ago

those all worked for me. you're on latest right? can I see your error stack?

avihaymenahem commented 9 years ago

im on latest, they didnt worked on getInfo function, and actually no error stack, it just says the link is forbidden on JS console (403)

avihaymenahem commented 9 years ago

Edit: (Attached the info you asked)

111

fent commented 9 years ago

Do you know if it worked before this issue?

Can you try running the irl-test.js tests and see if those work?

If I can't reproduce this myself, then you might have to post the info object that getInfo() returns.

avihaymenahem commented 9 years ago

It didnt worked in the last 4-5 days, before that it worked perfect.

the getinfo json: https://jsonblob.com/54be1ea3e4b0c8c85a144360

Edit: The youtube-dl works, but its a problem for me to tell the users to install python :D

avihaymenahem commented 9 years ago

Hi @fent any update? :)

andrewrk commented 9 years ago

It's working fine for me over here.

fent commented 9 years ago

I'm not sure what to do here, since I can't reproduce it.

What are the contents from this page?

http://www.youtube.com/watch?v=qDc_5zpBj7s

as accessed by node, not your browser, in this file

https://github.com/fent/node-ytdl-core/blob/master/lib/sig.js#L19

avihaymenahem commented 9 years ago

Here you go :) http://pastebin.com/YqsUqpXd

fent commented 9 years ago

Looking back at the info file, the list of formats show that they have an s key, which would make ytdl attempt to decipher it

https://github.com/fent/node-ytdl-core/blob/master/lib/sig.js#L18

but the url doesn't get a signature param added to it. is there an error from getInfo()? there shouldn't be a info object if there is though.

avihaymenahem commented 9 years ago

no error, getInfo returns a proper object, but the links he brings returning 403 (forbidden)

fent commented 9 years ago

it doesn't make sense that the formats have a s, but the url don't have a signature param and there is no error. if you look at the code for deciphering, the callback only gets called without error and with the info if it successfully deciphers the signature.

avihaymenahem commented 9 years ago

You are more than welcome to come and give me a hand, ill will appreciate it a lot! https://github.com/Atrac/Atraci-Angular

If you will help me fix it, ill give you a big donation :) (from the donations ive got)

fent commented 9 years ago

I see you're using youtube-dl on it, do you have changes where you're using ytdl?

avihaymenahem commented 9 years ago

sorry, ill commit the code i used with ytdl, (youtub-dl needs python as dependency), 10 minutes ill commit it :)

avihaymenahem commented 9 years ago

@fent i know what the problem is!! Vevo is currently restricted on israel! And so on all the channels who blocked their content to some areas. dont we bypass these restrictions?

fent commented 9 years ago

Hmm, no ytdl doesn't bypass it. If I can find some video that is blocked for my area, I can test if it's bypassable without a proxy.

avihaymenahem commented 9 years ago

There is one small problem :) Almost any content is available on us, try proxy from europe somewhere and download bbc videos or VEVO

arajajyothibabu commented 7 years ago

I still have this issue. Let me explain briefly, my node server actually in US where as I'm getting links from server and downloading using those links like ......googlevideo.com/.. from India etc.. Only few videos are downloading that too with name videoplayback and without any extension and remaining all are forbidden 403. Is there anything I'm missing here. I need a proper fix for this, since my project in production. Thank you.

fent commented 7 years ago

If I'm understanding right, you're downloading videos from India but getting the urls to download them from the US? And that gives you errors? If that's so, you'd need to download the videos from the server in the US too.

arajajyothibabu commented 7 years ago

Right now I'm doing what you said but it is expensive for my cloud bandwidth. Is is not possible to download using urls obtained from getInfo() in proper way?

fent commented 7 years ago

You can try changing the ip param of the video url to the one where you're downloading from.

arajajyothibabu commented 7 years ago

I tried that also. But the result is same 403. And some downloading videos are with name videoplayback and without any extension.

drepram commented 4 years ago

Hi, been 3 years but I had the same error today and I fixed it by generating another API key Google Developer Console

The error was:

{ Error: Request failed with status code 403
    at createError (/Users/acp/Works/Megana/ytscrape/node_modules/axios/lib/core/createError.js:16:15)
    at settle (/Users/acp/Works/Megana/ytscrape/node_modules/axios/lib/core/settle.js:18:12)
    at IncomingMessage.handleStreamEnd (/Users/acp/Works/Megana/ytscrape/node_modules/axios/lib/adapters/http.js:202:11)
    at IncomingMessage.emit (events.js:203:15)
    at endReadableNT (_stream_readable.js:1145:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
  config:
   { adapter: [Function: httpAdapter],
     transformRequest: { '0': [Function: transformRequest] },
     transformResponse: { '0': [Function: transformResponse] },
     timeout: 0,
     xsrfCookieName: 'XSRF-TOKEN',
     xsrfHeaderName: 'X-XSRF-TOKEN',
     maxContentLength: -1,
     validateStatus: [Function: validateStatus],
     headers:
      { Accept: 'application/json, text/plain, */*',
        'User-Agent': 'axios/0.18.1' },
     method: 'get',
     url:
      'https://www.googleapis.com/youtube/v3/search?q=APBN%20dan%20APBD%20dalam%20Pembangunan%20Ekonomi%20Kelas%2011&part=snippet&maxResults=12&key=AIzaSyBtKhNioZvkXRU-k-gXTOfn1oFSR4nCG48',
     data: undefined },
  request:
   ClientRequest {
     _events:
      [Object: null prototype] {
        socket: [Function],
        abort: [Function],
        aborted: [Function],
        error: [Function],
        timeout: [Function],
        prefinish: [Function: requestOnPrefinish] },
     _eventsCount: 6,
     _maxListeners: undefined,
     output: [],
     outputEncodings: [],
     outputCallbacks: [],
     outputSize: 0,
     writable: true,
     _last: true,
     chunkedEncoding: false,
     shouldKeepAlive: false,
     useChunkedEncodingByDefault: false,
     sendDate: false,
     _removedConnection: false,
     _removedContLen: false,
     _removedTE: false,
     _contentLength: 0,
     _hasBody: true,
     _trailer: '',
     finished: true,
     _headerSent: true,
     socket:
      TLSSocket {
        _tlsOptions: [Object],
        _secureEstablished: true,
        _securePending: false,
        _newSessionPending: false,
        _controlReleased: true,
        _SNICallback: null,
        servername: 'www.googleapis.com',
        alpnProtocol: false,
        authorized: true,
        authorizationError: null,
        encrypted: true,
        _events: [Object],
        _eventsCount: 8,
        connecting: false,
        _hadError: false,
        _handle: [TLSWrap],
        _parent: null,
        _host: 'www.googleapis.com',
        _readableState: [ReadableState],
        readable: true,
        _maxListeners: undefined,
        _writableState: [WritableState],
        writable: false,
        allowHalfOpen: false,
        _sockname: null,
        _pendingData: null,
        _pendingEncoding: '',
        server: undefined,
        _server: null,
        ssl: [TLSWrap],
        _requestCert: true,
        _rejectUnauthorized: true,
        parser: null,
        _httpMessage: [Circular],
        [Symbol(res)]: [TLSWrap],
        [Symbol(asyncId)]: 6,
        [Symbol(lastWriteQueueSize)]: 0,
        [Symbol(timeout)]: null,
        [Symbol(kBytesRead)]: 0,
        [Symbol(kBytesWritten)]: 0,
        [Symbol(connect-options)]: [Object] },
     connection:
      TLSSocket {
        _tlsOptions: [Object],
        _secureEstablished: true,
        _securePending: false,
        _newSessionPending: false,
        _controlReleased: true,
        _SNICallback: null,
        servername: 'www.googleapis.com',
        alpnProtocol: false,
        authorized: true,
        authorizationError: null,
        encrypted: true,
        _events: [Object],
        _eventsCount: 8,
        connecting: false,
        _hadError: false,
        _handle: [TLSWrap],
        _parent: null,
        _host: 'www.googleapis.com',
        _readableState: [ReadableState],
        readable: true,
        _maxListeners: undefined,
        _writableState: [WritableState],
        writable: false,
        allowHalfOpen: false,
        _sockname: null,
        _pendingData: null,
        _pendingEncoding: '',
        server: undefined,
        _server: null,
        ssl: [TLSWrap],
        _requestCert: true,
        _rejectUnauthorized: true,
        parser: null,
        _httpMessage: [Circular],
        [Symbol(res)]: [TLSWrap],
        [Symbol(asyncId)]: 6,
        [Symbol(lastWriteQueueSize)]: 0,
        [Symbol(timeout)]: null,
        [Symbol(kBytesRead)]: 0,
        [Symbol(kBytesWritten)]: 0,
        [Symbol(connect-options)]: [Object] },
     _header:
      'GET /youtube/v3/search?q=APBN%20dan%20APBD%20dalam%20Pembangunan%20Ekonomi%20Kelas%2011&part=snippet&maxResults=12&key=AIzaSyBtKhNioZvkXRU-k-gXTOfn1oFSR4nCG48 HTTP/1.1\r\nAccept: application/json, text/plain, */*\r\nUser-Agent: axios/0.18.1\r\nHost: www.googleapis.com\r\nConnection: close\r\n\r\n',
     _onPendingData: [Function: noopPendingOutput],
     agent:
      Agent {
        _events: [Object],
        _eventsCount: 1,
        _maxListeners: undefined,
        defaultPort: 443,
        protocol: 'https:',
        options: [Object],
        requests: {},
        sockets: [Object],
        freeSockets: {},
        keepAliveMsecs: 1000,
        keepAlive: false,
        maxSockets: Infinity,
        maxFreeSockets: 256,
        maxCachedSessions: 100,
        _sessionCache: [Object] },
     socketPath: undefined,
     timeout: undefined,
     method: 'GET',
     path:
      '/youtube/v3/search?q=APBN%20dan%20APBD%20dalam%20Pembangunan%20Ekonomi%20Kelas%2011&part=snippet&maxResults=12&key=AIzaSyBtKhNioZvkXRU-k-gXTOfn1oFSR4nCG48',
     _ended: true,
     res:
      IncomingMessage {
        _readableState: [ReadableState],
        readable: false,
        _events: [Object],
        _eventsCount: 3,
        _maxListeners: undefined,
        socket: [TLSSocket],
        connection: [TLSSocket],
        httpVersionMajor: 1,
        httpVersionMinor: 1,
        httpVersion: '1.1',
        complete: true,
        headers: [Object],
        rawHeaders: [Array],
        trailers: {},
        rawTrailers: [],
        aborted: false,
        upgrade: false,
        url: '',
        method: null,
        statusCode: 403,
        statusMessage: 'Forbidden',
        client: [TLSSocket],
        _consuming: true,
        _dumped: false,
        req: [Circular],
        responseUrl:
         'https://www.googleapis.com/youtube/v3/search?q=APBN%20dan%20APBD%20dalam%20Pembangunan%20Ekonomi%20Kelas%2011&part=snippet&maxResults=12&key=AIzaSyBtKhNioZvkXRU-k-gXTOfn1oFSR4nCG48',
        redirects: [] },
     aborted: undefined,
     timeoutCb: null,
     upgradeOrConnect: false,
     parser: null,
     maxHeadersCount: null,
     _redirectable:
      Writable {
        _writableState: [WritableState],
        writable: true,
        _events: [Object],
        _eventsCount: 2,
        _maxListeners: undefined,
        _options: [Object],
        _redirectCount: 0,
        _redirects: [],
        _requestBodyLength: 0,
        _requestBodyBuffers: [],
        _onNativeResponse: [Function],
        _currentRequest: [Circular],
        _currentUrl:
         'https://www.googleapis.com/youtube/v3/search?q=APBN%20dan%20APBD%20dalam%20Pembangunan%20Ekonomi%20Kelas%2011&part=snippet&maxResults=12&key=AIzaSyBtKhNioZvkXRU-k-gXTOfn1oFSR4nCG48' },
     [Symbol(isCorked)]: false,
     [Symbol(outHeadersKey)]:
      [Object: null prototype] { accept: [Array], 'user-agent': [Array], host: [Array] } },
  response:
   { status: 403,
     statusText: 'Forbidden',
     headers:
      { vary: 'X-Origin, Origin,Accept-Encoding',
        'content-type': 'application/json; charset=UTF-8',
        date: 'Thu, 04 Jun 2020 10:14:51 GMT',
        expires: 'Thu, 04 Jun 2020 10:14:51 GMT',
        'cache-control': 'private, max-age=0',
        'x-content-type-options': 'nosniff',
        'x-frame-options': 'SAMEORIGIN',
        'content-security-policy': 'frame-ancestors \'self\'',
        'x-xss-protection': '1; mode=block',
        server: 'GSE',
        'accept-ranges': 'none',
        'transfer-encoding': 'chunked',
        'alt-svc':
         'h3-27=":443"; ma=2592000,h3-25=":443"; ma=2592000,h3-T050=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"',
        connection: 'close' },
     config:
      { adapter: [Function: httpAdapter],
        transformRequest: [Object],
        transformResponse: [Object],
        timeout: 0,
        xsrfCookieName: 'XSRF-TOKEN',
        xsrfHeaderName: 'X-XSRF-TOKEN',
        maxContentLength: -1,
        validateStatus: [Function: validateStatus],
        headers: [Object],
        method: 'get',
        url:
         'https://www.googleapis.com/youtube/v3/search?q=APBN%20dan%20APBD%20dalam%20Pembangunan%20Ekonomi%20Kelas%2011&part=snippet&maxResults=12&key=AIzaSyBtKhNioZvkXRU-k-gXTOfn1oFSR4nCG48',
        data: undefined },
     request:
      ClientRequest {
        _events: [Object],
        _eventsCount: 6,
        _maxListeners: undefined,
        output: [],
        outputEncodings: [],
        outputCallbacks: [],
        outputSize: 0,
        writable: true,
        _last: true,
        chunkedEncoding: false,
        shouldKeepAlive: false,
        useChunkedEncodingByDefault: false,
        sendDate: false,
        _removedConnection: false,
        _removedContLen: false,
        _removedTE: false,
        _contentLength: 0,
        _hasBody: true,
        _trailer: '',
        finished: true,
        _headerSent: true,
        socket: [TLSSocket],
        connection: [TLSSocket],
        _header:
         'GET /youtube/v3/search?q=APBN%20dan%20APBD%20dalam%20Pembangunan%20Ekonomi%20Kelas%2011&part=snippet&maxResults=12&key=AIzaSyBtKhNioZvkXRU-k-gXTOfn1oFSR4nCG48 HTTP/1.1\r\nAccept: application/json, text/plain, */*\r\nUser-Agent: axios/0.18.1\r\nHost: www.googleapis.com\r\nConnection: close\r\n\r\n',
        _onPendingData: [Function: noopPendingOutput],
        agent: [Agent],
        socketPath: undefined,
        timeout: undefined,
        method: 'GET',
        path:
         '/youtube/v3/search?q=APBN%20dan%20APBD%20dalam%20Pembangunan%20Ekonomi%20Kelas%2011&part=snippet&maxResults=12&key=AIzaSyBtKhNioZvkXRU-k-gXTOfn1oFSR4nCG48',
        _ended: true,
        res: [IncomingMessage],
        aborted: undefined,
        timeoutCb: null,
        upgradeOrConnect: false,
        parser: null,
        maxHeadersCount: null,
        _redirectable: [Writable],
        [Symbol(isCorked)]: false,
        [Symbol(outHeadersKey)]: [Object] },
     data: { error: [Object] } } }