[Bug]: downloading from certain tracking sites fails

BerendWouters commented 1 month ago

What happened?

I'm attempting to download an episode from the WAN Show, using the following link: https://pdst.fm/e/chrt.fm/track/C76EG6/mgln.ai/e/94/claritaspod.com/measure/verifi.podscribe.com/rss/p/chtbl.com/track/289382/pfx.vpixl.com/j0JIg/traffic.megaphone.fm/ROOSTER8312874402.mp3?updated=1722970170 This link is redirecting through multiple tracking websites, and ending up at the https://pfx.vpixl.com/j0JIg/traffic.megaphone.fm/ROOSTER8312874402.mp3?updated=1722970170 url, which seems to timeout. However, if I run this in a browser, it works just fine.

It also occurs with other episodes

What did you expect to happen?

The episode being downloaded

Steps to reproduce the issue

Search for the The WAN Show by Linus Tech Tips
Select an episode to download to the server
Click download episode

Audiobookshelf version

2.12.1

How are you running audiobookshelf?

Docker

What OS is your Audiobookshelf server hosted from?

Linux

If the issue is being seen in the UI, what browsers are you seeing the problem on?

None

Logs

[ffmpegHelpers] Failed to download podcast episode with url "https://pdst.fm/e/chrt.fm/track/C76EG6/mgln.ai/e/94/claritaspod.com/measure/verifi.podscribe.com/rss/p/chtbl.com/track/289382/pfx.vpixl.com/j0JIg/traffic.megaphone.fm/ROOSTER8312874402.mp3?updated=1722970170" AxiosError: timeout of 30000ms exceeded

extended logs from the container:

24-08-07 17:00:24.034] ERROR: [ffmpegHelpers] Failed to download podcast episode with url "https://pdst.fm/e/chrt.fm/track/C76EG6/mgln.ai/e/94/claritaspod.com/measure/verifi.podscribe.com/rss/p/chtbl.com/track/289382/pfx.vpixl.com/j0JIg/traffic.megaphone.fm/ROOSTER8312874402.mp3?updated=1722970170" [AxiosError: timeout of 30000ms exceeded] {
  code: 'ECONNABORTED',
  config: {
    transitional: {
      silentJSONParsing: true,
      forcedJSONParsing: true,
      clarifyTimeoutError: false
    },
    adapter: [Function: httpAdapter],
    transformRequest: [ [Function: transformRequest] ],
    transformResponse: [ [Function: transformResponse] ],
    timeout: 30000,
    xsrfCookieName: 'XSRF-TOKEN',
    xsrfHeaderName: 'X-XSRF-TOKEN',
    maxContentLength: -1,
    maxBodyLength: -1,
    env: { FormData: [Function] },
    validateStatus: [Function: validateStatus],
    headers: {
      Accept: 'application/json, text/plain, */*',
      'User-Agent': 'audiobookshelf (+https://audiobookshelf.org; like iTMS)'
    },
    url: 'https://pdst.fm/e/chrt.fm/track/C76EG6/mgln.ai/e/94/claritaspod.com/measure/verifi.podscribe.com/rss/p/chtbl.com/track/289382/pfx.vpixl.com/j0JIg/traffic.megaphone.fm/ROOSTER8312874402.mp3?updated=1722970170',
    method: 'get',
    responseType: 'stream',
    data: undefined
  },
  request: <ref *1> Writable {
    _events: {
      close: undefined,
      error: [Function: handleRequestError],
      prefinish: undefined,
      finish: undefined,
      drain: undefined,
      response: [Function: handleResponse],
      socket: [Array],
      timeout: undefined,
      abort: undefined
    },
    _writableState: WritableState {
      highWaterMark: 16384,
      length: 0,
      corked: 0,
      onwrite: [Function: bound onwrite],
      writelen: 0,
      bufferedIndex: 0,
      pendingcb: 0,
      [Symbol(kState)]: 17580812,
      [Symbol(kBufferedValue)]: null
    },
    _maxListeners: undefined,
    _options: {
      maxRedirects: 21,
      maxBodyLength: 10485760,
      protocol: 'https:',
      path: '/j0JIg/traffic.megaphone.fm/ROOSTER8312874402.mp3?updated=1722970170',
      method: 'GET',
      headers: [Object],
      agent: undefined,
      agents: [Object],
      auth: null,
      hostname: 'pfx.vpixl.com',
      port: 0,
      nativeProtocols: [Object],
      pathname: '/j0JIg/traffic.megaphone.fm/ROOSTER8312874402.mp3',
      search: '?updated=1722970170',
      host: 'pfx.vpixl.com',
      href: 'https://pfx.vpixl.com/j0JIg/traffic.megaphone.fm/ROOSTER8312874402.mp3?updated=1722970170',
      query: 'updated=1722970170',
      hash: null
    },
    _ended: true,
    _ending: true,
    _redirectCount: 6,
    _redirects: [],
    _requestBodyLength: 0,
    _requestBodyBuffers: [],
    _eventsCount: 3,
    _onNativeResponse: [Function (anonymous)],
    _currentRequest: ClientRequest {
      _events: [Object: null prototype],
      _eventsCount: 2,
      _maxListeners: undefined,
      outputData: [],
      outputSize: 0,
      writable: true,
      destroyed: true,
      _last: true,
      chunkedEncoding: false,
      shouldKeepAlive: true,
      maxRequestsOnConnectionReached: false,
      _defaultKeepAlive: true,
      useChunkedEncodingByDefault: false,
      sendDate: false,
      _removedConnection: false,
      _removedContLen: false,
      _removedTE: false,
      strictContentLength: false,
      _contentLength: 0,
      _hasBody: true,
      _trailer: '',
      finished: true,
      _headerSent: true,
      _closed: false,
      socket: [TLSSocket],
      _header: 'GET /j0JIg/traffic.megaphone.fm/ROOSTER8312874402.mp3?updated=1722970170 HTTP/1.1\r\n' +
        'Accept: application/json, text/plain, */*\r\n' +
        'User-Agent: audiobookshelf (+https://audiobookshelf.org; like iTMS)\r\n' +
        'Host: pfx.vpixl.com\r\n' +
        'Connection: keep-alive\r\n' +
        '\r\n',
      _keepAliveTimeout: 0,
      _onPendingData: [Function: nop],
      agent: [Agent],
      socketPath: undefined,
      method: 'GET',
      maxHeaderSize: undefined,
      insecureHTTPParser: undefined,
      joinDuplicateHeaders: undefined,
      path: '/j0JIg/traffic.megaphone.fm/ROOSTER8312874402.mp3?updated=1722970170',
      _ended: false,
      res: null,
      aborted: true,
      timeoutCb: [Function: emitRequestTimeout],
      upgradeOrConnect: false,
      parser: [HTTPParser],
      maxHeadersCount: null,
      reusedSocket: false,
      host: 'pfx.vpixl.com',
      protocol: 'https:',
      _redirectable: [Circular *1],
      [Symbol(shapeMode)]: false,
      [Symbol(kCapture)]: false,
      [Symbol(kBytesWritten)]: 0,
      [Symbol(kNeedDrain)]: false,
      [Symbol(corked)]: 0,
      [Symbol(kOutHeaders)]: [Object: null prototype],
      [Symbol(errored)]: null,
      [Symbol(kHighWaterMark)]: 16384,
      [Symbol(kRejectNonStandardBodyWrites)]: false,
      [Symbol(kUniqueHeaders)]: null,
      [Symbol(kError)]: undefined
    },
    _currentUrl: 'https://pfx.vpixl.com/j0JIg/traffic.megaphone.fm/ROOSTER8312874402.mp3?updated=1722970170',
    _timeout: null,
    _isRedirect: true,
    [Symbol(shapeMode)]: true,
    [Symbol(kCapture)]: false
  }
} (ffmpegHelpers.js:111)

Additional Notes

I've tried recreating the request in Postman, and I get a Maximum response size reached, but that might be related to Postman not being able to process the download.

nichwall commented 1 month ago

Does this issue still occur using the edge container? There was a fix for https://github.com/advplyr/audiobookshelf/issues/3246, not sure if it's the same thing.

BerendWouters commented 1 month ago

I just ran it with the latest tag locally, and it seems to be working. Which is odd, because it's the same digest as being used by the actual server running it. Going to try edge on the actual server now.

BerendWouters commented 1 month ago

Sadly enough, no changes running it on the edge version either. Might be a docker networking related issue, unsure. Downloading other episodes works just fine.

advplyr commented 1 month ago

I tested and I'm not able to reproduce the issue. Are you still able to reproduce it? If it is reproducible still, is there specific episodes it happens with? Any other podcasts?

BerendWouters commented 1 month ago

The issue still occurs with any episode of The WAN Show

advplyr commented 1 month ago

This is most likely a network issue since it is timing out. Having a 30 second timeout seems reasonable

BerendWouters commented 3 weeks ago

It was a networking issue indeed. My docker/daemon.json also had the ip address of the docker server itself, which the internal networking didn't like. It's working now.

advplyr / audiobookshelf