[Bug]: Patreon Private RSS Podcast Feed - Unable to Download Episodes

mahuika88 commented 1 year ago

Describe the issue

I've added a private RSS feed from Patreon to my ABS library using the following RSS link (redacted): https://www.patreon.com/rss/SkepticsGuide?auth=g_xxxxx

The podcast is successfully added to the library with the correct cover art. I'm able to see all published episodes using the "Find Episodes" button:

I select an episode to download and click "Download 1 Episode". The "Started downloading episodes" banner appears momentarily.

However, the episode never gets added to the Episodes list of the podcast. I do notice that it gets added to the "Library files" section of the podcast, but with a file size of 0 MB:

Looking at the logs I'm seeing 2 errors:

ERROR[PodcastManager] Podcast Episode download failed AxiosError: Request failed with status code 400

and

2023-03-11 18:59:51 ERROR[MediaFileScanner] Invalid argument : "/audiobooks/podcasts/SGU/The Skeptics Guide #922 - Mar 11 2023 (Ad Free).mp3"

Troubleshooting steps I've attempted:

Delete podcast and re-add
Delete podcast and delete all folders/files from filesystem and re-add
Re-added the podcast and performed "Match" to identify additional metadata (and replacing the RSS link with my custom link)
Check for malformed RSS link in the metadata abs file. (Was called out in this semi-related issue)
Double checked I'm running the latest version of ABS (v2.2.16)
Successfully downloaded multiple episodes from this RSS link with another podcast app
Successfully downloaded multiple episodes for non-private RSS podcasts
Checked filesystem permissions and ensured they matched permissions within other podcasts' folders
Ran automated scan for podcast episodes using "Check & Download New Episodes" button from within the "Episodes" tab of podcast settings
Re-added podcast but changed file path to exclude spaces/special characters. i.e. from "The Skeptics' Guide" to "SGU"

Metatdata ABS file:

;ABMETADATA2
#audiobookshelf v2.2.16

media=podcast
tags=[]
title=The Skeptics' Guide to the Universe
author=Dr. Steven Novella
language=en-US
genres=Science, Podcasts
feedUrl=https://www.patreon.com/rss/SkepticsGuide?auth=g_xxxx
itunesId=128859062
explicit=N

[DESCRIPTION]
The Skeptics' Guide To The Universe's Private RSS Feed for Ian Mahuika - Patreon

Relevant DEBUG logs:

2023-03-11 18:59:38 DEBUG[podcastUtils] getPodcastFeed for "https://www.patreon.com/rss/SkepticsGuide?auth=g_xxxxx"

2023-03-11 18:59:39 DEBUG[podcastUtils] getPodcastFeed for "https://www.patreon.com/rss/SkepticsGuide?auth=g_xxxxx" success - parsing xml

2023-03-11 18:59:44 DEBUG[Watcher] Ignoring directory "/audiobooks/podcasts/SGU"

2023-03-11 18:59:44 DEBUG[fileUtils] Downloading file to /audiobooks/podcasts/SGU/The Skeptics Guide #921 - Mar 04 2023 (Ad Free).mp3

2023-03-11 18:59:44 ERROR[PodcastManager] Podcast Episode download failed AxiosError: Request failed with status code 400

2023-03-11 18:59:44 DEBUG[Watcher] No longer ignoring directory "/audiobooks/podcasts/SGU"

2023-03-11 18:59:47 DEBUG[Watcher] File Added /audiobooks/podcasts/SGU/The Skeptics Guide #921 - Mar 04 2023 (Ad Free).mp3

2023-03-11 18:59:47 DEBUG[Watcher] Modified file in library "Main" and folder "audiobooks" with relPath "/podcasts/SGU/The Skeptics Guide #921 - Mar 04 2023 (Ad Free).mp3"

2023-03-11 18:59:47 DEBUG[Watcher] File Added /audiobooks/podcasts/SGU/The Skeptics Guide #921 - Mar 04 2023 (Ad Free).mp3

2023-03-11 18:59:51

INFO[Server] 1 Files Changed

2023-03-11 18:59:51 DEBUG[Scanner] Scanning file update groups in folder "audiobooks" of library "Main"

2023-03-11 18:59:51 DEBUG[Scanner] scanFolderUpdates fileUpdateGrou[object Object]

2023-03-11 18:59:51 DEBUG[Scanner] Folder update for relative path "podcasts/SGU" is in library item "The Skeptics' Guide to the Universe" - scan for updates

2023-03-11 18:59:51 DEBUG[fileUtils] Ignoring path has . ".nfs0000000002a402d700000005"

2023-03-11 18:59:51 WARN[LibraryItem] Check scan item changed folder fol_jmwoqsd92hi0ep580i -> audiobooks

2023-03-11 18:59:51 ERROR[MediaFileScanner] Invalid argument : "/audiobooks/podcasts/SGU/The Skeptics Guide #922 - Mar 11 2023 (Ad Free).mp3"

2023-03-11 18:59:51 ERROR[MediaFileScanner] Invalid argument : "/audiobooks/podcasts/SGU/The Skeptics Guide #921 - Mar 04 2023 (Ad Free).mp3"

Steps to reproduce the issue

Add new podcast using private RSS link from Patreon
Click on "Find Episodes"
Select at least one episode to download
Click "Download 1 Episode" button

Audiobookshelf version

2.2.16

How are you running audiobookshelf?

Docker

advplyr commented 1 year ago

Is the episode file actually downloaded and in your folder?

mahuika88 commented 1 year ago

There is an episode file in the folder but it shows a size of 0MB.

mahuika88 commented 1 year ago

Oops, didn't mean to close the issue

mahuika88 commented 1 year ago

I ran some more testing and I was able to replicate the issue by running ABS from source and invoking the download podcast episode API.

My API call:

curl -X POST "http://localhost:3333/api/podcasts/li_uaue82f3jrqgoimwse/download-episodes"   -H "Authorization: Bearer <token>"   -H "Content-Type: application/json"   -d '[{"episodeType": "full", "title": "The Skeptics Guide #922 - Mar 11 2023 (Ad Free)", "subtitle": "...", "description":
  "", "enclosure": {"url":"https://c10.patreonusercontent.com/4/patreon-media/p/post/79879050/2a68262934bb4556972bea256199d7e8/eyJhIjoxLCJwIjoxfQ%3D%3D/1.mp3?token-time=1679270400&token-hash=<token>","length":"47270686","type":"audio/mpeg"}}]'

stdout from npm provided more details (I modified some console.log messages in utils/fileUtils.js to print the url and download path):

[fileUtils] Downloading file to /home/ian/audiobookshelf/SGU/The Skeptics Guide #922 - Mar 11 2023 (Ad Free).mp3
[fileUtils] url is https://c10.patreonusercontent.com/4/patreon-media/p/post/79879050/2a68262934bb4556972bea256199d7e8/eyJhIjoxLCJwIjoxfQ%253D%253D/1.mp3?token-time=1679270400&token-hash=xxxxxx
[2023-03-11 19:01:56] ERROR: [PodcastManager] Podcast Episode download failed [AxiosError: Request failed with status code 400] {
  code: 'ERR_BAD_REQUEST',
  config: {
    transitional: {
      silentJSONParsing: true,
      forcedJSONParsing: true,
      clarifyTimeoutError: false
    },
    adapter: [Function: httpAdapter],
    transformRequest: [ [Function: transformRequest] ],
    transformResponse: [ [Function: transformResponse] ],
    timeout: 30000,
    xsrfCookieName: 'XSRF-TOKEN',
    xsrfHeaderName: 'X-XSRF-TOKEN',
    maxContentLength: -1,
    maxBodyLength: -1,
    env: { FormData: [Function] },
    validateStatus: [Function: validateStatus],
    headers: {
      Accept: 'application/json, text/plain, */*',
      'User-Agent': 'axios/0.27.2'
    },
    url: 'https://c10.patreonusercontent.com/4/patreon-media/p/post/79879050/2a68262934bb4556972bea256199d7e8/eyJhIjoxLCJwIjoxfQ%253D%253D/1.mp3?token-time=1679270400&token-hash=xxxxxx',
    method: 'get',
    responseType: 'stream',
    data: undefined
  },
  request: <ref *1> ClientRequest {
    _events: [Object: null prototype] {
      abort: [Function (anonymous)],
      aborted: [Function (anonymous)],
      connect: [Function (anonymous)],
      error: [Function (anonymous)],
      socket: [Function (anonymous)],
      timeout: [Function (anonymous)],
      finish: [Function: requestOnFinish]
    },
    _eventsCount: 7,
    _maxListeners: undefined,
    outputData: [],
    outputSize: 0,
    writable: true,
    destroyed: false,
    _last: true,
    chunkedEncoding: false,
    shouldKeepAlive: false,
    maxRequestsOnConnectionReached: false,
    _defaultKeepAlive: true,
    useChunkedEncodingByDefault: false,
    sendDate: false,
    _removedConnection: false,
    _removedContLen: false,
    _removedTE: false,
    strictContentLength: false,
    _contentLength: 0,
    _hasBody: true,
    _trailer: '',
    finished: true,
    _headerSent: true,
    _closed: false,
    socket: TLSSocket {
      _tlsOptions: [Object],
      _secureEstablished: true,
      _securePending: false,
      _newSessionPending: false,
      _controlReleased: true,
      secureConnecting: false,
      _SNICallback: null,
      servername: 'c10.patreonusercontent.com',
      alpnProtocol: false,
      authorized: true,
      authorizationError: null,
      encrypted: true,
      _events: [Object: null prototype],
      _eventsCount: 9,
      connecting: false,
      _hadError: false,
      _parent: null,
      _host: 'c10.patreonusercontent.com',
      _closeAfterHandlingError: false,
      _readableState: [ReadableState],
      _maxListeners: undefined,
      _writableState: [WritableState],
      allowHalfOpen: false,
      _sockname: null,
      _pendingData: null,
      _pendingEncoding: '',
      server: undefined,
      _server: null,
      ssl: [TLSWrap],
      _requestCert: true,
      _rejectUnauthorized: true,
      parser: null,
      _httpMessage: [Circular *1],
      timeout: 30000,
      [Symbol(res)]: [TLSWrap],
      [Symbol(verified)]: true,
      [Symbol(pendingSession)]: null,
      [Symbol(async_id_symbol)]: 354,
      [Symbol(kHandle)]: [TLSWrap],
      [Symbol(lastWriteQueueSize)]: 0,
      [Symbol(timeout)]: Timeout {
        _idleTimeout: 30000,
        _idlePrev: [TimersList],
        _idleNext: [TimersList],
        _idleStart: 8635,
        _onTimeout: [Function: bound ],
        _timerArgs: undefined,
        _repeat: null,
        _destroyed: false,
        [Symbol(refed)]: false,
        [Symbol(kHasPrimitive)]: false,
        [Symbol(asyncId)]: 364,
        [Symbol(triggerId)]: 358
      },
      [Symbol(kBuffer)]: null,
      [Symbol(kBufferCb)]: null,
      [Symbol(kBufferGen)]: null,
      [Symbol(kCapture)]: false,
      [Symbol(kSetNoDelay)]: false,
      [Symbol(kSetKeepAlive)]: true,
      [Symbol(kSetKeepAliveInitialDelay)]: 60,
      [Symbol(kBytesRead)]: 0,
      [Symbol(kBytesWritten)]: 0,
      [Symbol(connect-options)]: [Object],
      [Symbol(RequestTimeout)]: undefined
    },
    _header: 'GET /4/patreon-media/p/post/79879050/2a68262934bb4556972bea256199d7e8/eyJhIjoxLCJwIjoxfQ%253D%253D/1.mp3?token-time=1679270400&token-hash=xxxxxx HTTP/1.1\r\n' +
      'Accept: application/json, text/plain, */*\r\n' +
      'User-Agent: axios/0.27.2\r\n' +
      'Host: c10.patreonusercontent.com\r\n' +
      'Connection: close\r\n' +
      '\r\n',
    _keepAliveTimeout: 0,
    _onPendingData: [Function: nop],
    agent: Agent {
      _events: [Object: null prototype],
      _eventsCount: 2,
      _maxListeners: undefined,
      defaultPort: 443,
      protocol: 'https:',
      options: [Object: null prototype],
      requests: [Object: null prototype] {},
      sockets: [Object: null prototype],
      freeSockets: [Object: null prototype] {},
      keepAliveMsecs: 1000,
      keepAlive: false,
      maxSockets: Infinity,
      maxFreeSockets: 256,
      scheduling: 'lifo',
      maxTotalSockets: Infinity,
      totalSocketCount: 1,
      maxCachedSessions: 100,
      _sessionCache: [Object],
      [Symbol(kCapture)]: false
    },
    socketPath: undefined,
    method: 'GET',
    maxHeaderSize: undefined,
    insecureHTTPParser: undefined,
    path: '/4/patreon-media/p/post/79879050/2a68262934bb4556972bea256199d7e8/eyJhIjoxLCJwIjoxfQ%253D%253D/1.mp3?token-time=1679270400&token-hash=xxxxxx',
    _ended: false,
    res: IncomingMessage {
      _readableState: [ReadableState],
      _events: [Object: null prototype],
      _eventsCount: 1,
      _maxListeners: undefined,
      socket: [TLSSocket],
      httpVersionMajor: 1,
      httpVersionMinor: 1,
      httpVersion: '1.1',
      complete: true,
      rawHeaders: [Array],
      rawTrailers: [],
      aborted: false,
      upgrade: false,
      url: '',
      method: null,
      statusCode: 400,
      statusMessage: 'Bad Request',
      client: [TLSSocket],
      _consuming: false,
      _dumped: false,
      req: [Circular *1],
      responseUrl: 'https://c10.patreonusercontent.com/4/patreon-media/p/post/79879050/2a68262934bb4556972bea256199d7e8/eyJhIjoxLCJwIjoxfQ%253D%253D/1.mp3?token-time=1679270400&token-hash=xxxxxx',
      redirects: [],
      [Symbol(kCapture)]: false,
      [Symbol(kHeaders)]: [Object],
      [Symbol(kHeadersCount)]: 18,
      [Symbol(kTrailers)]: null,
      [Symbol(kTrailersCount)]: 0,
      [Symbol(RequestTimeout)]: undefined
    },
    aborted: false,
    timeoutCb: null,
    upgradeOrConnect: false,
    parser: null,
    maxHeadersCount: null,
    reusedSocket: false,
    host: 'c10.patreonusercontent.com',
    protocol: 'https:',
    _redirectable: Writable {
      _writableState: [WritableState],
      _events: [Object: null prototype],
      _eventsCount: 3,
      _maxListeners: undefined,
      _options: [Object],
      _ended: true,
      _ending: true,
      _redirectCount: 0,
      _redirects: [],
      _requestBodyLength: 0,
      _requestBodyBuffers: [],
      _onNativeResponse: [Function (anonymous)],
      _currentRequest: [Circular *1],
      _currentUrl: 'https://c10.patreonusercontent.com/4/patreon-media/p/post/79879050/2a68262934bb4556972bea256199d7e8/eyJhIjoxLCJwIjoxfQ%253D%253D/1.mp3?token-time=1679270400&token-hash=xxxxxx',
      _timeout: null,
      [Symbol(kCapture)]: false
    },
    [Symbol(kCapture)]: false,
    [Symbol(kBytesWritten)]: 0,
    [Symbol(kEndCalled)]: true,
    [Symbol(kNeedDrain)]: false,
    [Symbol(corked)]: 0,
    [Symbol(kOutHeaders)]: [Object: null prototype] {
      accept: [Array],
      'user-agent': [Array],
      host: [Array]
    },
    [Symbol(kUniqueHeaders)]: null
  },
  response: {
    status: 400,
    statusText: 'Bad Request',
    headers: {
      date: 'Sun, 12 Mar 2023 03:01:56 GMT',
      'content-type': 'text/plain;charset=UTF-8',
      'content-length': '13',
      connection: 'close',
      'set-cookie': [Array],
      'report-to': '{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report\\/v3?s=HvZ9B8z63QiE3hkfAAAbHj5X4mgWe%2FBWqoQBoyDkD0EsJKUrmr1SvKkq3ZnlT8lkC9CDqQtZrGg6kHjs8oBAPo8Y6jV6%2BVDRx4L3Nan4YsrCAA8iag8yXx%2Fl4GO9msLDtsGsCp5KviF2Yxqr"}],"group":"cf-nel","max_age":604800}',
      nel: '{"success_fraction":0,"report_to":"cf-nel","max_age":604800}',
      server: 'cloudflare',
      'cf-ray': '7a68cba11e49681d-SEA'
    },
    config: {
      transitional: [Object],
      adapter: [Function: httpAdapter],
      transformRequest: [Array],
      transformResponse: [Array],
      timeout: 30000,
      xsrfCookieName: 'XSRF-TOKEN',
      xsrfHeaderName: 'X-XSRF-TOKEN',
      maxContentLength: -1,
      maxBodyLength: -1,
      env: [Object],
      validateStatus: [Function: validateStatus],
      headers: [Object],
      url: 'https://c10.patreonusercontent.com/4/patreon-media/p/post/79879050/2a68262934bb4556972bea256199d7e8/eyJhIjoxLCJwIjoxfQ%253D%253D/1.mp3?token-time=1679270400&token-hash=xxxxxx',
      method: 'get',
      responseType: 'stream',
      data: undefined
    },
    request: <ref *1> ClientRequest {
      _events: [Object: null prototype],
      _eventsCount: 7,
      _maxListeners: undefined,
      outputData: [],
      outputSize: 0,
      writable: true,
      destroyed: false,
      _last: true,
      chunkedEncoding: false,
      shouldKeepAlive: false,
      maxRequestsOnConnectionReached: false,
      _defaultKeepAlive: true,
      useChunkedEncodingByDefault: false,
      sendDate: false,
      _removedConnection: false,
      _removedContLen: false,
      _removedTE: false,
      strictContentLength: false,
      _contentLength: 0,
      _hasBody: true,
      _trailer: '',
      finished: true,
      _headerSent: true,
      _closed: false,
      socket: [TLSSocket],
      _header: 'GET /4/patreon-media/p/post/79879050/2a68262934bb4556972bea256199d7e8/eyJhIjoxLCJwIjoxfQ%253D%253D/1.mp3?token-time=1679270400&token-hash=xxxxxx HTTP/1.1\r\n' +
        'Accept: application/json, text/plain, */*\r\n' +
        'User-Agent: axios/0.27.2\r\n' +
        'Host: c10.patreonusercontent.com\r\n' +
        'Connection: close\r\n' +
        '\r\n',
      _keepAliveTimeout: 0,
      _onPendingData: [Function: nop],
      agent: [Agent],
      socketPath: undefined,
      method: 'GET',
      maxHeaderSize: undefined,
      insecureHTTPParser: undefined,
      path: '/4/patreon-media/p/post/79879050/2a68262934bb4556972bea256199d7e8/eyJhIjoxLCJwIjoxfQ%253D%253D/1.mp3?token-time=1679270400&token-hash=xxxxxx',
      _ended: false,
      res: [IncomingMessage],
      aborted: false,
      timeoutCb: null,
      upgradeOrConnect: false,
      parser: null,
      maxHeadersCount: null,
      reusedSocket: false,
      host: 'c10.patreonusercontent.com',
      protocol: 'https:',
      _redirectable: [Writable],
      [Symbol(kCapture)]: false,
      [Symbol(kBytesWritten)]: 0,
      [Symbol(kEndCalled)]: true,
      [Symbol(kNeedDrain)]: false,
      [Symbol(corked)]: 0,
      [Symbol(kOutHeaders)]: [Object: null prototype],
      [Symbol(kUniqueHeaders)]: null
    },
    data: IncomingMessage {
      _readableState: [ReadableState],
      _events: [Object: null prototype],
      _eventsCount: 1,
      _maxListeners: undefined,
      socket: [TLSSocket],
      httpVersionMajor: 1,
      httpVersionMinor: 1,
      httpVersion: '1.1',
      complete: true,
      rawHeaders: [Array],
      rawTrailers: [],
      aborted: false,
      upgrade: false,
      url: '',
      method: null,
      statusCode: 400,
      statusMessage: 'Bad Request',
      client: [TLSSocket],
      _consuming: false,
      _dumped: false,
      req: [ClientRequest],
      responseUrl: 'https://c10.patreonusercontent.com/4/patreon-media/p/post/79879050/2a68262934bb4556972bea256199d7e8/eyJhIjoxLCJwIjoxfQ%253D%253D/1.mp3?token-time=1679270400&token-hash=xxxxxx',
      redirects: [],
      [Symbol(kCapture)]: false,
      [Symbol(kHeaders)]: [Object],
      [Symbol(kHeadersCount)]: 18,
      [Symbol(kTrailers)]: null,
      [Symbol(kTrailersCount)]: 0,
      [Symbol(RequestTimeout)]: undefined
    }
  }
} (PodcastManager.js:97)

advplyr commented 1 year ago

Have you tried taking that URL and pasting in your browser to see if it is valid?

mahuika88 commented 1 year ago

Yes. I've tried pasting the URL directly into Firefox and it downloads the mp3 file. Also attempted with cURL and Insomnia and the request was successful in both cases.

advplyr commented 1 year ago

I found the problem here. Had to due with https://github.com/advplyr/audiobookshelf/pull/1515

Patreon already gives us URLs that are encoded, so I added a check before encoding.

advplyr commented 1 year ago

Fixed in v2.2.17

advplyr / audiobookshelf