Closed davidclaeysquinones closed 1 month ago
I use a channels file that looks similar to this
<?xml version="1.0" encoding="UTF-8"?> <channels> <channel site="pickx.be" lang="nl" xmltv_id="Een.be" site_id="UID50208">VRT 1 HD</channel> </channels>
I then use the following command
npm run grab -- --channels=channels.xml --maxConnections=10 --days=14 --gzip
Then I get the following error output
Error fetching API version: Request failed with status code 500 Error during script initialization: AxiosError: Request failed with status code 500 at settle (/usr/bin/epg/node_modules/axios/lib/core/settle.js:19:12) at IncomingMessage.handleStreamEnd (/usr/bin/epg/node_modules/axios/lib/adapters/http.js:589:11) at IncomingMessage.emit (node:events:531:35) at IncomingMessage.emit (node:domain:488:12) at endReadableNT (node:internal/streams/readable:1696:12) at processTicksAndRejections (node:internal/process/task_queues:82:21) at Axios.request (/usr/bin/epg/node_modules/axios/lib/core/Axios.js:45:41) at processTicksAndRejections (node:internal/process/task_queues:95:5) at /usr/bin/epg/sites/pickx.be/pickx.be.config.js:135:24 { code: 'ERR_BAD_RESPONSE', config: { transitional: { silentJSONParsing: true, forcedJSONParsing: true, clarifyTimeoutError: false }, adapter: [ 'xhr', 'http' ], transformRequest: [ [Function: transformRequest] ], transformResponse: [ [Function: transformResponse] ], timeout: 0, xsrfCookieName: 'XSRF-TOKEN', xsrfHeaderName: 'X-XSRF-TOKEN', maxContentLength: -1, maxBodyLength: -1, env: { FormData: [Function], Blob: [class Blob] }, validateStatus: [Function: validateStatus], headers: Object [AxiosHeaders] { Accept: 'application/json, text/plain, */*', 'Content-Type': undefined, Origin: 'https://www.pickx.be', Referer: 'https://www.pickx.be/', 'User-Agent': 'axios/1.6.8', 'Accept-Encoding': 'gzip, compress, deflate, br' }, method: 'get', url: 'https://px-epg.azureedge.net/version', data: undefined }, request: <ref *1> ClientRequest { _events: [Object: null prototype] { abort: [Function (anonymous)], aborted: [Function (anonymous)], connect: [Function (anonymous)], error: [Function (anonymous)], socket: [Function (anonymous)], timeout: [Function (anonymous)], finish: [Function: requestOnFinish] }, _eventsCount: 7, _maxListeners: undefined, outputData: [], outputSize: 0, writable: true, destroyed: true, _last: true, chunkedEncoding: false, shouldKeepAlive: true, maxRequestsOnConnectionReached: false, _defaultKeepAlive: true, useChunkedEncodingByDefault: false, sendDate: false, _removedConnection: false, _removedContLen: false, _removedTE: false, strictContentLength: false, _contentLength: 0, _hasBody: true, _trailer: '', finished: true, _headerSent: true, _closed: true, _header: 'GET /version HTTP/1.1\r\n' + 'Accept: application/json, text/plain, */*\r\n' + 'Origin: https://www.pickx.be\r\n' + 'Referer: https://www.pickx.be/\r\n' + 'User-Agent: axios/1.6.8\r\n' + 'Accept-Encoding: gzip, compress, deflate, br\r\n' + 'Host: px-epg.azureedge.net\r\n' + 'Connection: keep-alive\r\n' + '\r\n', _keepAliveTimeout: 0, _onPendingData: [Function: nop], agent: Agent { _events: [Object: null prototype], _eventsCount: 2, _maxListeners: undefined, defaultPort: 443, protocol: 'https:', options: [Object: null prototype], requests: [Object: null prototype] {}, sockets: [Object: null prototype] {}, freeSockets: [Object: null prototype], keepAliveMsecs: 1000, keepAlive: true, maxSockets: Infinity, maxFreeSockets: 256, scheduling: 'lifo', maxTotalSockets: Infinity, totalSocketCount: 1, maxCachedSessions: 100, _sessionCache: [Object], [Symbol(shapeMode)]: false, [Symbol(kCapture)]: false }, socketPath: undefined, method: 'GET', maxHeaderSize: undefined, insecureHTTPParser: undefined, joinDuplicateHeaders: undefined, path: '/version', _ended: true, res: IncomingMessage { _events: [Object], _readableState: [ReadableState], _maxListeners: undefined, socket: null, httpVersionMajor: 1, httpVersionMinor: 1, httpVersion: '1.1', complete: true, rawHeaders: [Array], rawTrailers: [], joinDuplicateHeaders: undefined, aborted: false, upgrade: false, url: '', method: null, statusCode: 500, statusMessage: 'Internal Server Error', client: [TLSSocket], _consuming: false, _dumped: false, req: [Circular *1], _eventsCount: 4, responseUrl: 'https://px-epg.azureedge.net/version', redirects: [], [Symbol(shapeMode)]: true, [Symbol(kCapture)]: false, [Symbol(kHeaders)]: [Object], [Symbol(kHeadersCount)]: 26, [Symbol(kTrailers)]: null, [Symbol(kTrailersCount)]: 0 }, aborted: false, timeoutCb: null, upgradeOrConnect: false, parser: null, maxHeadersCount: null, reusedSocket: false, host: 'px-epg.azureedge.net', protocol: 'https:', _redirectable: Writable { _events: [Object], _writableState: [WritableState], _maxListeners: undefined, _options: [Object], _ended: true, _ending: true, _redirectCount: 0, _redirects: [], _requestBodyLength: 0, _requestBodyBuffers: [], _eventsCount: 3, _onNativeResponse: [Function (anonymous)], _currentRequest: [Circular *1], _currentUrl: 'https://px-epg.azureedge.net/version', [Symbol(shapeMode)]: true, [Symbol(kCapture)]: false }, [Symbol(shapeMode)]: false, [Symbol(kCapture)]: false, [Symbol(kBytesWritten)]: 0, [Symbol(kNeedDrain)]: false, [Symbol(corked)]: 0, [Symbol(kChunkedBuffer)]: [], [Symbol(kChunkedLength)]: 0, [Symbol(kSocket)]: TLSSocket { _tlsOptions: [Object], _secureEstablished: true, _securePending: false, _newSessionPending: false, _controlReleased: true, secureConnecting: false, _SNICallback: null, servername: 'px-epg.azureedge.net', alpnProtocol: false, authorized: true, authorizationError: null, encrypted: true, _events: [Object: null prototype], _eventsCount: 9, connecting: false, _hadError: false, _parent: null, _host: 'px-epg.azureedge.net', _closeAfterHandlingError: false, _readableState: [ReadableState], _writableState: [WritableState], allowHalfOpen: false, _maxListeners: undefined, _sockname: null, _pendingData: null, _pendingEncoding: '', server: undefined, _server: null, ssl: [TLSWrap], _requestCert: true, _rejectUnauthorized: true, timeout: 5000, parser: null, _httpMessage: null, autoSelectFamilyAttemptedAddresses: [Array], [Symbol(alpncallback)]: null, [Symbol(res)]: [TLSWrap], [Symbol(verified)]: true, [Symbol(pendingSession)]: null, [Symbol(async_id_symbol)]: -1, [Symbol(kHandle)]: [TLSWrap], [Symbol(lastWriteQueueSize)]: 0, [Symbol(timeout)]: Timeout { _idleTimeout: 5000, _idlePrev: [TimersList], _idleNext: [TimersList], _idleStart: 1502, _onTimeout: [Function: bound ], _timerArgs: undefined, _repeat: null, _destroyed: false, [Symbol(refed)]: false, [Symbol(kHasPrimitive)]: false, [Symbol(asyncId)]: 269, [Symbol(triggerId)]: 267 }, [Symbol(kBuffer)]: null, [Symbol(kBufferCb)]: null, [Symbol(kBufferGen)]: null, [Symbol(shapeMode)]: true, [Symbol(kCapture)]: false, [Symbol(kSetNoDelay)]: false, [Symbol(kSetKeepAlive)]: true, [Symbol(kSetKeepAliveInitialDelay)]: 1, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0, [Symbol(connect-options)]: [Object] }, [Symbol(kOutHeaders)]: [Object: null prototype] { accept: [Array], origin: [Array], referer: [Array], 'user-agent': [Array], 'accept-encoding': [Array], host: [Array] }, [Symbol(errored)]: null, [Symbol(kHighWaterMark)]: 16384, [Symbol(kRejectNonStandardBodyWrites)]: false, [Symbol(kUniqueHeaders)]: null }, response: { status: 500, statusText: 'Internal Server Error', headers: Object [AxiosHeaders] { date: 'Tue, 07 May 2024 08:23:33 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '96', connection: 'keep-alive', 'cache-control': 'public, max-age=30, s-maxage=30', etag: 'W/"60-kz3eHA5kBYDXTM+FYlnhkhsYciQ"', expires: 'Tue, 07 May 2024 08:24:03 GMT', vary: 'Origin', 'request-context': 'appId=cid-v1:92c36e86-ad4d-4621-bb52-b2eb8c392b22', 'x-azure-ref': '20240507T082332Z-16d6d7b44b562s98y1fkkxf0vg00000004000000000170cm', 'x-fd-int-roxy-purgeid': '0', 'x-cache': 'TCP_MISS', 'access-control-allow-origin': '*' }, config: { transitional: [Object], adapter: [Array], transformRequest: [Array], transformResponse: [Array], timeout: 0, xsrfCookieName: 'XSRF-TOKEN', xsrfHeaderName: 'X-XSRF-TOKEN', maxContentLength: -1, maxBodyLength: -1, env: [Object], validateStatus: [Function: validateStatus], headers: [Object [AxiosHeaders]], method: 'get', url: 'https://px-epg.azureedge.net/version', data: undefined }, request: <ref *1> ClientRequest { _events: [Object: null prototype], _eventsCount: 7, _maxListeners: undefined, outputData: [], outputSize: 0, writable: true, destroyed: true, _last: true, chunkedEncoding: false, shouldKeepAlive: true, maxRequestsOnConnectionReached: false, _defaultKeepAlive: true, useChunkedEncodingByDefault: false, sendDate: false, _removedConnection: false, _removedContLen: false, _removedTE: false, strictContentLength: false, _contentLength: 0, _hasBody: true, _trailer: '', finished: true, _headerSent: true, _closed: true, _header: 'GET /version HTTP/1.1\r\n' + 'Accept: application/json, text/plain, */*\r\n' + 'Origin: https://www.pickx.be\r\n' + 'Referer: https://www.pickx.be/\r\n' + 'User-Agent: axios/1.6.8\r\n' + 'Accept-Encoding: gzip, compress, deflate, br\r\n' + 'Host: px-epg.azureedge.net\r\n' + 'Connection: keep-alive\r\n' + '\r\n', _keepAliveTimeout: 0, _onPendingData: [Function: nop], agent: [Agent], socketPath: undefined, method: 'GET', maxHeaderSize: undefined, insecureHTTPParser: undefined, joinDuplicateHeaders: undefined, path: '/version', _ended: true, res: [IncomingMessage], aborted: false, timeoutCb: null, upgradeOrConnect: false, parser: null, maxHeadersCount: null, reusedSocket: false, host: 'px-epg.azureedge.net', protocol: 'https:', _redirectable: [Writable], [Symbol(shapeMode)]: false, [Symbol(kCapture)]: false, [Symbol(kBytesWritten)]: 0, [Symbol(kNeedDrain)]: false, [Symbol(corked)]: 0, [Symbol(kChunkedBuffer)]: [], [Symbol(kChunkedLength)]: 0, [Symbol(kSocket)]: [TLSSocket], [Symbol(kOutHeaders)]: [Object: null prototype], [Symbol(errored)]: null, [Symbol(kHighWaterMark)]: 16384, [Symbol(kRejectNonStandardBodyWrites)]: false, [Symbol(kUniqueHeaders)]: null }, data: { message: 'Internal server error', originalMessage: 'This function is deprecated', status: 500 } } }
So instead of showing that the grab for the channel failed the whole process comes to an end. Meaning that when you have channels from other sources these won't be scraped.
The reason for this error is that they changed their version endpoint to 'https://www.pickx.be/api/s-3b36540f3cef64510112f3f95c2c0cdca321997ed2b1042ad778523235e155eb'. I guess they don't like people getting the api version and that's why they have obfuscated the endpoint.
Describe your issue
I use a channels file that looks similar to this
I then use the following command
npm run grab -- --channels=channels.xml --maxConnections=10 --days=14 --gzipThen I get the following error output
So instead of showing that the grab for the channel failed the whole process comes to an end. Meaning that when you have channels from other sources these won't be scraped.
The reason for this error is that they changed their version endpoint to 'https://www.pickx.be/api/s-3b36540f3cef64510112f3f95c2c0cdca321997ed2b1042ad778523235e155eb'. I guess they don't like people getting the api version and that's why they have obfuscated the endpoint.