search

prashantghimire / sofifa-web-scraper

JavaScript scrapper for over 18k detailed players info and stats from EA FC 25 from SoFIFA.com.
33 stars 5 forks source link

There is a bug in the web process #20

Open loa123 opened 3 months ago

loa123 commented 3 months ago

When I ran it for the second time, the following error occurred

`

sofifa-web-scraper@1.0.0 full node main.js full

running full scan.

/www/wwwroot/sofifa-web-scraper/node_modules/request-promise-core/lib/errors.js:14
Error.captureStackTrace(this);
^
RequestError: AggregateError
at new RequestError (/www/wwwroot/sofifa-web-scraper/node_modules/request-promise-core/lib/errors.js:14:15)
at plumbing.callback (/www/wwwroot/sofifa-web-scraper/node_modules/request-promise-core/lib/plumbing.js:87:29)
at Request.RP$callback [as _callback] (/www/wwwroot/sofifa-web-scraper/node_modules/request-promise-core/lib/plumbing.js:46:31)
at self.callback (/www/wwwroot/sofifa-web-scraper/node_modules/request/request.js:185:22)
at Request.emit (node:events:519:28)
at Request.onRequestError (/www/wwwroot/sofifa-web-scraper/node_modules/request/request.js:877:8)
at ClientRequest.emit (node:events:519:28)
at TLSSocket.socketErrorListener (node:_http_client:500:9)
at TLSSocket.emit (node:events:519:28)
at emitErrorNT (node:internal/streams/destroy:169:8) {
cause: AggregateError [ETIMEDOUT]:
at internalConnectMultiple (node:net:1117:18)
at internalConnectMultiple (node:net:1185:5)
at Timeout.internalConnectMultipleTimeout (node:net:1711:5)
at listOnTimeout (node:internal/timers:575:11)
at process.processTimers (node:internal/timers:514:7) {
code: 'ETIMEDOUT',
[errors]: [
Error: connect ETIMEDOUT 172.67.38.45:443
at createConnectionError (node:net:1647:14)
at Timeout.internalConnectMultipleTimeout (node:net:1706:38)
at listOnTimeout (node:internal/timers:575:11)
at process.processTimers (node:internal/timers:514:7) {
errno: -110,
code: 'ETIMEDOUT',
syscall: 'connect',
address: '172.67.38.45',
port: 443
},
Error: connect ENETUNREACH 2606:4700:10::ac43:262d:443 - Local (:::0)
at internalConnectMultiple (node:net:1181:16)
at Timeout.internalConnectMultipleTimeout (node:net:1711:5)
at listOnTimeout (node:internal/timers:575:11)
at process.processTimers (node:internal/timers:514:7) {
errno: -101,
code: 'ENETUNREACH',
syscall: 'connect',
address: '2606:4700:10::ac43:262d',
port: 443
},
Error: connect ETIMEDOUT 104.22.31.196:443
at createConnectionError (node:net:1647:14)
at Timeout.internalConnectMultipleTimeout (node:net:1706:38)
at listOnTimeout (node:internal/timers:575:11)
at process.processTimers (node:internal/timers:514:7) {
errno: -110,
code: 'ETIMEDOUT',
syscall: 'connect',
address: '104.22.31.196',
port: 443
},
Error: connect ENETUNREACH 2606:4700:10::6816:1ec4:443 - Local (:::0)
at internalConnectMultiple (node:net:1181:16)
at Timeout.internalConnectMultipleTimeout (node:net:1711:5)
at listOnTimeout (node:internal/timers:575:11)
at process.processTimers (node:internal/timers:514:7) {
errno: -101,
code: 'ENETUNREACH',
syscall: 'connect',
address: '2606:4700:10::6816:1ec4',
port: 443
},
Error: connect ETIMEDOUT 104.22.30.196:443
at createConnectionError (node:net:1647:14)
at Timeout.internalConnectMultipleTimeout (node:net:1706:38)
at listOnTimeout (node:internal/timers:575:11)
at process.processTimers (node:internal/timers:514:7) {
errno: -110,
code: 'ETIMEDOUT',
syscall: 'connect',
address: '104.22.30.196',
port: 443
},
Error: connect ENETUNREACH 2606:4700:10::6816:1fc4:443 - Local (:::0)
at internalConnectMultiple (node:net:1181:16)
at Timeout.internalConnectMultipleTimeout (node:net:1711:5)
at listOnTimeout (node:internal/timers:575:11)
at process.processTimers (node:internal/timers:514:7) {
errno: -101,
code: 'ENETUNREACH',
syscall: 'connect',
address: '2606:4700:10::6816:1fc4',
port: 443
}
]
},
error: AggregateError [ETIMEDOUT]:
at internalConnectMultiple (node:net:1117:18)
at internalConnectMultiple (node:net:1185:5)
at Timeout.internalConnectMultipleTimeout (node:net:1711:5)
at listOnTimeout (node:internal/timers:575:11)
at process.processTimers (node:internal/timers:514:7) {
code: 'ETIMEDOUT',
[errors]: [
Error: connect ETIMEDOUT 172.67.38.45:443
at createConnectionError (node:net:1647:14)
at Timeout.internalConnectMultipleTimeout (node:net:1706:38)
at listOnTimeout (node:internal/timers:575:11)
at process.processTimers (node:internal/timers:514:7) {
errno: -110,
code: 'ETIMEDOUT',
syscall: 'connect',
address: '172.67.38.45',
port: 443
},
Error: connect ENETUNREACH 2606:4700:10::ac43:262d:443 - Local (:::0)
at internalConnectMultiple (node:net:1181:16)
at Timeout.internalConnectMultipleTimeout (node:net:1711:5)
at listOnTimeout (node:internal/timers:575:11)
at process.processTimers (node:internal/timers:514:7) {
errno: -101,
code: 'ENETUNREACH',
syscall: 'connect',
address: '2606:4700:10::ac43:262d',
port: 443
},
Error: connect ETIMEDOUT 104.22.31.196:443
at createConnectionError (node:net:1647:14)
at Timeout.internalConnectMultipleTimeout (node:net:1706:38)
at listOnTimeout (node:internal/timers:575:11)
at process.processTimers (node:internal/timers:514:7) {
errno: -110,
code: 'ETIMEDOUT',
syscall: 'connect',
address: '104.22.31.196',
port: 443
},
Error: connect ENETUNREACH 2606:4700:10::6816:1ec4:443 - Local (:::0)
at internalConnectMultiple (node:net:1181:16)
at Timeout.internalConnectMultipleTimeout (node:net:1711:5)
at listOnTimeout (node:internal/timers:575:11)
at process.processTimers (node:internal/timers:514:7) {
errno: -101,
code: 'ENETUNREACH',
syscall: 'connect',
address: '2606:4700:10::6816:1ec4',
port: 443
},
Error: connect ETIMEDOUT 104.22.30.196:443
at createConnectionError (node:net:1647:14)
at Timeout.internalConnectMultipleTimeout (node:net:1706:38)
at listOnTimeout (node:internal/timers:575:11)
at process.processTimers (node:internal/timers:514:7) {
errno: -110,
code: 'ETIMEDOUT',
syscall: 'connect',
address: '104.22.30.196',
port: 443
},
Error: connect ENETUNREACH 2606:4700:10::6816:1fc4:443 - Local (:::0)
at internalConnectMultiple (node:net:1181:16)
at Timeout.internalConnectMultipleTimeout (node:net:1711:5)
at listOnTimeout (node:internal/timers:575:11)
at process.processTimers (node:internal/timers:514:7) {
errno: -101,
code: 'ENETUNREACH',
syscall: 'connect',
address: '2606:4700:10::6816:1fc4',
port: 443
}
]
},
options: {
resolveWithFullResponse: true,
jar: RequestJar {
_jar: CookieJar {
enableLooseMode: true,
store: MemoryCookieStore { idx: {} }
}
},
simple: false,
gzip: true,
encoding: null,
headers: {
Accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Encoding': 'gzip, deflate, br',
Connection: 'keep-alive',
Host: 'sofifa.com',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36'
},
method: 'GET',
uri: 'https://sofifa.com/player/239085/erling-haaland/240046/\r',
callback: [Function: RP$callback],
transform: undefined,
transform2xxOnly: false
},
response: undefined
}

Node.js v20.15.1
npm notice
npm notice New minor version of npm available! 10.7.0 -> 10.8.2
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.2
npm notice To update run: npm install -g npm@10.8.2
npm notice

`

prashantghimire commented 4 weeks ago

Try using node and npm versions mentioned in the readme file. Also, it seems like you're machine is not able to connect to sofifa.com, are you able to curl or connect from the browser?