Open Elchava opened 4 years ago
Can you give me the url you try to download? Looks like the timeout problem caused by the eurosport backend, you also see that sometimes with the browser.
Hello,
Here is the url from source code
And the web url is
https://www.eurosportplayer.com/videos/eurosport-1-france/tour-de-france2014-5e-etape
Thanks !
Sorry the link doesn't work for me, maybe geo-blocking?
But i have a different video that behaves the same i think. The page displays an error message and removes the player, that makes the script fail because it cannot find the title/date. I have now a different method the get the title, but for some reason it doesn't work in headless mode, the json url returns a 204 - no content.
Got it working, please try the newest version.
I have a new error, at the firts step, with the last version (like 10 or 15 loops with the same error) :
Error: expected data object not found 'included'
at video (/usr/local/bin/download.js:222:13)
at async /usr/local/bin/download.js:265:20
looking for www.eurosportplayer.com
Error: expected data object not found 'included'
at video (/usr/local/bin/download.js:222:13)
at runNextTicks (internal/process/task_queues.js:62:5)
at listOnTimeout (internal/timers.js:518:9)
at processTimers (internal/timers.js:492:7)
at async /usr/local/bin/download.js:265:20
looking for www.eurosportplayer.com
Error: expected data object not found 'included'
at video (/usr/local/bin/download.js:222:13)
at async /usr/local/bin/download.js:265:20
looking for www.eurosportplayer.com
Error: expected data object not found 'included'
at video (/usr/local/bin/download.js:222:13)
at async /usr/local/bin/download.js:265:20
looking for www.eurosportplayer.com
Error: expected data object not found 'included'
at video (/usr/local/bin/download.js:222:13)
at async /usr/local/bin/download.js:265:20
looking for www.eurosportplayer.com
Error: expected data object not found 'included'
at video (/usr/local/bin/download.js:222:13)
at async /usr/local/bin/download.js:265:20
looking for www.eurosportplayer.com
Error: expected data object not found 'included'
at video (/usr/local/bin/download.js:222:13)
at runMicrotasks (
Added a fallback solution if the title is not found, the url will be used as filename. Give it a try.
Now I have this message before putting my password and the captcha and after 5 loops the script fails :
TimeoutError: Timeout exceeded while waiting for event
at Timeout.
Maybe timeout too short to put my ID and password ?
If you use the -a option with a existing profile you don't need the login again. Start with -d and --trace command line parameter, that gives you more debug output and a browser window.
edit: you need to update first, i just added trace
Thanks, here is the result :
args {
'login-timeout': 120000,
tmp: 'tmp',
out: './',
language: 'eng',
trace: true,
email: 'mail',
password: 'pass',
debug: true,
'user-data-dir': 'profile',
url: [ 'https://www.eurosportplayer.com' ]
}
testLoggin
goto https://auth.eurosportplayer.com/my-account
search "sign in" result -1
processing url https://www.eurosportplayer.com
video download https://www.eurosportplayer.com
looking for www.eurosportplayer.com
dataResp waitFor https://www.eurosportplayer.com/manifest.480eb9ea9b9eb064d2b06035c464f8b0.json
dataResp https://www.eurosportplayer.com/manifest.480eb9ea9b9eb064d2b06035c464f8b0.json
data {
icons: [
{
src: '/icon_512x512.604e2be3d212185553fba9b9413fd08e.png',
sizes: '512x512',
type: 'image/png'
},
{
src: '/icon_384x384.c0be9805ab5f96277b71df146406946c.png',
sizes: '384x384',
type: 'image/png'
},
{
src: '/icon_256x256.20031b64df8aac90c9437bc69a3b3880.png',
sizes: '256x256',
type: 'image/png'
},
{
src: '/icon_192x192.7e84a809c90c1eae56bc28d5d02eca34.png',
sizes: '192x192',
type: 'image/png'
},
{
src: '/icon_128x128.e3c5e7a1c1a15154169147e225ec53c9.png',
sizes: '128x128',
type: 'image/png'
},
{
src: '/icon_96x96.97e59ac52e12131ca720abf7e37f1422.png',
sizes: '96x96',
type: 'image/png'
},
{
src: '/icon_64x64.30a9d319d5735974d1aba73c8163568e.png',
sizes: '64x64',
type: 'image/png'
},
{
src: '/icon_32x32.8899f028ecd06d6805ea0c41991be771.png',
sizes: '32x32',
type: 'image/png'
},
{
src: '/icon_16x16.1856416143144af5fac164345037f4ec.png',
sizes: '16x16',
type: 'image/png'
}
],
name: 'Eurosport',
short_name: 'Eurosport',
display: 'standalone',
start_url: '.',
description: 'Eurosport an svod/avod streaming service',
theme_color: '#141b4d',
background_color: '#151726',
prefer_related_applications: true,
related_applications: [ { platform: 'play', id: 'com.eurosport.player' } ]
}
fallback title
waitFor m3u8
TimeoutError: Timeout exceeded while waiting for event
at Timeout.
Oh, need to filter out email/password command line options. But you are already logged in with that profile in tmp, try to download with:
node download.js -a tmp --trace <url to download>
With your last method, I obtain this result :
args { 'login-timeout': 30000, tmp: 'tmp', out: './', language: 'eng', trace: true, 'user-data-dir': 'tmp', url: [ 'https://www.eurosportplayer.com/videos/eurosport-2-france/cincinnati-1993-s-edberg-p-sampras' ] } testLoggin goto https://auth.eurosportplayer.com/my-account search "sign in" result 1126 login need email to login
And I have tried to put the timeout to 120000 with my email and password and here is the result:
args {
'login-timeout': 120000,
tmp: 'tmp',
out: './',
language: 'eng',
trace: true,
email: 'mail',
password: 'pass',
'user-data-dir': 'tmp',
url: [
'https://www.eurosportplayer.com/videos/eurosport-2-france/cincinnati-1993-s-edberg-p-sampras'
]
}
testLoggin
goto https://auth.eurosportplayer.com/my-account
search "sign in" result 1126
login
input email and password
click submit and wait with timeout 120000
unable to login TimeoutError: waiting for selector "button[class*="styles-authButton"]" failed: timeout 120000ms exceeded
at new WaitTask (/usr/local/bin/node_modules/puppeteer/lib/DOMWorld.js:549:28)
at DOMWorld._waitForSelectorOrXPath (/usr/local/bin/node_modules/puppeteer/lib/DOMWorld.js:478:22)
at DOMWorld.waitForSelector (/usr/local/bin/node_modules/puppeteer/lib/DOMWorld.js:432:17)
at Frame.waitForSelector (/usr/local/bin/node_modules/puppeteer/lib/FrameManager.js:627:47)
at Frame.
Sorry misread the trace, the chrome profile folder should be 'profile'. So the command has to be
node download.js -a profile --trace <url to download>
I also removed email/password from the trace.
Same error, if I don't put the email, I have : 'need email to login'
Otherwise, I get:
'login-timeout': 30000,
tmp: 'tmp',
out: './',
language: 'eng',
trace: true,
'user-data-dir': 'profile',
email: '',
password: '',
url: [
'https://www.eurosportplayer.com/videos/eurosport-2-france/cincinnati-1993-s-edberg-p-sampras'
]
}
testLoggin
goto https://auth.eurosportplayer.com/my-account
search "sign in" result 1127
login
input email and password
click submit and wait with timeout 30000
unable to login TimeoutError: waiting for selector "button[class*="styles-authButton"]" failed: timeout 30000ms exceeded
at new WaitTask (/usr/local/bin/node_modules/puppeteer/lib/DOMWorld.js:549:28)
at DOMWorld._waitForSelectorOrXPath (/usr/local/bin/node_modules/puppeteer/lib/DOMWorld.js:478:22)
at DOMWorld.waitForSelector (/usr/local/bin/node_modules/puppeteer/lib/DOMWorld.js:432:17)
at Frame.waitForSelector (/usr/local/bin/node_modules/puppeteer/lib/FrameManager.js:627:47)
at Frame.
I added 2 more command line options for debugging: --har saves the requests and responses --save-html saves the html pages You also need to run 'npm install' again, i added a library. Add both options to the last command, it will generate har and html files, email and password should be removed. But to be sure mail me these files and the output: m4eba.0@gmail.com
Hello,
Could you help me with this issue for the second step ?
Error: Protocol error (Page.navigate): Cannot navigate to invalid URL at /usr/local/bin/node_modules/puppeteer/lib/Connection.js:183:56 at new Promise ()
at CDPSession.send (/usr/local/bin/node_modules/puppeteer/lib/Connection.js:182:12)
at navigate (/usr/local/bin/node_modules/puppeteer/lib/FrameManager.js:118:39)
at FrameManager.navigateFrame (/usr/local/bin/node_modules/puppeteer/lib/FrameManager.js:95:7)
at Frame.goto (/usr/local/bin/node_modules/puppeteer/lib/FrameManager.js:406:37)
at Frame. (/usr/local/bin/node_modules/puppeteer/lib/helper.js:112:23)
at Page.goto (/usr/local/bin/node_modules/puppeteer/lib/Page.js:672:49)
at Page. (/usr/local/bin/node_modules/puppeteer/lib/helper.js:112:23)
at video (/usr/local/bin/download.js:210:14)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async /usr/local/bin/download.js:242:18
TypeError: Cannot read property 'm3u8' of null
at /usr/local/bin/download.js:249:18
at processTicksAndRejections (internal/process/task_queues.js:97:5)
Thanks a lot !