search

Gaarv / kadenze-dl

Small application to download Kadenze (https://www.kadenze.com) videos for courses you enrolled in
MIT License
50 stars 16 forks source link

Fails to download after 50 attempts #23

Closed nazikus closed 2 years ago

nazikus commented 2 years ago

Environment:

$ DEBUG=pw:api python kadenze-dl.py
Signing in www.kadenze.com ...
  pw:api   navigated to "about:blank" +0ms
  pw:api   navigated to "about:blank" +23ms
  pw:api navigating to "https://www.kadenze.com", waiting until "load" +2ms
  pw:api   navigated to "https://www.kadenze.com/" +1s
  pw:api   "domcontentloaded" event fired +1s
  pw:api   navigated to "about:blank" +281ms
  pw:api   navigated to "about:blank" +0ms
  pw:api   navigated to "https://01206b9fc98f43749e421adfcf839755.pages.ubembed.com/78fbe0a3-711a-4278-9ef8-3e1236a8855a/a.html?closedAt=0" +485ms
  pw:api   navigated to "https://01206b9fc98f43749e421adfcf839755.pages.ubembed.com/8beffd51-1295-4c9c-b06b-4773c3c8b50d/a.html?closedAt=0" +1ms
  pw:api   navigated to "https://player.vimeo.com/video/112505592?api=1" +1ms
  pw:api   "domcontentloaded" event fired +167ms
  pw:api   "load" event fired +280ms
  pw:api waiting for selector "#email-login-btn" +4ms
  pw:api   selector resolved to visible <a href="javascript:;" id="email-login-btn" class="…>…</a> +32ms
  pw:api attempting click action +9ms
  pw:api   waiting for element to be visible, enabled and stable +0ms
  pw:api   navigated to "https://cdn.iubenda.com/cookie_solution/iframe_bridge.html?origin=https%3A%2F%2Fwww.kadenze.com%2F&meth=%22get%22&c_name=%22_iub_cs-56288926%22" +30ms
  pw:api   "domcontentloaded" event fired +1ms
  pw:api   "load" event fired +1ms
  pw:api   element is visible, enabled and stable +8ms
  pw:api   scrolling into view if needed +0ms
  pw:api   done scrolling +1ms
  pw:api   checking that element receives pointer events at (1004.83,270.59) +2ms
  pw:api   <iframe class="ub-emb-iframe" src="//01206b9fc98f43749e4…></iframe> from <div class="ub-emb-container">…</div> subtree intercepts pointer events +2ms
  pw:api retrying click action, attempt #1 +0ms
  pw:api   waiting for element to be visible, enabled and stable +0ms
  pw:api   element is visible, enabled and stable +28ms
  pw:api   scrolling into view if needed +0ms
  pw:api   done scrolling +1ms
  pw:api   checking that element receives pointer events at (1097.83,270.59) +1ms
  pw:api   <div class="ub-emb-scroll-wrapper">…</div> from <div class="ub-emb-container">…</div> subtree intercepts pointer events +10ms
  pw:api retrying click action, attempt #2 +0ms
  pw:api   waiting 20ms +0ms
  pw:api   waiting for element to be visible, enabled and stable +25ms
  pw:api   element is visible, enabled and stable +30ms
  pw:api   scrolling into view if needed +0ms
  pw:api   done scrolling +11ms
  pw:api   checking that element receives pointer events at (1004.83,270.59) +4ms
  pw:api   <iframe class="ub-emb-iframe" src="//01206b9fc98f43749e4…></iframe> from <div class="ub-emb-container">…</div> subtree intercepts pointer events +13ms
  pw:api retrying click action, attempt #3 +1ms
  pw:api   waiting 100ms +0ms
...

goes on with 50 attemps and fails. Tried with several different courses.

Gaarv commented 2 years ago

Hey, thanks for reporting the issue.

It seems theres currently a popup banner for Black Friday, which prevent the scraper to operate as usual.

image

Will try to find a workaround, but it might goes away by itlseft before that (December 6th according to banner)

Gaarv commented 2 years ago

It was pretty easy actually, I updated the code give it a try :)

nazikus commented 2 years ago

worked like a charm, thanks for the prompt response.