Download stopped in the middle of the progress due to wrong condition

[Ziad-Tawfik]

Hello, Using colab notebook the download stopped with the below error, noting that it stopped on aws practitioner quiz page

I think the error is due to the condition in models.py lessons_names = topic.find_all("span", class_="ld-topic-title")

as this page doesn't contain any videos however it has the same class .learndash-wrapper .ld-table-list .ld-table-list-items div.ld-table-list-item a.ld-table-list-item-preview span.ld-topic-title.video_topic_enabled:before { content: '\eef7'; border: 1px solid; font-size: 13px; }

[info] 840335603: Downloading 1 format(s): hls-google_mediacdn_sep-279+dash-google_mediacdn_sep-audio-7e21444a
Deleting existing file /content/drive/MyDrive/KodeKloud/AWS Cloud Practitioner Certification/5 - Technology – Part Two/7 - Core AWS Services – Migration and Transfer 1634.en-x-autogen.vtt
[info] Writing video subtitles to: /content/drive/MyDrive/KodeKloud/AWS Cloud Practitioner Certification/5 - Technology – Part Two/7 - Core AWS Services – Migration and Transfer 1634.en-x-autogen.vtt
[download] Destination: /content/drive/MyDrive/KodeKloud/AWS Cloud Practitioner Certification/5 - Technology – Part Two/7 - Core AWS Services – Migration and Transfer 1634.en-x-autogen.vtt
[download] 100% of   23.88KiB in 00:00:00 at 142.25KiB/s
[download] /content/drive/MyDrive/KodeKloud/AWS Cloud Practitioner Certification/5 - Technology – Part Two/7 - Core AWS Services – Migration and Transfer 1634.mkv has already been downloaded
2023-09-25 14:19:01,809 - INFO - Writing video file... /content/drive/MyDrive/KodeKloud/AWS Cloud Practitioner Certification/5 - Technology – Part Two/8 - AWS Migration and Transfer – Demo 0504...
2023-09-25 14:19:01,810 - INFO - Parsing url: https://kodekloud.com/topic/aws-migration-and-transfer-demo/
[generic] Extracting URL: https://kodekloud.com/topic/aws-migration-and-transfer-demo/
[generic] aws-migration-and-transfer-demo: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] aws-migration-and-transfer-demo: Extracting information
[vimeo] Extracting URL: https://player.vimeo.com/video/840431488?dnt=1&app_id=122963&controls=1&autoplay=1&muted=1&api=1#...fer-demo%2F%22%7D%7D
[vimeo] 840431488: Downloading webpage
[vimeo] 840431488: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 840431488: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 840431488: Downloading fastly_skyfire m3u8 information
[vimeo] 840431488: Downloading fastly_skyfire m3u8 information
[vimeo] 840431488: Downloading google_mediacdn m3u8 information
[vimeo] 840431488: Downloading google_mediacdn m3u8 information
[vimeo] 840431488: Downloading akfire_interconnect_quic MPD information
[vimeo] 840431488: Downloading akfire_interconnect_quic MPD information
[vimeo] 840431488: Downloading fastly_skyfire MPD information
[vimeo] 840431488: Downloading fastly_skyfire MPD information
[vimeo] 840431488: Downloading google_mediacdn MPD information
[vimeo] 840431488: Downloading google_mediacdn MPD information
[info] 840431488: Downloading subtitles: en-x-autogen
[info] 840431488: Downloading 1 format(s): hls-google_mediacdn_sep-355+dash-google_mediacdn_sep-audio-2d2d19af
Deleting existing file /content/drive/MyDrive/KodeKloud/AWS Cloud Practitioner Certification/5 - Technology – Part Two/8 - AWS Migration and Transfer – Demo 0504.en-x-autogen.vtt
[info] Writing video subtitles to: /content/drive/MyDrive/KodeKloud/AWS Cloud Practitioner Certification/5 - Technology – Part Two/8 - AWS Migration and Transfer – Demo 0504.en-x-autogen.vtt
[download] Destination: /content/drive/MyDrive/KodeKloud/AWS Cloud Practitioner Certification/5 - Technology – Part Two/8 - AWS Migration and Transfer – Demo 0504.en-x-autogen.vtt
[download] 100% of    7.60KiB in 00:00:00 at 42.47KiB/s
[download] /content/drive/MyDrive/KodeKloud/AWS Cloud Practitioner Certification/5 - Technology – Part Two/8 - AWS Migration and Transfer – Demo 0504.mkv has already been downloaded
Your cookie might have expired or you don't have access to the course.
Please refresh/regenerate the cookie or enroll in the course and try again.

[AliFarajzade]

I get the same error with different IPs, cookies and accounts. Some specific courses will result an error (like "AWS Cloud Practitioner Certification")

[debakarr]

I am kindof busy with work but saw some issue are reported. The above one looks like is because the course have duplicate videos:

The current cookie check logic is written in such a way that if there are any two videos with same URL in the entire course then it fails.

I can make some changes to handle this for now.

[debakarr]

Please try out using pip install -U git+https://github.com/debakarr/kodekloud-downloader.git

[Ziad-Tawfik]

I commented out this part of logic so the bot is working without failure. raise SystemExit( "Your cookie might have expired or you don't have access to the course." "\nPlease refresh/regenerate the cookie or enroll in the course and try again." )

however the main problem was in scraping the data as the value of below two variables (main_lesson__content & topics) don't include the correct values for videos when using soup.find and zip function

main_lesson_content = soup.find("div", class_="lessons_main__content") or soup.find("div", class_="ld-lesson-list") topics = main_lesson_content.find_all("div", class_="w-dyn-item") or main_lesson_content.find_all( "div", class_="ld-item-list-items")

I investigated and printed both of them found that scraped part of the "billing and pricing" topic is zipped again with urls of the previous part which is "Technology - Part Two", so this raises an error however the above fix will just create a folder of "billing and pricing" topic but downloads all the videos in the "Technology - Part Two" again