search

Puyodead1 / udemy-downloader

A Udemy downloader that can download courses, with DRM support.
MIT License
1.26k stars 292 forks source link

[Bug]: Skip download some lectures in long courses #129

Closed mossman07 closed 1 year ago

mossman07 commented 2 years ago

What happened?

I detected a problem on the download process of the last lectures of long courses (courses with 500 lectures for example). The downloader continue downloading the captions adn the assets but not the videos.

[05:11:01] [udemy-downloader] [handle_segments:1091] INFO: > Decryption complete
[05:11:01] [udemy-downloader] [handle_segments:1092] INFO: > Merging video and audio, this might take a minute...
[05:11:02] [udemy-downloader] [handle_segments:1097] INFO: > Merging complete, removing temporary files...
[05:11:02] [udemy-downloader] [parse_new:1293] INFO:   > Processing lecture 405 of 490
[05:11:02] [udemy-downloader] [process_lecture:1221] INFO:       > Lecture '015 [Windows]​ Local Environment Setup for React Development' has DRM, attempting to download
--- Logging error ---
Traceback (most recent call last):
  File "C:\Users\Pau\AppData\Local\Programs\Python\Python310\lib\logging\__init__.py", line 1101, in emit
    stream.write(msg + self.terminator)
  File "C:\Users\Pau\AppData\Local\Programs\Python\Python310\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u200b' in position 89: character maps to <undefined>
Call stack:
  File "I:\repositorios\test\udemy-downloader-master-2\main.py", line 1696, in <module>
    main()
  File "I:\repositorios\test\udemy-downloader-master-2\main.py", line 1689, in main
    parse_new(_udemy)
  File "I:\repositorios\test\udemy-downloader-master-2\main.py", line 1312, in parse_new
    process_lecture(lecture, lecture_path, lecture_file_name, chapter_dir)
  File "I:\repositorios\test\udemy-downloader-master-2\main.py", line 1221, in process_lecture
    logger.info(f"      > Lecture '%s' has DRM, attempting to download" % lecture_title)
Message: "      > Lecture '015 [Windows]\u200b Local Environment Setup for React Development' has DRM, attempting to download"
Arguments: ()
[05:11:02] [udemy-downloader] [handle_segments:1034] INFO: > Downloading Lecture Tracks...
WARNING: You have asked for UNPLAYABLE formats to be listed/downloaded. This is a developer option intended for debugging.
         If you experience any issues while using this option, DO NOT open a bug report
[generic] index.mpd?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJwYXRoIjoib3V0L3YxLzMxNzVmZTFhYzE2NjQ4MDFiOTE4M2UyODA4N2UwNTE2LzkxMjQyYTcyYzBmYTQwM2Y4YWNjZDcwMWE4OWM3MGEzLzJmNWViNjRkOTc3YjRkN2M4ZGIxZGI4MTIyNDFjM2M5LyIsImV4cCI6MTY1ODYzMjI0MX0: Downloading webpage
ERROR: [generic] Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U

Expected Result

Download all the videos, captions and assets of all the course.

[04:59:10] [udemy-downloader] [handle_segments:1059] INFO: > Lecture Tracks Downloaded
[04:59:10] [udemy-downloader] [handle_segments:1067] INFO: KID for video file is: 5A5E35364607449DACC841513A230224
[04:59:10] [udemy-downloader] [handle_segments:1074] INFO: KID for audio file is: 5A5E35364607449DACC841513A230224
[04:59:10] [udemy-downloader] [handle_segments:1080] INFO: > Decrypting video, this might take a minute...
[0724/045910:INFO:demuxer.cc(89)] Demuxer::Run() on file '013 Finishing Up the App - Letting Users Submit Secrets.encrypted.mp4'.
[0724/045910:INFO:demuxer.cc(155)] Initialize Demuxer for file '013 Finishing Up the App - Letting Users Submit Secrets.encrypted.mp4'.
[0724/045911:INFO:single_segment_segmenter.cc(111)] Update media header (moov) and rewrite the file to '013 Finishing Up the App - Letting Users Submit Secrets.decrypted.mp4'.
[0724/045911:INFO:mp4_muxer.cc(186)] MP4 file '013 Finishing Up the App - Letting Users Submit Secrets.decrypted.mp4' finalized.
Packaging completed successfully.
[04:59:11] [udemy-downloader] [handle_segments:1085] INFO: > Decryption complete
[04:59:11] [udemy-downloader] [handle_segments:1086] INFO: > Decrypting audio, this might take a minute...
[0724/045911:INFO:demuxer.cc(89)] Demuxer::Run() on file '013 Finishing Up the App - Letting Users Submit Secrets.encrypted.m4a'.
[0724/045911:INFO:demuxer.cc(155)] Initialize Demuxer for file '013 Finishing Up the App - Letting Users Submit Secrets.encrypted.m4a'.
[0724/045911:INFO:single_segment_segmenter.cc(111)] Update media header (moov) and rewrite the file to '013 Finishing Up the App - Letting Users Submit Secrets.decrypted.m4a'.
[0724/045911:INFO:mp4_muxer.cc(186)] MP4 file '013 Finishing Up the App - Letting Users Submit Secrets.decrypted.m4a' finalized.
Packaging completed successfully.
[04:59:11] [udemy-downloader] [handle_segments:1091] INFO: > Decryption complete
[04:59:11] [udemy-downloader] [handle_segments:1092] INFO: > Merging video and audio, this might take a minute...
[04:59:12] [udemy-downloader] [handle_segments:1097] INFO: > Merging complete, removing temporary files...

Branch

master/main

What operating systems are you seeing the problem on?

Windows

Relevant log output

[05:11:01] [udemy-downloader] [handle_segments:1091] INFO: > Decryption complete
[05:11:01] [udemy-downloader] [handle_segments:1092] INFO: > Merging video and audio, this might take a minute...
[05:11:02] [udemy-downloader] [handle_segments:1097] INFO: > Merging complete, removing temporary files...
[05:11:02] [udemy-downloader] [parse_new:1293] INFO:   > Processing lecture 405 of 490
[05:11:02] [udemy-downloader] [process_lecture:1221] INFO:       > Lecture '015 [Windows]​ Local Environment Setup for React Development' has DRM, attempting to download
--- Logging error ---
Traceback (most recent call last):
  File "C:\Users\Pau\AppData\Local\Programs\Python\Python310\lib\logging\__init__.py", line 1101, in emit
    stream.write(msg + self.terminator)
  File "C:\Users\Pau\AppData\Local\Programs\Python\Python310\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u200b' in position 89: character maps to <undefined>
Call stack:
  File "I:\repositorios\test\udemy-downloader-master-2\main.py", line 1696, in <module>
    main()
  File "I:\repositorios\test\udemy-downloader-master-2\main.py", line 1689, in main
    parse_new(_udemy)
  File "I:\repositorios\test\udemy-downloader-master-2\main.py", line 1312, in parse_new
    process_lecture(lecture, lecture_path, lecture_file_name, chapter_dir)
  File "I:\repositorios\test\udemy-downloader-master-2\main.py", line 1221, in process_lecture
    logger.info(f"      > Lecture '%s' has DRM, attempting to download" % lecture_title)
Message: "      > Lecture '015 [Windows]\u200b Local Environment Setup for React Development' has DRM, attempting to download"
Arguments: ()
[05:11:02] [udemy-downloader] [handle_segments:1034] INFO: > Downloading Lecture Tracks...
WARNING: You have asked for UNPLAYABLE formats to be listed/downloaded. This is a developer option intended for debugging.
         If you experience any issues while using this option, DO NOT open a bug report
[generic] index.mpd?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJwYXRoIjoib3V0L3YxLzMxNzVmZTFhYzE2NjQ4MDFiOTE4M2UyODA4N2UwNTE2LzkxMjQyYTcyYzBmYTQwM2Y4YWNjZDcwMWE4OWM3MGEzLzJmNWViNjRkOTc3YjRkN2M4ZGIxZGI4MTIyNDFjM2M5LyIsImV4cCI6MTY1ODYzMjI0MX0: Downloading webpage
ERROR: [generic] Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U

Other information

No response

mossman07 commented 2 years ago

Any idea about how to fix the problem? Thank you

ogeeDeveloper commented 2 years ago

Let the downloader uses the course id instead of the course name

mossman07 commented 2 years ago

What do you mean with course id?

I use the course ID, this is the command that I use to download the courses: python main.py -c https://www.udemy.com/course/ios-13-app-development-bootcamp/learn/lecture/16606448?start=15#overview --download-assets --download-captions --disable-ipv6 -l en -l es --log-level INFO

Puyodead1 commented 2 years ago

What do you mean with course id?

I use the course ID, this is the command that I use to download the courses:

python main.py -c https://www.udemy.com/course/ios-13-app-development-bootcamp/learn/lecture/16606448?start=15#overview --download-assets --download-captions --disable-ipv6 -l en -l es --log-level INFO

That's not using the course ID. --id-as-course-name

mossman07 commented 2 years ago

Thank you Puyodead! I will try it.