Incomplete download of syllabus: Error 400 Client Error

[felker]

Subject of the issue

With some of my Coursera courses, the syllabus is not downloaded completely when it is parsed. Some of the later lectures emit non-fatal error such as:

Error 400 Client Error: Bad Request for url: https://api.coursera.org/api/openCourseAssets.v1/v6wEuppCEeeN9RJghr4Fsg getting page https://api.coursera.org/api/openCourseAssets.v1/v6wEuppCEeeN9RJghr4Fsg
The server replied: Confused deputy attack detected.

resulting in a final summary error:

--------------------------------------------------------------------------------
The following classes had errors during the syllabus parsing stage. You may want to review error messages and courses (sometimes enrolling to the course or switching session helps):
neural-networks-deep-learning (https://www.coursera.org/learn/neural-networks-deep-learning)

and causing some content to be missing after the subsequent download stage.

The errors seem to be random and occur only for later lectures in the longer courses.

Possibly related to #702.

Your environment

• Operating System (name/version): macOS 10.15.2
• Python version: Python 3.7.3
• coursera-dl version: 0.11.5

Steps to reproduce

coursera-dl  --playlist -sl en --about --video-resolution 720p --download-quizzes --netrc --cauth XXX neural-networks-deep-learning

I am using the cookie workaround detailed in https://github.com/coursera-dl/coursera-dl/issues/702#issuecomment-568228372.

I think using --only-syllabus might avoid these errors, thus enabling a workaround with --cache-syllabus when downloading the actual content, but I havent tested this thoroughly.

Expected behaviour

The syllabus is (nearly?) always downloaded and parsed completely for the relatively short course machine-learning-projects; I would hope this would work 100% for the two courses I am having problems with in the same specialization, neural-networks-deep-learning and deep-neural-network

Actual behaviour

Click to expand!

``` Downloading class: machine-learning-projects (1 / 1) Parsing syllabus of on-demand course (id=-7ZymCvBEee6gA5XksfBbg). This may take some time, please be patient ... Processing module ml-strategy-1 Processing section introduction-to-ml-strategy Processing lecture why-ml-strategy (lecture) Processing lecture orthogonalization (lecture) Processing section setting-up-your-goal Processing lecture single-number-evaluation-metric (lecture) Processing lecture satisficing-and-optimizing-metric (lecture) Processing lecture train-dev-test-distributions (lecture) Processing lecture size-of-the-dev-and-test-sets (lecture) Processing lecture when-to-change-dev-test-sets-and-metrics (lecture) Processing section comparing-to-human-level-performance Processing lecture why-human-level-performance (lecture) Processing lecture avoidable-bias (lecture) Processing lecture understanding-human-level-performance (lecture) Processing lecture surpassing-human-level-performance (lecture) Processing lecture improving-your-model-performance (lecture) Processing section machine-learning-flight-simulator Processing lecture machine-learning-flight-simulator (supplement) Processing lecture bird-recognition-in-the-city-of-peacetopia-case-study (exam) Processing section heroes-of-deep-learning-optional Processing lecture andrej-karpathy-interview (lecture) Processing module ml-strategy-2 Processing section error-analysis Processing lecture carrying-out-error-analysis (lecture) Processing lecture cleaning-up-incorrectly-labeled-data (lecture) Processing lecture build-your-first-system-quickly-then-iterate (lecture) Processing section mismatched-training-and-dev-test-set Processing lecture training-and-testing-on-different-distributions (lecture) Processing lecture bias-and-variance-with-mismatched-data-distributions (lecture) Processing lecture addressing-data-mismatch (lecture) Processing section learning-from-multiple-tasks Processing lecture transfer-learning (lecture) Processing lecture multi-task-learning (lecture) Processing section end-to-end-deep-learning Processing lecture what-is-end-to-end-deep-learning (lecture) Processing lecture whether-to-use-end-to-end-deep-learning (lecture) Error 400 Client Error: Bad Request for url: https://api.coursera.org/api/openCourseAssets.v1/v6wEuppCEeeN9RJghr4Fsg getting page https://api.coursera.org/api/openCourseAssets.v1/v6wEuppCEeeN9RJghr4Fsg The server replied: Confused deputy attack detected. Could not download lecture H56eb: 400 Client Error: Bad Request for url: https://api.coursera.org/api/openCourseAssets.v1/v6wEuppCEeeN9RJghr4Fsg Processing section machine-learning-flight-simulator Processing lecture autonomous-driving-case-study (exam) Error 400 Client Error: Bad Request for url: https://api.coursera.org/api/onDemandExamSessions.v1 getting page https://api.coursera.org/api/onDemandExamSessions.v1 The server replied: Confused deputy attack detected. Could not download exam Cre8k: 400 Client Error: Bad Request for url: https://api.coursera.org/api/onDemandExamSessions.v1 Processing section heroes-of-deep-learning-optional Processing lecture ruslan-salakhutdinov-interview (lecture) Error 400 Client Error: Bad Request for url: https://api.coursera.org/api/onDemandLectureAssets.v1/-7ZymCvBEee6gA5XksfBbg~kR8gk/?includes=openCourseAssets getting page https://api.coursera.org/api/onDemandLectureAssets.v1/-7ZymCvBEee6gA5XksfBbg~kR8gk/?includes=openCourseAssets The server replied: Confused deputy attack detected. Could not download lecture kR8gk: 400 Client Error: Bad Request for url: https://api.coursera.org/api/onDemandLectureAssets.v1/-7ZymCvBEee6gA5XksfBbg~kR8gk/?includes=openCourseAssets ... The following classes had errors during the syllabus parsing stage. You may want to review error messages and courses (sometimes enrolling to the course or switching session helps): machine-learning-projects (https://www.coursera.org/learn/machine-learning-projects) ```

[priya-27-zz]

similar issue

[jainsiddharth21]

I'm too facing similar issue

[abdullahal25]

I had similar issue.. I logged out, signed in again and everything worked fine

[kmdupr33]

Similar issue. Tried logging out and back in. It didn't help. Also tried --syllabus-only and got the same error :(

[kmdupr33]

In case this helps anyone else: I actually had a warning with requests when I ran this command:

/Users/mattdupree/.pyenv/versions/3.6.3/lib/python3.6/site-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.25.8) or chardet (3.0.4) doesn't match a supported version!
  RequestsDependencyWarning) 

I fixed the warning with pip install requests and it worked!

[moctarjallo]

Same here.

Confused deputy attack detected.

[sagarkar10]

Same Here Please help someone!

[adamaiken89]

Any progress?

[uzam9048]

anything yet?

[jasoncbraatz]

I have found a useful workaround, though it's not a fix, but it gets the job done.

I re-run the command several times, as this error seems to be related to the cookie (#702) but as mentioned, it's random as to what works and what doesn't.

the easiest way to do this is to create a .sh script

vi cdl.sh a [to add]

``

!/bin/sh

coursera-dl --playlist -sl en --about --video-resolution 720p --download-quizzes --netrc --cauth XXX neural-networks-deep-learning cdl.sh `` [ctrl-c] and 'w' to write and exit.

NOTE that this will re-run over and over and over again (since this shell script doesn't stop, it just re-spawns itself). However, on my box it'll take about 3-4 passes before it appears that I get all of the files. Luckily since the error shows up different places it seems that by brute force of re-downloading everything, you'll eventually get it all.

[sasimac]

(jasoncbraatz) how to run this cdl.sh after create it .