Linkedin Learning is broken again, extractor error caused by KeyError('JSESSIONID')

j-rahman commented 5 years ago

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2019.07.16
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--cookies', 'cookies.txt', 'https://www.linkedin.com/learning/c-plus-plus-essential-training-2', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.07.16
[debug] Python version 3.7.3 (CPython) - Darwin-18.7.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.1.4, ffprobe 4.1.4, rtmpdump 2.4
[debug] Proxy map: {}
ERROR: An extractor error has occurred. (caused by KeyError('JSESSIONID')); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/linkedin.py", line 158, in _real_extract
    course_data = self._call_api(course_slug, 'chapters,description,title')
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/linkedin.py", line 36, in _call_api
    'Csrf-Token': self._get_cookies(api_url)['JSESSIONID'].value,
KeyError: 'JSESSIONID'
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/linkedin.py", line 158, in _real_extract
    course_data = self._call_api(course_slug, 'chapters,description,title')
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/linkedin.py", line 36, in _call_api
    'Csrf-Token': self._get_cookies(api_url)['JSESSIONID'].value,
KeyError: 'JSESSIONID'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 796, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 543, in extract
    raise ExtractorError('An extractor error has occurred.', cause=e)
youtube_dl.utils.ExtractorError: An extractor error has occurred. (caused by KeyError('JSESSIONID')); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description

I think Linkedin-Learning has changed its website design again, which is making youtube-dl unable to download any playlists from the website. I was trying the following among others.

youtube-dl --cookies cookies.txt https://www.linkedin.com/learning/c-plus-plus-essential-training-2

It fails for every course on Linkedin-Learning. I have an active subscription and I can view the videos online, but having downloaded the cookies after logging in, I wanted to have the course offline. Seems like youtube-dl fails to help me there.

rgalang81 commented 5 years ago

Still an issue with youtube-dl version 2019.08.13

abundiscarlos commented 5 years ago

> ---------------------------------------------------------------------------
> λ youtube-dl.exe -U
> Updating to version 2019.08.13 ...
> Waiting for file handle to be closed ...
> 
> λ Updated youtube-dl to version 2019.08.13.
> ---------------------------------------------------------------------------
> λ youtube-dl.exe --verbose --cookies .\cookies.txt https://www.linkedin.com/learning/after-effects-advanced-compositing-tracking-and-roto-techniques
> [debug] System config: []
> [debug] User config: []
> [debug] Custom config: []
> [debug] Command-line args: ['--verbose', '--cookies', '.\\cookies.txt', 'https://www.linkedin.com/learning/after-effects-advanced-compositing-tracking-and-roto-techniques']
> [debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
> [debug] youtube-dl version 2019.08.13
> [debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
> [debug] exe versions: ffmpeg N-93949-g9c35285aea, ffprobe N-93949-g9c35285aea
> [debug] Proxy map: {}
> [linkedin:learning:course] Downloading JSON metadata
> ERROR: Unable to download JSON metadata: HTTP Error 401: Unauthorized (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
>   File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp5b8vs72_\build\youtube_dl\extractor\common.py", line 627, in _request_webpage
>   File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp5b8vs72_\build\youtube_dl\YoutubeDL.py", line 2229, in urlopen
>   File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
>   File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
>   File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
>   File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
>   File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default
> ---------------------------------------------------------------------------

The path "C:\Users\dst\" is completely unknown to my environment. Thanks.!

joykingwar commented 5 years ago

version 2019.08.13 works for me, using firefox and This Cookie Addon to extract the cookie.txt file. Then use this code to download FULL COURSE videos well organized into specific chapters as subfolders into the Main folder: youtube-dl.exe --verbose --cookies .\cookies.txt "https://www.linkedin.com/learning/giving-your-elevator-pitch" -o "%(playlist)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s - %(title)s.%(ext)s" --write-sub

abbeyjackson-mc commented 5 years ago

@joykingwar using the cookies file downloaded from Chrome as suggested in the docs for youtube-dl I get the extractor error using your snippet (both for the course you have listed as well as any other course I try). Could you post the cookies file that your cookie add on uses? Remove any idenfying info. I am wanting to see the format so that we can compare. This is mine:

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This is a generated file!  Do not edit.

.ads.linkedin.com   TRUE    /   FALSE   0   lang    v=2&amp;lang=en-us
.ca.linkedin.com    TRUE    /   FALSE   0   JSESSIONID  ajax:##########
.ca.linkedin.com    TRUE    /   FALSE   ##########  aam_uuid    ##########
.ca.linkedin.com    TRUE    /   TRUE    ##########  bscookie    &quot;v=1&amp;##########;
.linkedin.com   TRUE    /   FALSE   0   AMCVS_##########AdobeOrg    1
.linkedin.com   TRUE    /   FALSE   ##########  AMCV_##########AdobeOrg -##########CvVersion%7C3.3.0
.linkedin.com   TRUE    /   TRUE    ##########  UserMatchHistory    ##########
.linkedin.com   TRUE    /   FALSE   ##########  _ga ##########
.linkedin.com   TRUE    /   TRUE    ##########  _guid   ##########
.linkedin.com   TRUE    /   FALSE   ##########  _lipt   ##########
.linkedin.com   TRUE    /   FALSE   ##########  bcookie &quot;v=2&amp;####################
.linkedin.com   TRUE    /   FALSE   0   lang    v=2&amp;lang=en-US
.linkedin.com   TRUE    /   TRUE    ##########  li_oatml    ##########
.linkedin.com   TRUE    /   TRUE    ##########  li_sugr##########
.linkedin.com   TRUE    /   FALSE   ##########  liap    true
.linkedin.com   TRUE    /   FALSE   ##########  lidc    ####################
.linkedin.com   TRUE    /   FALSE   0   spectroscopyId  ##########
.linkedin.com   TRUE    /   FALSE   1604170427  utag_main   v_id:##########$_sn:2$_se:10$_ss:0$_st:##########$ses_id:##########exp-session$_pn:1%3Bexp-session
.www.linkedin.com   TRUE    /   TRUE    ##########  JSESSIONID  &quot;ajax:##########
.www.linkedin.com   TRUE    /   TRUE    ##########  bscookie    ##########
.www.linkedin.com   TRUE    /   TRUE    ##########  li_at   ##########
.www.linkedin.com   TRUE    /   TRUE    ##########  lissc1  1
.www.linkedin.com   TRUE    /   TRUE    ##########  lissc2  1
.www.linkedin.com   TRUE    /   FALSE   ##########  mst v=1&amp;JpIWa
.www.linkedin.com   TRUE    /   FALSE   ##########  sl  v=1&amp;40m-N
www.linkedin.com    FALSE   /   FALSE   0   PLAY_LANG   en
www.linkedin.com    FALSE   /   TRUE    0   PLAY_SESSION    ##########
www.linkedin.com    FALSE   /   FALSE   0   fcookie AQHTcAxQ-##########
www.linkedin.com    FALSE   /   FALSE   0   lil-lang    en_US

abbeyjackson-mc commented 5 years ago

I didn't realize you could install Firefox on Mac 🤦‍♂️. I just tried using the cookies file from that Firefox add on and it DOES work. What I notice is different is that the one from Firefox includes credentials from the third party service I logged in through and these are not in the cookies file that I got from the Chrome add on. Other than that they look the same (I did not diff, they just look the same with a quick glance). It makes sense though without those credentials I wouldn't have permission.

However it does still fail when it goes to retrieve the second video:


[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '--cookies', 'cookies.txt', 'https://www.linkedin.com/learning/giving-your-elevator-pitch', '-o', '%(playlist)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s - %(title)s.%(ext)s', '--write-sub']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.10.29
[debug] Python version 3.7.4 (CPython) - Darwin-18.7.0-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
[linkedin:learning:course] Downloading JSON metadata
[download] Downloading playlist: Giving Your Elevator Pitch
[linkedin:learning:course] playlist Giving Your Elevator Pitch: Collected 5 video ids (downloading 5 of them)
[download] Downloading video 1 of 5
[linkedin:learning] welcome: Downloading 360p JSON metadata
[linkedin:learning] welcome: Downloading 540p JSON metadata
[linkedin:learning] welcome: Downloading 720p JSON metadata
[linkedin:learning] welcome: Downloading m3u8 information
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://files3.lynda.com/secure/courses/157139/VBR_MP4h264_main_HD720/157139_00_01_WL30_Intro.mp4?########'
[download] Giving Your Elevator Pitch/1 - Introduction/1 - Welcome.mp4 has already been downloaded
[download] 100% of 31.87MiB
[download] Downloading video 2 of 5
[linkedin:learning] benefits-to-an-elevator-speech: Downloading 360p JSON metadata
[linkedin:learning] benefits-to-an-elevator-speech: Downloading 540p JSON metadata
[linkedin:learning] benefits-to-an-elevator-speech: Downloading 720p JSON metadata
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 796, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/linkedin.py", line 127, in _real_extract
    self._sort_formats(formats, ('width', 'height', 'source_preference', 'tbr', 'abr'))
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 1327, in _sort_formats
    raise ExtractorError('No video formats found')
youtube_dl.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.```

abbeyjackson-mc commented 5 years ago

SO FYI the problem for this issue seems to be with the cookies file however after getting a proper cookies files you probably will still not be able to download the course.

WoodpeckerBaby commented 4 years ago

@ remitamine most public libraries offer free premium accounts to Linkedin Learning or Lynda, which are the same. Linkedin learning also has free trials.

ytdl-org / youtube-dl