ikeboy / pluralsight-scraper

Pluralsight video downloader
https://www.knyz.org/blog/post/pluralsight-scraper-released/
GNU General Public License v2.0
139 stars 50 forks source link

April is free, will the maintainer return? #10

Closed Qiangong2 closed 4 years ago

Qiangong2 commented 4 years ago

Pluralsight is giving everyone April for free, will you still work on it?

iifast2 commented 4 years ago

i doubt so , i tried every download app / extension ... im still trying to find something that could download pluralsight vids , if you find any tool out there just tell me 👍

vezaynk commented 4 years ago

Oh wow! This is news to me, thanks. I will definitely try find time to get this project working again.

Qiangong2 commented 4 years ago

Awesome. I look forward to your progress :)

MustafaAlQadoum commented 4 years ago

Good luck knyzorg, waiting for the update :)

brofist01 commented 4 years ago

guys , none of the onlune apps/extensions works, nothing works for this april ! the only working pluralsight downloader is decrypter but the repo got deleted ! So here's a new one !

https://github.com/brofist01/Pluralsight-Decrypt

i put a description for u , i hope they don't delete that youtube video too

vezaynk commented 4 years ago

@Brofist01 Decrypter as in decrypting files downloaded by the offline player?

Either way, the source code appears to be missing. Do you have it?

iifast2 commented 4 years ago

the decrypt app works for me @brofist01 thanks !

brofist01 commented 4 years ago

@knyzorg yes , you download pluralsight offline courses in thier app , then take the data , decrypt them .

nope :/

this is the original repo ( it was deleted by the DMC for some reason ) :

https://github.com/vinhloc1996/DecryptPluralSightVideos

im just reposting for research purposes .... you can ask @vinhloc1996 if he got source code , last time i checked he had V1.3 , the one im sharing is v1.0

Qiangong2 commented 4 years ago

@brofist01 Thanks for the decrypter, but there is no Net/Free/OpenBSD (Or even Linux) version of the offline player, so it will not work for me. Hopefully the pluralsight downloader will work fully.

vezaynk commented 4 years ago

I have most of the code written for the new version today. It’s cleaner, faster, overall better.

I have swapped out the backend from nightmarejs to puppeteer. It’s now solely used for login and nothing more. All the rest of the work is now done in node proper.

This will allow it run with much less overhead after the initial login.

I am currently figuring out a clean way around the rate-limiting such that it wastes the minimal amount of time waiting around.

Edit: For sake of a faster release, I think this will come later.

After that its just some documentation and configurability.

I will have more improvements down the line (post-release).

To close, you can expect a release tomorrow night.

Qiangong2 commented 4 years ago

Awesome! Thanks for getting on this so quickly!

vezaynk commented 4 years ago

I have a work in progress here: https://github.com/knyzorg/pluralsight-scraper/tree/version-2

There is however, a very serious problem. Pluralsight seems to have generally upped their defences against this sort of thing and is actively blocking access.

I made the mistake of doing this from my own account. It took a lot of activity to get it suspended. Oh well, I wasn't using it anyways.

I then proceeded to make a new free account, but it got flagged before it even had time to download a single course:

Email from pluralsight

This was despite waiting 10 seconds between requests. I'm not sure what number is safe. We could try 20? 30? 60 seconds?

Anyways, I got 2 accounts suspended tonight, I'll continue messing around with it tomorrow. The magic waiting number is on line 36, feel free to play with it (But err on the side of caution). The place to modify the course to download is line 102.

Worst case scenario I'll set it to something like 2 minutes between each request.

Beyond timeouts, we could experiment with rotating accounts. I rather not go through the hassle of it myself, but implementing it would be fairly trivial.

Qiangong2 commented 4 years ago

Could it be an IP issue? That might be why you got the ban so quick. Maybe try a different dns (like 1.1.1.1)?

candidodmv commented 4 years ago

I also try to solve the initial problem of "blob:url", but with other mechanisms, using theier own infrastructured, that is with their codes.

I've seen that on the player there's a global variable called "initialState" that has almost the enougth information for do the work, but the magic remais in only one request([POST]https://app.pluralsight.com/video/clips/v3/viewclip), that return all real information about the video(paths), but for any reason, i cant reproduce this request with injected code, the server reject the request(eveni sending the same parameters/headers).

Another way is using the m3u8, obteined easily with filtering the request with electron webRequest events(url.toLowerCase().endsWith('/master.m3u8')). but, i don't find a way to download the video file from this format.🤔

vezaynk commented 4 years ago

@Qiangong2 I'm inclined to believe it's tracking per account and not per IP. It may also be a mix of both, but there's definitely some per-account controls going on.

@candidodmv It is exactly what my new code is doing, with the difference of getting the clipIds from the API instead of InitialState.

vezaynk commented 4 years ago

I have pushed to master. I did not test it end-to-date due to my accounts getting banned. I have increased the timeouts to avoid it happening. You can modify the code to increase them further. I don't know how much it matters.

Anyways. Version 2 has arrived. Tell me how it breaks!