Error exception and What is the recommendation delay?

[KoichaDev]

so I noticed I can't finish to download the whole the course since I get the error:

Exception in thread Thread-1: Traceback (most recent call last): File "threading.py", line 932, in _bootstrap_inner File "threading.py", line 870, in run File "Loader.py", line 470, in downloader File "site-packages\selenium\webdriver\remote\webelement.py", line 80, in click File "site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute File "site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute File "site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element ... is not clickable at point (874, 651). Other element would receive the click: ... (Session info: chrome=80.0.3987.132)

I assume this is because I am using the default setting on the delay option, and I downloading to fast. I might even get temporarily banned.

My question now is what is the recommended option I should add on the delay settings if I want to download a bunch of videos for the whole day?

[r00tmebaby]

Play with it, that's the only way to find what works best for you. The speed depends on many circumstances. The program works great with 0's with me but I have very quick SSD and 300 Mbit's internet connection so it's really hard to judge and/or advise based on my experience with the program.

To give you more control I have just added the slow time on few more critical places. The option that slows down the whole process is "Download Delay" so you can play with it at first. Re-download version 0.16.5.

[KoichaDev]

Thanks. I just took a quick google search of the typical error of Element is not clickable at point since I actually getting this error from different courses. It could be that LinkedIn Learning has different elements implemented on different courses.

https://stackoverflow.com/questions/11908249/debugging-element-is-not-clickable-at-point-error

Is there some logic you can do with the python where you can implement a "click" as in JavaScript instead? Seems like there is an element that is "hidden" and the python can't find the button to click.

[r00tmebaby]

I have to reproduce the problem first and see what can be done. It's less likely Linked in to change the style for some courses. It does not make sense because the ajax won't work with different requests. I don't think that this is the actual problem it's most probably something else. Which courses are you trying to download?

If you do a video on the browser behaviours so I can see where the program stuck can be very useful too.

In regards to Python, well Python is a general-purpose language and has a huge library base, so it's very powerful by its own. Of course, a combination of both will still work some but why should you do it if python already has all you need out of box-like ActionChains method from selenium driver library 'https://www.selenium.dev/selenium/docs/api/py/webdriver/selenium.webdriver.common.action_chains.html that can move the mouse to a specific hidden area and interact with it as a human bean. You can even use windows based libraries to move the mouse to a particular pixel based on the window size to do what is needed (which is old-fashion but will work). So that been said you do not need Javascript.

[KoichaDev]

One of the courses I have a problem to download is :

https://www.linkedin.com/learning/starting-a-business-with-family-and-friends/understand-tax-implications?pathUrn=urn%3Ali%3AlyndaLearningPath%3A56dfb8ae92015a33b4908fdb&u=57692065

https://www.linkedin.com/learning/seo-foundations-2/understanding-the-importance-of-links?pathUrn=urn%3Ali%3AlyndaLearningPath%3A56d7a3f83dd559b764b88a8f&u=57692065

https://www.linkedin.com/learning/finance-foundations-income-taxes-2/how-many-sets-of-books-does-a-large-company-keep?pathUrn=urn%3Ali%3AlyndaLearningPath%3A56dfb8ae92015a33b4908fdb&u=57692065

Not sure what you meant by Browser behaviour, unless you mean the browser support from your app? If so, then I am using the Mozilla/5.0 (Window NT 10:0; Win64; x64; rv:67) Gecko/20100101 Firefox/67

Sorry for my ignorant. I am not familiar with the Python, since I have never touched it. I just googled the exception error, and what I just came across is that the element was hidden, and therefore the app couldn't find the element to "click" on it, so I thought the solution was maybe trying to using the Vanilla JavaScript might be a solution, but I totally see your viewpoint on this.

Hopefully, you might reproduce the issue and can debug it.

[r00tmebaby]

I meant that you could make a video on the browser window (bot interaction) and console so I can clearly see what the bot is doing and what is the error in the console to be able to check the code and see where it stops, to find the problem.

But I will try these courses later today and come back with a solution if the problem is related to the courses.

[r00tmebaby]

Hello @KoichaDev,

I found what caused the problem. It was caused in occasions whenever the same video was attempted to be download twice and the program was made to break the loop considering that this is very likely to be the last video from the course

I have added an additional checking if that occurs three times in a row not once as it was before then it will break since sometimes if the speed is very fast it may happen to have the same video downloaded twice before the next video is loaded. That should not happen three times in a row and if does there is another issue too so no point for more downloading attempts anyway since it will download the same video over and over again.

Download the last version please, I haven't changed the version number because it's a minor change but only the date from 06/03/2020 to 08/03/202

Should you want to play with that behaviour you can find my addition at line 493 from the source: if last_video_src.count(video_src) > 2:

Regards,

[KoichaDev]

Fantastic you found the issue! It's working perfectly now! Thank you! I will notify you if there are some other issues I will find on the way.

EDIT: I am getting another error here from this course: https://www.linkedin.com/learning/seo-foundations-2/exploring-ecommerce-information-architecture

I added a screenshot here: https://imgur.com/a/nwmVsYq

Any idea why I am getting this?

[r00tmebaby]

You can try the executable version that I have just added. It works with cookies and downloads the videos directly from lynda.com where the original courses are anyway it's a bit slower variant but more reliable. There is no change in the GUI, you add the courses from LinkedIn as before the logic though is different.

https://github.com/r00tmebaby/LinkedIn-Downloader/tree/master/Windows%20Executable's/Version%200.16.5%20v2

I will check your report in the next few days and update the python version

[KoichaDev]

Thank you, good sir! Really appreciate this! :)