soraxas / echo360

Commandline tool for automated downloads of echo360 videos hosted by university
https://cs.tinyiu.com/echo360
MIT License
262 stars 51 forks source link

Fails after logging in #1

Closed irrgent closed 4 years ago

irrgent commented 6 years ago

Program fails to download and produces the following error after login credentials provided.

` Download will use "PhantomJS" webdriver from LOCAL executable <<<

Logging into "https://canvas.sydney.edu.au/ess/portal/section/956"... Credentials needed... Unikey: Traceback (most recent call last): File "usydEcho360.py", line 148, in main() File "usydEcho360.py", line 128, in main downloader.download_all() File "/home/username/echo_downloader/USYDecho360-master/USYDecho360/downloader.py", line 138, in download_all self.login() File "/home/username/echo_downloader/USYDecho360-master/USYDecho360/downloader.py", line 73, in login self.loginWithCredentials() File "/home/username/echo_downloader/USYDecho360-master/USYDecho360/downloader.py", line 105, in loginWithCredentials user_name.clear() File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 95, in clear self._execute(Command.CLEAR_ELEMENT) File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 628, in _execute return self._parent.execute(command, params) File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 314, in execute self.error_handler.check_response(response) File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.InvalidElementStateException: Message: {"errorMessage":"Element is not currently interactable and may not be manipulated","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:43767","User-Agent":"selenium/3.12.0 (python linux)"},"httpVersion":"1.1","method":"POST","post":"{\"sessionId\": \"437210c0-59bd-11e8-8cf9-6d249373a31f\", \"id\": \":wdc:1526553007974\"}","url":"/clear","urlParsed":{"anchor":"","query":"","file":"clear","directory":"/","path":"/clear","relative":"/clear","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/clear","queryKey":{},"chunks":["clear"]},"urlOriginal":"/session/437210c0-59bd-11e8-8cf9-6d249373a31f/element/:wdc:1526553007974/clear"}} Screenshot: available via screen

`

change-agent commented 6 years ago

This is also happening for me -- my university is University of Melbourne. Oddly enough, the script says "No need to login", but the Echo360 website always prompts me for a login in the browser.

Happy to assist in any way I can...

Download will use "PhantomJS" webdriver from LOCAL executable <<< Logging into "https://echo360.org.au/ess/portal/section/bdfad1dd-dddb-4104-8fe7-a8da365db288"... INFO: No need to login :) Done! Retrieving echo360 Course Info... Traceback (most recent call last): File "echo360.py", line 151, in main() File "echo360.py", line 131, in main downloader.download_all() File "C:\Users\Daniel\Downloads\echo360-master\echo360\downloader.py", line 141, in download_all videos = self._course.get_videos().videos File "C:\Users\Daniel\Downloads\echo360-master\echo360\course.py", line 25, in get_videos course_data_json = self._get_course_data() File "C:\Users\Daniel\Downloads\echo360-master\echo360\course.py", line 85, in _get_course_data json_str = self.driver.find_element_by_tag_name("pre").text File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 523, in find_element_by_tag_name return self.find_element(by=By.TAG_NAME, value=name) File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 957, in find_element 'value': value})['value'] File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 314, in execute self.error_handler.check_response(response) File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.NoSuchElementException: Message: {"errorMessage":"Unable to find element with tag name 'pre'","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"90","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:63419","User-Agent":"selenium/3.12.0 (python windows)"},"httpVersion":"1.1","method":"POST","post":"{\"using\": \"tag name\", \"sessionId\": \"cd507490-5d7c-11e8-a5c6-b5335dddc289\", \"value\": \"pre\"}","url":"/element","urlParsed":{"anchor":"","query":"","file":"element","directory":"/","path":"/element","relative":"/element","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/element","queryKey":{},"chunks":["element"]},"urlOriginal":"/session/cd507490-5d7c-11e8-a5c6-b5335dddc289/element"}} Screenshot: available via screen

soraxas commented 6 years ago

@irrgent It appears that you are using the canvas url instead of the actual course url for the download link. Check this video tutorial that I just created to clears up how to get the actual link.

soraxas commented 6 years ago

@change-agent Thanks for letting me know! It appears that your uni is using echo360 Cloud instead of the institution's own echo360 system (which i haven't encountered yet) judging by the hostname. It would be very helpful if you can help support this by providing the log of your runs.

I have incorporated logging into the module in the latest commit (download zip here). I have explicitly excluded the username/password in the logging. It will log the page source in order to figure out the html tag I needed for capturing the video source. It might include your email in the page source if you successfully login into the institution (but I think the problem here is the url format for echo360 Cloud is different than institutional echo360 system). If that's the case, you can perform a search and replace to rename it to something appropriate.

Invoke the debugging with

run.bat COURSE_URL --debug

Please provide the result log file echo360downloader.log (in the same folder as the script) onto website such as https://pastebin.com/. Thanks!

change-agent commented 6 years ago

@soraxas Thank you for your prompt response!

Here is the requested log. Hope that helps!

soraxas commented 4 years ago

Hi! This is an old post, but with PR #20 this module should now support echo360 cloud platform (echo360.org). If you are still using echo360, you can try it out in the latest master branch!:)