Letterboxd: Unable to locate element: //form[@id='signin-form']//input[@id='signin-username']

[vn-ki]

Describe the bug Exception

Expected behavior No excpetion

Desktop (please complete the following information):

• Version: latest master
• Platform OS:arch
• Geckodriver version: geckodriver 0.24.0
• Python version: <!-- python3 --version -->
• PIP version: <!-- pip --version -->
• Running in a virtualenv?:
• Running in Docker?:
• Executed command: python3 transfer_ratings.py --source letterboxd --destination movielens

Stacktrace

$ python3 transfer_ratings.py --source letterboxd --destination movielens                                                                                                                  
===== Letterboxd: performing loginTraceback (most recent call last):                                                                                                                       
  File "transfer_ratings.py", line 227, in <module>                                                                                                                                        
    main()
  File "transfer_ratings.py", line 79, in main 
    execute(args)
  File "transfer_ratings.py", line 127, in execute
    parser = get_parser_from_arg(args.source)(args)
  File "/home/vn-ki/random/RatS/RatS/letterboxd/letterboxd_ratings_parser.py", line 11, in __init__
    super(LetterboxdRatingsParser, self).__init__(Letterboxd(args), args)
  File "/home/vn-ki/random/RatS/RatS/letterboxd/letterboxd_site.py", line 10, in __init__
    super(Letterboxd, self).__init__(args)
  File "/home/vn-ki/random/RatS/RatS/base/base_site.py", line 35, in __init__
    self._init_browser()
  File "/home/vn-ki/random/RatS/RatS/base/base_site.py", line 68, in _init_browser
    self.login()
  File "/home/vn-ki/random/RatS/RatS/base/base_site.py", line 87, in login
    raise e
  File "/home/vn-ki/random/RatS/RatS/base/base_site.py", line 83, in login
    self._insert_login_credentials()
  File "/home/vn-ki/random/RatS/RatS/base/base_site.py", line 119, in _insert_login_credentials
    login_field_user = self.browser.find_element_by_xpath(self.LOGIN_USERNAME_SELECTOR)
  File "/home/vn-ki/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 394, in find_element_by_xpath
    return self.find_element(by=By.XPATH, value=xpath)
  File "/home/vn-ki/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element
    return self.execute(Command.FIND_ELEMENT, {
  File "/home/vn-ki/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/vn-ki/.local/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: //form[@id='signin-form']//input[@id='signin-username']

[vn-ki]

I did some digging around and it looks like RatS tries to login even after trying to login.

The page is redirected to home page and thus cannot find the login form.

Unfortunately, I do not know the code base thus cannot understand what could be causing this.

[vn-ki]

Inserting break after this line

https://github.com/StegSchreck/RatS/blob/01bd362bd941f4008609ee230ecac347ce752b59/RatS/base/base_site.py#L84

fixes the problem.

This leads to me to think that _user_is_not_logged_in is not working correctly, at least for letterboxd.

[vn-ki]

Letterboxd to Movielens doesn't work anyway, I guess :(

ERROR: There are no movies with an IMDB id in the parsed data. As the target site is looking for this id to match the data, there is nothing left to do. A workaround would be to upload th
e data to a third site, which knows the IMDB id, and parse again from there.

[StegSchreck]

Hey @vn-ki Thank you for bringing this to my attention and your detailed description. _user_is_not_logged_in() is implemented individually for letterboxd to look for the username.

https://github.com/StegSchreck/RatS/blob/01bd362bd941f4008609ee230ecac347ce752b59/RatS/letterboxd/letterboxd_site.py#L17

The standard would actually be to look for the login form elements. I see why you suspect this to be the issue and I came to the same conclusion after reading your first comment. However, after multiple runs, I couldn't reproduce the behaviour with the additional login attempt after successful login. I removed the special handling now, so the error you pasted should not occur anymore.

As for the issue with the Letterboxd to Movielens migration of data, I can only say the same thing here as I already put in the console output you saw. As Movielens' own CSV Importer matches the data by IMDB id. The Alternative would be to let the script rate the movies individually (if the source does not provide sufficient data for the destination for matching movies), which will of course increase the runtime of the script significantly. I will create a separate ticket for this particular case to be considered.