search

StegSchreck / RatS

Movie Ratings Synchronization with Python
GNU Affero General Public License v3.0
272 stars 31 forks source link

letterboxd does not work as an inserter #143

Closed frazzydee closed 3 years ago

frazzydee commented 3 years ago

Describe the bug

I am running RatS on a headless machine with a clean installation of firefox.

To Reproduce Steps to reproduce the behavior:

  1. Have credentials for criticker & letterboxd in credentials.cfg
  2. $ python3 transfer_ratings.py --source criticker --destination letterboxd

Expected behavior Ratings successfully parsed from criticker (this happens OK), and inserted into letterboxd (this didn't happen)

Desktop (please complete the following information):

Stacktrace

$ python3 transfer_ratings.py --source criticker --destination letterboxd
===== Criticker: Retrieving ratings XML
===== Criticker: saved 247 parsed movies to /home/faraaz/src/RatS/RatS/exports/20210126210139_Criticker.json
===== Letterboxd: performing loginTraceback (most recent call last):
  File "transfer_ratings.py", line 234, in <module>
    main()
  File "transfer_ratings.py", line 86, in main
    execute(args)
  File "transfer_ratings.py", line 136, in execute
    execute_inserting(args, movies, parser)
  File "transfer_ratings.py", line 153, in execute_inserting
    inserter = get_inserter_from_arg(destination)(args)
  File "/home/faraaz/src/RatS/RatS/letterboxd/letterboxd_ratings_inserter.py", line 24, in __init__
    super(LetterboxdRatingsInserter, self).__init__(Letterboxd(args), args)
  File "/home/faraaz/src/RatS/RatS/letterboxd/letterboxd_site.py", line 10, in __init__
    super(Letterboxd, self).__init__(args)
  File "/home/faraaz/src/RatS/RatS/base/base_site.py", line 36, in __init__
    self._init_browser()
  File "/home/faraaz/src/RatS/RatS/base/base_site.py", line 69, in _init_browser
    self.login()
  File "/home/faraaz/src/RatS/RatS/base/base_site.py", line 85, in login
    self._click_login_button()
  File "/home/faraaz/src/RatS/RatS/base/base_site.py", line 129, in _click_login_button
    login_button.click()
  File "/home/faraaz/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/home/faraaz/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "/home/faraaz/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/faraaz/.local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementClickInterceptedException: Message: Element <input class="button -action button-green" type="submit"> is not clickable at point (593,365) because another element <iframe id="zIframe_tyche_trendi_video" class="zeus_iframe" src="//cdn.playwire.com/bolt4/js/zeus/releases/5.1.2/frame/frame.html#id=tyche_trendi_video___pv=1"> obscures it

Geckodriver log

1611712899298   geckodriver INFO    Listening on 127.0.0.1:40115
1611712901130   Marionette  WARN    TLS certificate errors will be ignored for this session
JavaScript error: https://www.criticker.com/js/javascript.js, line 33: ReferenceError: adblock is not defined
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
1611712914299   geckodriver INFO    Listening on 127.0.0.1:47883
1611712916220   Marionette  WARN    TLS certificate errors will be ignored for this session

Additional context I note from the stacktrace it appears that the login button may be getting blocked by a video. Letterboxd uses advertisement videos on its login page, and I don't have any adblock installed (since it is a headless machine with a clean firefox install).

StegSchreck commented 3 years ago

Hey @frazzydee, Thanks for bringing this to my attention. I run a pi-hole in my local network which prevents me from running into this video you mentioned. But after deactivating that I could in fact reproduce your issue. I have created a fix for that. Please let me know if this also works on your setup.

frazzydee commented 3 years ago

@StegSchreck, thank you very much for the fix.

Unfortunately I still get an error, below. Please let me know if I can provide any more information

faraaz@debian:~/src/RatS$ python3 transfer_ratings.py --source criticker --destination letterboxd
===== Criticker: Retrieving ratings XML
===== Criticker: saved 267 parsed movies to /home/faraaz/src/RatS/RatS/exports/20210211133807_Criticker.json
===== Letterboxd: performing loginTraceback (most recent call last):
  File "transfer_ratings.py", line 234, in <module>
    main()
  File "transfer_ratings.py", line 86, in main
    execute(args)
  File "transfer_ratings.py", line 136, in execute
    execute_inserting(args, movies, parser)
  File "transfer_ratings.py", line 153, in execute_inserting
    inserter = get_inserter_from_arg(destination)(args)
  File "/home/faraaz/src/RatS/RatS/letterboxd/letterboxd_ratings_inserter.py", line 24, in __init__
    super(LetterboxdRatingsInserter, self).__init__(Letterboxd(args), args)
  File "/home/faraaz/src/RatS/RatS/letterboxd/letterboxd_site.py", line 12, in __init__
    super(Letterboxd, self).__init__(args)
  File "/home/faraaz/src/RatS/RatS/base/base_site.py", line 36, in __init__
    self._init_browser()
  File "/home/faraaz/src/RatS/RatS/base/base_site.py", line 69, in _init_browser
    self.login()
  File "/home/faraaz/src/RatS/RatS/base/base_site.py", line 85, in login
    self._click_login_button()
  File "/home/faraaz/src/RatS/RatS/base/base_site.py", line 129, in _click_login_button
    login_button.click()
  File "/home/faraaz/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/home/faraaz/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "/home/faraaz/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/faraaz/.local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementClickInterceptedException: Message: Element <input class="button -action button-green" type="submit"> is not clickable at point (593,365) because another element <iframe id="zIframe_tyche_trendi_video" class="zeus_iframe" src="//cdn.playwire.com/bolt4/js/zeus/releases/4.3.40/frame/frame.html#id=tyche_trendi_video___pv=1"> obscures it
StegSchreck commented 3 years ago

Ah, it seems that the video didn't get dismissed properly in your case. I'll have another look.

StegSchreck commented 3 years ago

Hey @frazzydee I have found a fix that worked for me. Sorry that I didn't have time to do this earlier. Could you verify that this works for you as well?