Altimis / Scweet

A simple and unlimited twitter scraper : scrape tweets, likes, retweets, following, followers, user info, images...
MIT License
1k stars 223 forks source link

Could not locate the login Element #116

Open Qmeasure opened 2 years ago

Qmeasure commented 2 years ago

NoSuchElementException Traceback (most recent call last) /var/folders/l6/r_4g46d57vq67tth9s0dh2sh0000gn/T/ipykernel_6708/2251003829.py in 6 env_path = ".env" 7 ----> 8 following = get_users_following(users=users, env=env_path, verbose=0, headless=True, wait=2, limit=50, file_path=None)

~/opt/anaconda3/lib/python3.9/site-packages/Scweet/user.py in get_users_following(users, env, verbose, headless, wait, limit, file_path) 110 111 def get_users_following(users, env, verbose=1, headless=True, wait=2, limit=float('inf'), file_path=None): --> 112 following = utils.get_users_follow(users, headless, env, "following", verbose, wait=wait, limit=limit) 113 114 if file_path == None:

~/opt/anaconda3/lib/python3.9/site-packages/Scweet/utils.py in get_users_follow(users, headless, env, follow, verbose, wait, limit) 316 # log in (the .env file should contain the username and password) 317 # driver.get('https://www.twitter.com/login') --> 318 log_in(driver, env, wait=wait) 319 sleep(wait) 320 # followers and following dict of each user

~/opt/anaconda3/lib/python3.9/site-packages/Scweet/utils.py in log_in(driver, env, timeout, wait) 236 237 # enter email --> 238 email_el = driver.find_element_by_xpath(email_xpath) 239 sleep(random.uniform(wait, wait + 1)) 240 email_el.send_keys(email)

~/opt/anaconda3/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py in find_element_by_xpath(self, xpath) 519 stacklevel=2, 520 ) --> 521 return self.find_element(by=By.XPATH, value=xpath) 522 523 def find_elements_by_xpath(self, xpath) -> List[WebElement]:

~/opt/anaconda3/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py in find_element(self, by, value) 1246 value = '[name="%s"]' % value 1247 -> 1248 return self.execute(Command.FIND_ELEMENT, { 1249 'using': by, 1250 'value': value})['value']

~/opt/anaconda3/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py in execute(self, driver_command, params) 423 response = self.command_executor.execute(driver_command, params) 424 if response: --> 425 self.error_handler.check_response(response) 426 response['value'] = self._unwrap_value( 427 response.get('value', None))

~/opt/anaconda3/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py in check_response(self, response) 245 alert_text = value['alert'].get('text') 246 raise exception_class(message, screen, stacktrace, alert_text) # type: ignore[call-arg] # mypy is not smart enough here --> 247 raise exception_class(message, screen, stacktrace) 248 249 def _value_or_default(self, obj: Mapping[_KT, _VT], key: _KT, default: _VT) -> _VT:

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//input[@autocomplete="username"]"} (Session info: headless chrome=99.0.4844.83) Stacktrace: 0 chromedriver 0x00000001034c33c9 chromedriver + 5120969 1 chromedriver 0x0000000103451773 chromedriver + 4654963 2 chromedriver 0x0000000103040ba8 chromedriver + 392104 3 chromedriver 0x0000000103075d21 chromedriver + 609569 4 chromedriver 0x0000000103075ee1 chromedriver + 610017 5 chromedriver 0x00000001030a8574 chromedriver + 816500 6 chromedriver 0x000000010309334d chromedriver + 729933 7 chromedriver 0x00000001030a625a chromedriver + 807514 8 chromedriver 0x0000000103093213 chromedriver + 729619 9 chromedriver 0x000000010306933d chromedriver + 557885 10 chromedriver 0x000000010306a315 chromedriver + 561941 11 chromedriver 0x0000000103480cfd chromedriver + 4848893 12 chromedriver 0x000000010349a778 chromedriver + 4953976 13 chromedriver 0x000000010349ff4e chromedriver + 4976462 14 chromedriver 0x000000010349b0aa chromedriver + 4956330 15 chromedriver 0x0000000103475f10 chromedriver + 4804368 16 chromedriver 0x00000001034b5488 chromedriver + 5063816 17 chromedriver 0x00000001034b560f chromedriver + 5064207 18 chromedriver 0x00000001034ca565 chromedriver + 5150053 19 libsystem_pthread.dylib 0x00007ff806a3d4f4 _pthread_start + 125 20 libsystem_pthread.dylib 0x00007ff806a3900f thread_start + 15

It seems that the xpath is wrong

10frazier commented 2 years ago

I am getting the same error in headless mode, but when headless is set to false it functions perfectly.

heis71 commented 2 years ago

Duplicate of #122

ZaneH commented 2 years ago

I'm experiencing a similar issue:

Traceback (most recent call last):
  File "main.py", line 99, in <module>
    items = get_users_followers(opts['<username>'], './env.secret', 0, False, 15, 200)
  File "/home/user/miniconda3/lib/python3.7/site-packages/Scweet/user.py", line 99, in get_users_followers
    followers = utils.get_users_follow(users, headless, env, "followers", verbose, wait=wait, limit=limit)
  File "/home/user/miniconda3/lib/python3.7/site-packages/Scweet/utils.py", line 318, in get_users_follow
    log_in(driver, env, wait=wait)
  File "/home/user/miniconda3/lib/python3.7/site-packages/Scweet/utils.py", line 238, in log_in
    email_el = driver.find_element_by_xpath(email_xpath)
AttributeError: 'WebDriver' object has no attribute 'find_element_by_xpath'

I tried disabling headless mode + waiting for 15 seconds It can't get past the login box (Debian 10, w/ Chrome, Python 3.7)

sethwheway commented 1 year ago

also encountering the above issue ^

Altimis commented 1 year ago

I'm sorry guys, I haven't had time to update the library lately. Will update it as soon as I find a slot

PedramMarandi commented 1 year ago

I have been able to fix the issue using this extension on firefox https://addons.mozilla.org/en-US/firefox/addon/submit-me/

Frocean commented 1 year ago

I also have the same attributeError: 'WebDriver' object has no attribute 'find_elements_by_xpath'. There isn't anything wrong when I use headless=False to check the Google Chrome web (without logging in), but it reports the error after the web closed.

RazeBerry commented 1 year ago

Also encountering the above issue. Is it ever fixed?

MarlonJerold commented 2 months ago

Here I am having the same error

image