scraper.get_tweets IndexError: list index out of range

[ehengirmen]

Hi, first of all thank you for your project. I am trying to get tweets from a user and using code below but it gives me an error. can you help me please? my code:

from ntscraper import Nitter

scraper = Nitter(log_level=1)

bezos_tweets = scraper.get_tweets("JeffBezos", mode='user') print(bezos_tweets)

error: 27-Jul-23 15:14:30 - No instance specified, using random instance https://nitter.in.projectsegfau.lt Traceback (most recent call last): File "C:\Users\eheng\PycharmProjects\twitter_selenium_1\main.py", line 5, in bezos_tweets = scraper.get_tweets("JeffBezos", mode='user') File "C:\Users\eheng\PycharmProjects\twitter_selenium_1\venv\lib\site-packages\ntscraper\nitter.py", line 582, in get_tweets return self.__search(term, mode, number, since, until, max_retries, instance) File "C:\Users\eheng\PycharmProjects\twitter_selenium_1\venv\lib\site-packages\ntscraper\nitter.py", line 487, in search instance, soup = self.get_page(endpoint, instance, max_retries) File "C:\Users\eheng\PycharmProjects\twitter_selenium_1\venv\lib\site-packages\ntscraper\nitter.py", line 113, in __get_page if soup.findall("div", class="show-more")[-1].find("a").text == "Load newest": IndexError: list index out of range

[psegovias]

https://github.com/bocchilorenzo/ntscraper/issues/5#issuecomment-1653782540

[ehengirmen]

Thank you, I was able to solve the problem by applying the change here:

https://github.com/bocchilorenzo/ntscraper/pull/6/commits/4e7f2d604fa37d2276868d49b4b3a2e544208876

• instance_new, soup = self.__get_page("/Twitter", instance)
  • instance_new, soup = self.__get_page("/x", instance)

I think indeed IndexError: list index out of range error is connected to instance. After running couple of times, different instances get selected and I was able to get scrape results.

Thank you again @psegovias