search

bocchilorenzo / ntscraper

Scrape from Twitter using Nitter instances
MIT License
168 stars 29 forks source link

Help with getting the scraper running #62

Closed Troajan closed 7 months ago

Troajan commented 7 months ago

Hi,

I tried to install and test out ntscraper, but it would only give me this error message. Can anyone explain what I might need to do to fix it?

Thanks in advance 

 File "/Users/Library/Application Support/Sublime Text/Packages/User/Test.py", line 3, in <module>
    scraper = Nitter(log_level=1, skip_instance_check=False)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/ntscraper/nitter.py", line 52, in __init__
    self.instances = self._get_instances()
                     ^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/ntscraper/nitter.py", line 128, in _get_instances
    official = soup.find_all("tbody")[0]
               ~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
AritzUMA commented 7 months ago

I just get the same error in my scripts @bocchilorenzo

Seems like the Nitter function doens't find any instances

bocchilorenzo commented 7 months ago

Hi, I just checked and the error is thrown because the list of instances was removed from nitter's wiki. At this point I think the project is truly in its final stages. Right now there are only like 10 public instances that work, and working with a new locally hosted instance seems to be unfeasible because of the guest account limitations. I've changed the way instances are retrieved, but it doesn't have any health checks so verifying the instances will take a little longer than before.