bisguzar / twitter-scraper

Scrape the Twitter Frontend API without authentication.
MIT License
3.88k stars 601 forks source link

Does this work? #191

Open CarloCobal opened 2 years ago

CarloCobal commented 2 years ago

If so how?

bisguzar commented 2 years ago

Not working for now, refactor/fix required.

nsde commented 2 years ago
Traceback (most recent call last):
  File "/home/lix/.local/lib/python3.9/site-packages/requests/models.py", line 910, in json
    return complexjson.loads(self.text, **kwargs)
  File "/home/lix/.local/lib/python3.9/site-packages/simplejson/__init__.py", line 525, in loads
    return _default_decoder.decode(s)
  File "/home/lix/.local/lib/python3.9/site-packages/simplejson/decoder.py", line 370, in decode
    obj, end = self.raw_decode(s)
  File "/home/lix/.local/lib/python3.9/site-packages/simplejson/decoder.py", line 400, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/lix/anaconda3/lib/python3.9/site-packages/twitter_scraper/modules/tweets.py", line 170, in get_tweets
    yield from gen_tweets(pages)
  File "/home/lix/anaconda3/lib/python3.9/site-packages/twitter_scraper/modules/tweets.py", line 37, in gen_tweets
    html=r.json()["items_html"], url="bunk", default_encoding="utf-8"
  File "/home/lix/.local/lib/python3.9/site-packages/requests/models.py", line 917, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: [Errno Expecting value] : 0
bisguzar commented 2 years ago

Still not working @nsde . I'm not sure will I fix it. PRs are welcome.

psylopunk commented 2 years ago

Still not working :(

FarisHijazi commented 1 year ago

do you know where the issue is? to get us started and save us time

shelsoloa commented 1 year ago

@FarisHijazi I'm going to look into this a bit today. Any chance you found the start of the issue? The error linked by @nsde is the same one I'm experiencing.

shelsoloa commented 1 year ago

@mahrtayyab works for me - thanks!

ege-del commented 1 year ago

@bisguzar can the repo owner please weigh in ? Since I am not familiar with Twitter's API (they wont let me use it), I cant figure out why it's not working. Which one of these is more likely ?

  1. They blocked the method this project uses
  2. Updated API causes faulty requests
  3. Other bugs in the project itself
bisguzar commented 1 year ago

I guess the first one, but I didn't debug it yet. I'm just predicting @ege-del

david1asher commented 1 year ago

not working for me