Open linglin506 opened 3 years ago
@estebanpdl Yes it does!
@himanshudabas Great. Just saw the warning about twint.exe. Thanks so much.
We need to find a new endpoint. I loaded the site and saw this link New Api link?. It is important to note that if you copy the request and past it to notepad you can see that the request is provided with headers and you can see that it provides a x-csrf-token and x-guest-token which are required. You can find these tokens in the cookie fields "gt" and "ct0".
This is somewhat the official twitter api and you need a bearer token to access this. The bearertoken is provided in the https://abs.twimg.com/responsive-web/client-web/main.6b1bdbc5.js file. Just search for "AAAAAA" and you can find it.
Now you can craft a GET request in postman with the x-csrf-token, the x-guest-token and the bearer token for authentification.
Thank you.
Can you give an example on how to to do it? Can we use the endpoint 'https://twitter.com/search?' with your suggestion?
Thank you
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This works!!
@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.
thank you so much!!! it worked
@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.
Thank you so much. It worked for me :)
@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.
Does this version install the command-line tool as well?
I have the same question. After reinstallation, I cannot find the command 'twint'. Any body can help me ?
@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.
Does this version install the command-line tool as well?
I have the same question. After reinstallation, I cannot find the command 'twint'. Any body can help me ?
Ubuntu 18.04 Python 3.6.9
@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.
Does this version install the command-line tool as well?
I have the same question. After reinstallation, I cannot find the command 'twint'. Any body can help me ?
Ubuntu 18.04 Python 3.6.9
DONOT uninstall twint, just upgrade will be OK!
@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.
Does this version install the command-line tool as well?
I have the same question. After reinstallation, I cannot find the command 'twint'. Any body can help me ?
Ubuntu 18.04 Python 3.6.9
DONOT uninstall twint, just upgrade will be OK!
Thank youu!!
I'm having the same issue even after installing this one: git+https://github.com/twintproject/twint.git@origin/master#egg=twint
I'm having the same issue even after installing this one: mgit+https://github.com/twintproject/twint.git@origin/master#egg=twint
Try this sequence once, it worked for me on Google Colab.
!pip install twint
!pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
import twint
import nest_asyncio
c = twint.Config()
c.Username = "github"
c.Search = "code"
nest_asyncio.apply()
twint.run.Search(c)
I got the same issue and the previous solution doesn't work now
@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.
Thank you so much !!
This isn't working as of today!
@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.
This worked. In case it throws an error in a virtual environment, run again without --user
This worked :)
@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.
Thank you, it worked for me!
Jupyter notebook workaround for this issue:
!pip3 uninstall twint -y
!pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
Then continue as usual
import twint
import nest_asyncio
nest_asyncio.apply()
c = twint.Config()
c.Search = "code"
c.Limit = 10
twint.run.Search(c)
EDIT: This somehow doesn't work in Colab. Use Kaggle Notebook instead. You only need to install the package anyway, no need to uninstall it first. So the script will look like this
!pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
import twint
import nest_asyncio
nest_asyncio.apply()
c = twint.Config()
c.Search = "code"
c.Limit = 10
twint.run.Search(c)
after installing the new one, I am getting
~/.local/lib/python3.7/site-packages/twint/format.py in Tweet(config, t) 21 output = output.replace("{hashtags}", ",".join(t.hashtags)) 22 output = output.replace("{cashtags}", ",".join(t.cashtags)) ---> 23 output = output.replace("{replies}", t.replies_count) 24 output = output.replace("{retweets}", t.retweets_count) 25 output = output.replace("{likes}", t.likes_count)
TypeError: replace() argument 2 must be str, not int
Is there any specification for installing it on ubuntu?
Is there any specification for installing it on ubuntu?
I have no problem installing it on Ubuntu 16.04 under Conda environtment. I used this command:
pip install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
I got the same issue and the previous solution doesn't work now
create a new virtual environment in anaconda and install python 3.6 in there.
conda create -n yourenvname python=3.6 anaconda
Then, follow these steps:-
git clone https://github.com/twintproject/twint.git
cd twint
pip3 install . -r requirements.txt
pip3 install --user --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint
Then try to run the script:-
!pip install nest_asyncio
import nest_asyncio
nest_asyncio.apply()
import twint
config = twint.Config()
config.Search = "Dogecoin"
config.Limit = 10
config.Store_csv = True
config.Output = "doge.csv"
twint.run.Search(config)
Hi everyone, when I run the following command:
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
Does it work on MAC OS? I have been trying to use it since 2019
#
Hi everyone, when I run the following command:
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
Does it work on MAC OS? I have been trying to use it since 2019
@romellfudi Yes it works on my macOS. It gave me error message at the beginning but after I deleted the "--user" part it works
@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.
thank you so much!!! it worked
worked for me
Me Too ^-^
@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.
thank you so much!!! it worked
Also worked for me, as of September 2021
^ This worked for me too!
upgrading twint works.
pip3 uninstall twint
@tsaaii can you normally scrape all the tweets without encountering limit/date constraints? Thanks!
it limits tweets on time line as mentioned in documentation. I haven't seen any date constraints so far.
@tsaaii Thanks for your reply! Did you clone the git repo as well or just pips upgrade? I can only get a few tweets after uninstall and reinstall.
clone the repo first, upgrade it next. It works for me. Make sure python version is 3.6.
@tsaaii Thanks! the 3.6 version and upgrade works for me but only for the latest tweets. Twint does not return anything when specifying since and until. Just want to know if you also cannot retrieve historical tweets or it's just me!
This issue fixed after running this command: pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
However, following this upgrade, it resumed collect briefly (16 tweets recovered then the following print out before the error:
RefreshTokenException Traceback (most recent call last)
/var/folders/7z/866jzymn52zcbnymflj7xhbw0000gn/T/ipykernel_6392/906502173.py in
~/.local/lib/python3.8/site-packages/twint/run.py in Search(config, callback) 408 config.Followers = False 409 config.Profile = False --> 410 run(config, callback) 411 if config.Pandas_au: 412 storage.panda._autoget("tweet")
~/.local/lib/python3.8/site-packages/twint/run.py in run(config, callback) 327 raise 328 --> 329 get_event_loop().run_until_complete(Twint(config).main(callback)) 330 331
~/.local/lib/python3.8/site-packages/twint/run.py in init(self, config)
34 # USAGE : to get a new guest token simply do self.token.refresh()
35 self.token = token.Token(config)
---> 36 self.token.refresh()
37 self.conn = db.Conn(config.Database)
38 self.d = datelock.Set(self.config.Until, self.config.Since)
~/.local/lib/python3.8/site-packages/twint/token.py in refresh(self) 67 else: 68 self.config.Guest_token = None ---> 69 raise RefreshTokenException('Could not find the Guest token in HTML')
RefreshTokenException: Could not find the Guest token in HTML
已收到您的邮件,会尽快回复
!pip install twint
!pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
import twint
import nest_asyncio
c = twint.Config()
c.Username = "github"
c.Search = "code"
nest_asyncio.apply()
twint.run.Search(c)
Hey all... I am having this problem now in Dec 2021. I have tried all the fixes, but none worked for me. I used Google Cloud Shell and I was able to get twint to work, but then I was getting the same errors.
I was wondering how to use the above date? Do I run it on the command line? Just wondering.
Thanks.
I am facing the same issue ..I have tried all the solutions. Can anyone help me
已收到您的邮件,会尽快回复
@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.
Thank you very much Himanshu! Was really struggling with this.
I am facing the same issue ..I have tried all the solutions. Can anyone help me
Have you tried what Himanshu suggested?
@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.
Thank you very much Himanshu! Was really struggling with this.
This worked for me! Thanks.
已收到您的邮件,会尽快回复
@eabanoz @jiwoochoi-kor @hnodaro Aşağıdaki komutu kullanarak önceki twint kurulumunu kaldırın
pip3 uninstall twint
Ardından aşağıdaki komutu kullanarak twint'i yeniden yükleyin
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
Bu çalışmalı.
Himanshu'ya çok teşekkür ederim! Bu konuda gerçekten zorlanıyordu.
this didn't work for me
已收到您的邮件,会尽快回复
已收到您的邮件,会尽快回复
thanks
This error will be reported from 0:00 on September 18, 2020. It is found through debug that the https://twitter.com/i/search/timeline page does not exist. Can you fix it?By changing the IP, it can be determined that Twitter is not banning the IP
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) sleeping for 1.0 secs CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) sleeping for 1.0 secs CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) sleeping for 1.0 secs CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) sleeping for 1.0 secs CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) sleeping for 1.0 secs CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) sleeping for 1.0 secs CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) sleeping for 1.0 secs