batuhaniskr / twitter-intelligence

Twitter Intelligence OSINT project performs tracking and analysis of the Twitter
MIT License
228 stars 67 forks source link

0 tweet received always #9

Closed gusleig closed 4 years ago

gusleig commented 5 years ago

I managed to install but I'm getting always 0 tweet received.

any way to debug it?

thanks

batuhaniskr commented 5 years ago

@gusleig Can you send me the command when you're running?

Make sure that you install pip3 with the dependencies and python3 with the running application.

gusleig commented 5 years ago

I've tried many commands but this is one that also doens't bring any tweets

python3 tracking.py --username "jack"

batuhaniskr commented 5 years ago

Has TweetAnalysis.db named database been created?

gusleig commented 5 years ago

Yes, its there. image

gusleig commented 5 years ago

debugging with pycharm, the problem is with line

response = opener.open(url)

gets an exception

the value of url is:

url='https://twitter.com/i/search/timeline?f=tweets&q=%20sakarya&src=typd&max_position='

when I open this url in my browser I get a file

gusleig commented 5 years ago

Looking to the exception, what it gets:

<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)>

gusleig commented 5 years ago

I fixed it using those lines in tweet_operation.py:

import ssl . . . ctx = ssl.create_default_context() ctx.check_hostname = False ctx.verify_mode = ssl.CERT_NONE

THEN

opener = urllib2.build_opener(urllib2.HTTPSHandler(context=ctx), urllib2.HTTPCookieProcessor(cookiejar))

batuhaniskr commented 5 years ago

I am testing with Python 3.6.2 and debian based linux distro but I am not getting this error. Which you use python version?

gusleig commented 5 years ago

Tested with linux Ubuntu 16.04.5 LTS and macOS

I just tested today again with a brand new linux installation in my virtual machine, same SSL error, this kind of error happens too when making the Google Map analysis.

Anyway I managed to fix it with the lines above and others (for the map analysis) I can share when I get back home from work.

Things I added in my local version that I think its cool.

  1. Timestamp to every printed line
  2. A tweets per second ratio at the end.

I think your code is great, very helpful. Thank you very much

ghost commented 5 years ago

@batuhaniskr ,Hi, I have same issue,

batuhaniskr commented 5 years ago

@ahmedjavad Hi, Go to the folder where Python is installed, example in my case it is installed in the Applications folder with the folder name 'Python 3.6'. Now double click on 'Install Certificates.command'.

batuhaniskr commented 5 years ago

Tested with linux Ubuntu 16.04.5 LTS and macOS

I just tested today again with a brand new linux installation in my virtual machine, same SSL error, this kind of error happens too when making the Google Map analysis.

Anyway I managed to fix it with the lines above and others (for the map analysis) I can share when I get back home from work.

Things I added in my local version that I think its cool.

1. Timestamp to every printed line

2. A tweets per second ratio at the end.

I think your code is great, very helpful. Thank you very much

@gusleig Thank you so much.

If you want, you can open a pull request.

ghost commented 5 years ago

@batuhaniskr What you means "installed in the Applications folder with the folder name 'Python 3.6'"? I have debian 9 with python3.5

batuhaniskr commented 5 years ago

@ahmedjavad Can you try pip3 install certifi command on your environment?

ghost commented 5 years ago

@batuhaniskr . Output : Requirement already satisfied: certifi in /usr/local/lib/python3.5/dist-packages

batuhaniskr commented 5 years ago

@ahmedjavad Ok. I have arranged. You can pull from origin master.

Can you tell me if the error is resolved?

ghost commented 5 years ago

@batuhaniskr Hi, same result,

python3 tracking.py --username "rihanna"
[+] Searching...

[+] 0 tweet received...

[+] Succesfully saved to the database.
batuhaniskr commented 5 years ago

@ahmedjavad Hi, Do you have a internet connection when you tested this project?

If there is wrong/different network settings, it may caused be this result.

ghost commented 5 years ago

Do you have a internet connection when you tested this project?

Yes, ^_^.

batuhaniskr commented 5 years ago

Sorry, I can not resolved your that situation.

devil67 commented 5 years ago

Me too [+] Searching...

[+] 0 tweet received...

[+] Succesfully saved to the database.

tobozo commented 5 years ago

Same symptoms: I always get the same response (although different order) whatever the query/username:

{'min_position': 'cm+55m--', 'focused_refresh_interval': 30000, 'has_more_items': False, 'new_latent_count': 0, 'items_html': '\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n \n'}

I'm didn't expect this to work anyway, Python seems to have sticky problems with TLS everywhere its version 2 is bundled with a system.

batuhaniskr commented 5 years ago

@tobozo

Can you try with python 3.6.2 or higher version?

I added create default ssl context for 3.6 and less verion but not resolved result.

tobozo commented 5 years ago

@batuhaniskr thanks for your answer :+1:

I had the same results after upgrading to python 3.6.7 + reinstalling, however after upgrading pip from 8.11 to 18.1 it seems to give some results

now I'm experiencing redundancy, as if the pagination is broken our cycling through

image

do you want me to open a different issue for that?

batuhaniskr commented 5 years ago

@tobozo Thank you.

If there are 0 Tweet received problem, the pip should be upgraded.

I resolved pagination same tweets problem. Published on master branch.

devil67 commented 5 years ago

Yeah good man this tool work very well thanks.I've upgrade python3 to Python 3.7.1 and pip to 18.1