search

Greenwolf / social_mapper

A Social Media Enumeration & Correlation Tool by Jacob Wilkin(Greenwolf)
GNU General Public License v3.0
3.81k stars 788 forks source link

Could not authenticate on LinkedIn. NoneType' object has no attribute '__getitem__ #112

Closed venturaEffect closed 5 years ago

venturaEffect commented 5 years ago

Hi Greenwolf!

First of all congrats! Love it! Have seen your speech on a youtube video. Priceless!

I'm trying to start with it. Got through some issues that have already solved. But this one I just can't get it. It looks like the method is empty, but evrything looks fine on the social_mapper.py document. The username is good and the password too.

What could cause this ERROR?

Thanks a lot!

Really appreciate your work and your time :)

Greenwolf commented 5 years ago

Hi @zaesarius, thank you very much, I'm glad you enjoyed the presentation!

Is there any more error output to go along with that? What OS are you running it on? Did you try Kali/Ubuntu following the instructions exactly?

venturaEffect commented 5 years ago

Thanks @Greenwolf for the quick answer.

This is the error:

python social_mapper.py -f company -i "Noho Comunicación" -m accurate -fb -t strict
[!] Could not authenticate to linkedin. 'NoneType' object has no attribute '__getitem__'

I'm using Ubuntu, the following version:

            Kernel: Linux 4.15.0-47-generic
      Architecture: x86-64

I'm using python2 and all the requirements updated.

I've followed the instructions plus after looked on youtube tutorials for more info.

But still :(

Greenwolf commented 5 years ago

@zaesarius just checking but you have provided the script with valid credentials right?

venturaEffect commented 5 years ago

Mmmm... No sure....

You mean username and password right? on social_mapper.py?

If so, they're right

I tried them with username, email. checked like four times the password. It seems everything is fine. Also logged on firefox browser to check. It works all as usual. Only this error like there would be nothing on the authentication...

Greenwolf commented 5 years ago

@zaesarius ok i'll look into it, It might take me a few days though.

venturaEffect commented 5 years ago

Ok

my soul is on your hands @Greenwolf

lorenzoromani1983 commented 5 years ago

I am failing LinkedIn and Instagram Login. I am on Buscador. Other social networks work.

Greenwolf commented 5 years ago

Hi @lorenzoromani1983, try spinning up a fresh kali or ubuntu VM to install it on, sometimes other tools change the defaults and it doesn't work on a non default install.

lorenzoromani1983 commented 5 years ago

I tried on Ubuntu. Same problem as zaesarius, only on linkedin and instagram: login failed when trying imagefolder mode and getitem error when trying to search by company on linkedin. Maybe something has changed in the underlying html?

venturaEffect commented 5 years ago

Hi @Greenwolf, I'm testing social_mapper in diferent ways. Trying to log on each social account of facebook, twitter, ig. Get diferent kind of errors. Login keys are fine.

Facebook:

[-] Error Filling out Facebook Profiles [-]
Message: invalid argument: can't kill an exited process

[-]

Results file: SM-Results/results-social-mapper.csv
HTML file: SM-Results/results-social-mapper.html

Twitter:

Traceback (most recent call last):
  File "social_mapper.py", line 1112, in <module>
    peoplelist = fill_twitter(peoplelist)
  File "social_mapper.py", line 218, in fill_twitter
    TwitterfinderObject = twitterfinder.Twitterfinder(showbrowser)
  File "/usr/local/bin/social_mapper/modules/twitterfinder.py", line 19, in __init__
    self.driver = webdriver.Firefox()
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py", line 174, in __init__
    keep_alive=True)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: invalid argument: can't kill an exited process

Instagram:

Traceback (most recent call last):
  File "social_mapper.py", line 1117, in <module>
    peoplelist = fill_instagram(peoplelist)
  File "social_mapper.py", line 296, in fill_instagram
    InstagramfinderObject = instagramfinder.Instagramfinder(showbrowser)
  File "/usr/local/bin/social_mapper/modules/instagramfinder.py", line 20, in __init__
    self.driver = webdriver.Firefox()
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py", line 174, in __init__
    keep_alive=True)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: invalid argument: can't kill an exited process

Just in case you're working on it. For me, I think I'm putting all parameters right. Maybe I'm wrong 😅

MakingL commented 5 years ago

@zaesarius I also have gotten the same error. Have you solved the error?

venturaEffect commented 5 years ago

Hi,

No, @Greenwolf is looking to solve it :)

@zaesarius ok i'll look into it, It might take me a few days though.

Greenwolf commented 5 years ago

Hi all, I got round to having a look into this today and made a few tweaks, so they changed the title of the login page which was stopping the login, so thats working now. However its not grabbing images or matching still so i'll need to work on this a bit more.

venturaEffect commented 5 years ago

No problem @Greenwolf . Thanks in advance!

Greenwolf commented 5 years ago

@zaesarius @lorenzoromani1983 @MakingL, just pushed a fix for the LinkedIn Module, could you give it a try and let me know if it works for you now please?

Greenwolf commented 5 years ago

Instagram should also be working again now

Greenwolf commented 5 years ago 
$ python social_mapper.py -f imagefolder -i ./Input-Examples/imagefolder/ -m fast -ig -fb -li -tw -vv   

[+] Facebook Login Page loaded successfully [+]
[+] Facebook Login Success [+]

Facebook Check 1/3 : Bill Gates
cannot identify image file 'potential_target_image.jpg'
        Match found: Bill Gates
        Facebook: https://www.facebook.com/gatesbill.mc?ref=br_rs
Facebook Check 2/3 : Linus Torvalds
        Match found: Linus Torvalds
        Facebook: https://www.facebook.com/linus.torvalds.1671?ref=br_rs
Facebook Check 3/3 : Steve Jobs
        Match found: Steve Jobs
        Facebook: https://www.facebook.com/jobs.steve.75457?ref=br_rs

[+] Twitter Login Page loaded successfully [+]
[+] Twitter Login Success [+]

Twitter Check 1/3 : Bill Gates
        Match found: Bill Gates
        Twitter: https://twitter.com/BillGates
Twitter Check 2/3 : Linus Torvalds
        Match found: Linus Torvalds
        Twitter: https://twitter.com/TorvaldsChinese
Twitter Check 3/3 : Steve Jobs
        Match found: Steve Jobs
        Twitter: https://twitter.com/SteveJobs_BOT

[+] Instagram Login Page loaded successfully [+]
[+] Instagram Login Success [+]

Instagram Check 1/3 : Bill Gates
        Match found: Bill Gates
        Instagram: https://instagram.com/billgates_daddy/
Instagram Check 2/3 : Linus Torvalds
        Match found: Linus Torvalds
        Instagram: https://instagram.com/linux_linus.t/
Instagram Check 3/3 : Steve Jobs
        Match found: Steve Jobs
        Instagram: https://instagram.com/stevejobsok/

[+] LinkedIn Login Page loaded successfully [+]
[+] LinkedIn Login Success [+]

LinkedIn Check 1/3 : Bill Gates
        Match found: Bill Gates
        LinkedIn: https://linkedin.com/in/williamhgates/
LinkedIn Check 2/3 : Linus Torvalds
        Match found: Linus Torvalds
        LinkedIn: https://linkedin.com/in/linustorvalds/
LinkedIn Check 3/3 : Steve Jobs
        Match found: Steve Jobs
        LinkedIn: https://linkedin.com/in/steve-jobs-47358646/

Bill Gates
        LinkedIn: https://linkedin.com/in/williamhgates/
        Facebook: https://www.facebook.com/gatesbill.mc?ref=br_rs
        Twitter: https://twitter.com/BillGates
        Instagram: https://instagram.com/billgates_daddy/

Linus Torvalds
        LinkedIn: https://linkedin.com/in/linustorvalds/
        Facebook: https://www.facebook.com/linus.torvalds.1671?ref=br_rs
        Twitter: https://twitter.com/TorvaldsChinese
        Instagram: https://instagram.com/linux_linus.t/

Steve Jobs
        LinkedIn: https://linkedin.com/in/steve-jobs-47358646/
        Facebook: https://www.facebook.com/jobs.steve.75457?ref=br_rs
        Twitter: https://twitter.com/SteveJobs_BOT
        Instagram: https://instagram.com/stevejobsok/

Results file: SM-Results/results-social-mapper.csv
HTML file: SM-Results/results-social-mapper.html

Task Duration: 0:03:52.76586
venturaEffect commented 5 years ago

Hi!

Works perfectly just Instagram Login Failed. HAve switched with diferent accounts putting password but still seems to not work...

Will still try to make some more tries.

Greenwolf commented 5 years ago

If you use the '-s' option to show the browser, can you see where it is going wrong? Can you post the new output showing what bit it failed on?

venturaEffect commented 5 years ago

Well. I've done that. But don't see on the terminal any browser info or in results. Should it be on other place?

$ python social_mapper.py -f imagefolder -i ./Input-Examples/imagefolder/ -m fast -s -ig -fb -li -tw

[+] Facebook Login Page loaded successfully [+]
[+] Facebook Login Success [+]

Facebook Check 1/1 : Bill Gates                                cannot identify image file 'potential_target_image.jpg'

[+] Twitter Login Page loaded successfully [+]
[-] Twitter Login Failed [-]

Twitter Check 1/1 : Bill Gates                                
[+] Instagram Login Page loaded successfully [+]
[-] Instagram Login Failed [-]

Instagram Check 1/1 : Bill Gates                                
[+] Instagram Login Page loaded successfully [+]
[-] Instagram Login Failed [-]

Instagram Timeout Error, session has expired and attempts to reestablish have failed
LinkedIn Check 1/1 : Bill Gates                                LinkedIn Timeout Error, session has expired and attempts to reestablish have failed

Bill Gates
    Facebook: https://www.facebook.com/gatesbill.mc?ref=br_rs
    Twitter: https://twitter.com/BillGates

Results file: SM-Results/results-social-mapper.csv
HTML file: SM-Results/results-social-mapper.html

Task Duration: 0:02:08.615666

There seems also now that logins failed on other social accounts even if I didn't change anything.... Maybe I'm missing something?

Greenwolf commented 5 years ago

It's strange that your facebook is succeeding. I want to say somethings wrong with your install because so many are failing, but selenium must be ok because of the facebook success. Are you accessing the sites from a non english speaking region? I wonder if a VPN to the UK or the US would make them work as perhaps it's an issue with the sites languages.

venturaEffect commented 5 years ago

Yes I'm in Spain.

Still not working. Will look tomorrow with VPN

MakingL commented 5 years ago

@Greenwolf Hi, I pulled the code again, and run with python social_mapper.py -f imagefolder -i ./Input-Examples/imagefolder/ -m fast -ig -fb -li -tw -vv. I got error:

[-] Error Filling out Facebook Profiles [-]
Message: Reached error page: about:neterror?e=connectionFailure&u=https%3A//www.facebook.com/login&c=UTF-8&f=regular&d=Firefox%20can%E2%80%99t%20establish%20a%20connection%20to%20the%20server%20at%20www.facebook.com.

[-]
Traceback (most recent call last):
  File "social_mapper.py", line 1112, in <module>
    peoplelist = fill_twitter(peoplelist)
  File "social_mapper.py", line 219, in fill_twitter
    TwitterfinderObject.doLogin(twitter_username,twitter_password)
  File "/home/mlee/social_mapper/modules/twitterfinder.py", line 27, in doLogin
    self.driver.get("https://twitter.com/login")
  File "/anaconda3/envs/py2_scrawl/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
    self.execute(Command.GET, {'url': url})
  File "/anaconda3/envs/py2_scrawl/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/anaconda3/envs/py2_scrawl/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Reached error page: about:neterror?e=connectionFailure&u=https%3A//twitter.com/login&c=UTF-8&f=regular&d=Firefox%20can%E2%80%99t%20establish%20a%20connection%20to%20the%20server%20at%20twitter.com.

I installed VXFB by sudo apt-get install xvfb -y and sudo apt install xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic.

I run it in vm-ubuntu. And I'm in China, I connect to internet through VPN in HK. I can access these social networking sites in my vm-linux firefox.

Maybe some of the dependent tools are not installed correctly, but I checked my running environment as described in the readme and found no difference.

Thank you.

venturaEffect commented 5 years ago

VPN too slow... Have now tried with proxy cause it works for me on Facebook.

But with social mapper get:

python social_mapper.py -f imagefolder -i ./Input-Examples/imagefolder/ -m fast -ig -fb -li -tw -s
[-] Error Filling out Facebook Profiles [-]
Traceback (most recent call last):
  File "social_mapper.py", line 1106, in <module>
    print(e)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u201c' in position 158: ordinal not in range(128)

https://stackoverflow.com/questions/24264892/unicodeencodeerror-ascii-codec-cant-encode-character-u-u201c-in-position-3

IS there something in the code afecting or not encoding in utf-8 ?

lorenzoromani1983 commented 5 years ago

I have re-installed the whole package with git clone but I still cannot search by company. Maybe i should only download the single updated files?

Greenwolf commented 5 years ago

Ah i remember some of the problems on the social media sites are if you dont have the languages set to english. Try logging in manually and changing the language to english in your settings on each of the sites if your in spanish etc? In addition to using a UK or US VPN so it doesn't auto detect location and switch (which also may be an issue).

With regards to searching by company, i'll need to double check that again when i get time, i checked that the imagefolder and csv based searches are working. But it's possible linkedIn changed their API which is how the company search works.

MakingL commented 5 years ago

@Greenwolf Hi, I pulled the code again, and run with python social_mapper.py -f imagefolder -i ./Input-Examples/imagefolder/ -m fast -ig -fb -li -tw -vv. I got error:

[-] Error Filling out Facebook Profiles [-]
Message: Reached error page: about:neterror?e=connectionFailure&u=https%3A//www.facebook.com/login&c=UTF-8&f=regular&d=Firefox%20can%E2%80%99t%20establish%20a%20connection%20to%20the%20server%20at%20www.facebook.com.

[-]
Traceback (most recent call last):
  File "social_mapper.py", line 1112, in <module>
    peoplelist = fill_twitter(peoplelist)
  File "social_mapper.py", line 219, in fill_twitter
    TwitterfinderObject.doLogin(twitter_username,twitter_password)
  File "/home/mlee/social_mapper/modules/twitterfinder.py", line 27, in doLogin
    self.driver.get("https://twitter.com/login")
  File "/anaconda3/envs/py2_scrawl/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
    self.execute(Command.GET, {'url': url})
  File "/anaconda3/envs/py2_scrawl/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/anaconda3/envs/py2_scrawl/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Reached error page: about:neterror?e=connectionFailure&u=https%3A//twitter.com/login&c=UTF-8&f=regular&d=Firefox%20can%E2%80%99t%20establish%20a%20connection%20to%20the%20server%20at%20twitter.com.

I installed VXFB by sudo apt-get install xvfb -y and sudo apt install xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic.

I run it in vm-ubuntu. And I'm in China, I connect to internet through VPN in HK. I can access these social networking sites in my vm-linux firefox.

Maybe some of the dependent tools are not installed correctly, but I checked my running environment as described in the readme and found no difference.

Thank you.

Hi, I checked my settings and found out that I used the socks5 proxy VPN which should be set up in Firefox. I changed the self.driver = webdriver.Firefox() as:

profile = webdriver.FirefoxProfile()
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 1080)  # My socks_v5 proxy port is 1080
profile.set_preference("network.proxy.socks_remote_dns", True)
profile.update_preferences()

self.driver = webdriver.Firefox(profile)

, and then successfully logged into the social networking site, but there are still some errors in the crawling process, I am trying to find out the reason.

python social_mapper.py -f imagefolder -i ./Input-Examples/imagefolder/ -m fast -ig -fb -li -tw -vv

[+] Facebook Login Page loaded successfully [+]
[+] Facebook Login Success [+]

Facebook Check 1/3 : Bill Gates
('Connection aborted.', error(0, 'Error'))
('Connection aborted.', error(0, 'Error'))
('Connection aborted.', error(0, 'Error'))
SOCKSHTTPSConnectionPool(host='www.facebook.com', port=443): Max retries exceeded with url: /search/async/profile_picture/?fbid=100006530588717&width=8000&height=8000 (Caused by NewConnectionError('<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x7f03beb64950>: Failed to establish a new connection: [Errno 111] Connection refused',))

......

SOCKSHTTPSConnectionPool(host='www.facebook.com', port=443): Max retries exceeded with url: /search/async/profile_picture/?fbid=100023654640261&width=8000&height=8000 (Caused by NewConnectionError('<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x7f03beb649d0>: Failed to establish a new connection: [Errno 111] Connection refused',))
Facebook Check 2/3 : Linus Torvalds
Facebook Check 3/3 : Steve Jobs
('Connection aborted.', error(0, 'Error'))
('Connection aborted.', error(0, 'Error'))
('Connection aborted.', error(0, 'Error'))
('Connection aborted.', error(0, 'Error'))
('Connection aborted.', error(0, 'Error'))
    Match found: Steve Jobs
    Facebook: https://www.facebook.com/jobs.steve.12139862?ref=br_rs

[+] Twitter Login Page loaded successfully [+]
[+] Twitter Login Success [+]

Twitter Check 1/3 : Bill Gates
Twitter Check 2/3 : Linus Torvalds
Twitter Check 3/3 : Steve Jobs
    Match found: Steve Jobs
    Twitter: https://twitter.com/forealstevejobs

[+] Instagram Login Page loaded successfully [+]
[+] Instagram Login Success [+]

Instagram Check 1/3 : Bill Gates
ERROR
[Errno socket error] [Errno -2] Name or service not known
ERROR

......

[Errno socket error] [Errno -2] Name or service not known
ERROR
Instagram Check 2/3 : Linus Torvalds
ERROR
[Errno socket error] [Errno -2] Name or service not known
ERROR

......

[Errno socket error] [Errno -2] Name or service not known
ERROR
Instagram Check 3/3 : Steve Jobs
ERROR
[Errno socket error] [Errno -2] Name or service not known
ERROR

......

[Errno socket error] [Errno -2] Name or service not known
ERROR
[Errno socket error] [Errno -2] Name or service not known
ERROR

[+] LinkedIn Login Page loaded successfully [+]
[+] LinkedIn Login Success [+]

LinkedIn Check 1/3 : Bill Gates
cannot identify image file 'potential_target_image.jpg'
cannot identify image file 'potential_target_image.jpg'
cannot identify image file 'potential_target_image.jpg'
LinkedIn Check 2/3 : Linus Torvalds
LinkedIn Check 3/3 : Steve Jobs
    Match found: Steve Jobs
    LinkedIn: https://linkedin.com/in/steve-jobs-47358646/

Steve Jobs
    LinkedIn: https://linkedin.com/in/steve-jobs-47358646/
    Facebook: https://www.facebook.com/jobs.steve.12139862?ref=br_rs
    Twitter: https://twitter.com/forealstevejobs

Results file: SM-Results/results-social-mapper.csv
HTML file: SM-Results/results-social-mapper.html

Task Duration: 1:03:26.009995
MakingL commented 5 years ago

@Greenwolf Hey, I changed some code to use the chrome browser:

display = Display(visible=0, size=(1600, 1024))
display.start()
if not showbrowser:
    os.environ['MOZ_HEADLESS'] = '1'
self.driver = webdriver.Firefox()

the change code is:

chrome_options = Options()
if not showbrowser:
    chrome_options.add_argument('--headless')
self.driver = webdriver.Chrome(chrome_options=chrome_options)

I successfully ran it in Windows and got this message:

E:\anaconda\envs\python2\python.exe "M:/social_mapper/social_mapper/social_mapper.py" -f imagefolder -i ./Input-Examples/imagefolder/ -m fast -ig -li -tw -s
Twitter Check 1/1 : Bill Gates                                Instagram Login Page title field seems to have changed, please make an issue on: https://github.com/Greenwolf/social_mapper
Instagram Check 1/1 : Bill Gates                                Instagram Login Page title field seems to have changed, please make an issue on: https://github.com/Greenwolf/social_mapper
Instagram Timeout Error, session has expired and attempts to reestablish have failed
LinkedIn Login Page title field seems to have changed, please make an issue on: https://github.com/Greenwolf/social_mapper
LinkedIn Check 1/1 : Bill Gates                                LinkedIn Login Page title field seems to have changed, please make an issue on: https://github.com/Greenwolf/social_mapper
LinkedIn Timeout Error, session has expired and attempts to reestablish have failed

Results file: SM-Results/results-social-mapper.csv
HTML file: SM-Results/results-social-mapper.html

Task Duration: 1:31:57.214000

It seems show linkedIn and Instagram API has changed.

venturaEffect commented 5 years ago

Is it possible that is not working because of new API changes these last day? Like on twitter?

So far I'm using a paid US proxy. Set on firefox browrse and on linux settings. All browsers set in english even the social media accounts.

user@user:/usr/local/bin/social_mapper$ python social_mapper.py -f imagefolder -i ./Input-Examples/imagefolder/ -m fast -ig -fb -li -tw -s

[+] Facebook Login Page loaded successfully [+]
[+] Facebook Login Success [+]

Facebook Check 1/1 : Bill Gates                                cannot identify image file 'potential_target_image.jpg'

[+] Twitter Login Page loaded successfully [+]
[-] Twitter Login Failed [-]

Twitter Check 1/1 : Bill Gates                                
[+] Instagram Login Page loaded successfully [+]
[-] Instagram Login Failed [-]

Instagram Check 1/1 : Bill Gates                                
[+] Instagram Login Page loaded successfully [+]
[-] Instagram Login Failed [-]

Instagram Timeout Error, session has expired and attempts to reestablish have failed
LinkedIn Check 1/1 : Bill Gates                                LinkedIn Timeout Error, session has expired and attempts to reestablish have failed

Bill Gates
    Facebook: https://www.facebook.com/gatesbill.mc?ref=br_rs
    Twitter: https://twitter.com/BillGates

Results file: SM-Results/results-social-mapper.csv
HTML file: SM-Results/results-social-mapper.html

Task Duration: 0:02:46.011283

Whithout touching anything running for second time and get this error...

user@user:/usr/local/bin/social_mapper$ python social_mapper.py -f imagefolder -i ./Input-Examples/imagefolder/ -m fast -ig -fb -li -tw -s

[-] Error Filling out Facebook Profiles [-]

Traceback (most recent call last):

  File "social_mapper.py", line 1106, in <module>
    print(e)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u201c' in position 158: ordinal not in range(128)

It seems now again a lnaguage problem. But can't understand why it appears the second time...

When I look at results-social-mapper.html I see that the first photo isn't loading. But got Bill Gates photo on fb and on tw. Same on the csv. There is the url profile.

MakingL commented 5 years ago

@zaesarius I ran the code on the server in USA a few minutes ago, it works, it looks like the API has not changed, maybe some of your configuration is wrong.

python social_mapper.py -f imagefolder -i ./Input-Examples/imagefolder/ -m fast -ig -fb -li -tw -vv 

[+] Facebook Login Page loaded successfully [+]
[+] Facebook Login Success [+]

Facebook Check 1/3 : Bill Gates
cannot identify image file 'potential_target_image.jpg'
    Match found: Bill Gates
    Facebook: https://www.facebook.com/gatesbill.mc?ref=br_rs
Facebook Check 2/3 : Steve Jobs
    Match found: Steve Jobs
    Facebook: https://www.facebook.com/jobs.steve.12139862?ref=br_rs
Facebook Check 3/3 : Linus Torvalds
    Match found: Linus Torvalds
    Facebook: https://www.facebook.com/linus.torvalds.1671?ref=br_rs

[+] Twitter Login Page loaded successfully [+]
[-] Twitter Login Failed [-]

Twitter Check 1/3 : Bill Gates
    Match found: Bill Gates
    Twitter: https://twitter.com/BillGates
Twitter Check 2/3 : Steve Jobs
    Match found: Steve Jobs
    Twitter: https://twitter.com/forealstevejobs
Twitter Check 3/3 : Linus Torvalds
    Match found: Linus Torvalds
    Twitter: https://twitter.com/TorvaldsChinese

[+] Instagram Login Page loaded successfully [+]
[+] Instagram Login Success [+]

Instagram Check 1/3 : Bill Gates
    Match found: Bill Gates
    Instagram: https://instagram.com/thisisbillgates/
Instagram Check 2/3 : Steve Jobs
    Match found: Steve Jobs
    Instagram: https://instagram.com/stevejobsok/
Instagram Check 3/3 : Linus Torvalds
    Match found: Linus Torvalds
    Instagram: https://instagram.com/linux_linus.t/

[+] LinkedIn Login Page loaded successfully [+]
[+] LinkedIn Login Success [+]

LinkedIn Check 1/3 : Bill Gates

[+] LinkedIn Login Page loaded successfully [+]
[+] LinkedIn Login Success [+]

LinkedIn Timeout Error, session has expired and attempts to reestablish have failed
LinkedIn Check 2/3 : Steve Jobs

[+] LinkedIn Login Page loaded successfully [+]
[+] LinkedIn Login Success [+]

LinkedIn Timeout Error, session has expired and attempts to reestablish have failed
LinkedIn Check 3/3 : Linus Torvalds

[+] LinkedIn Login Page loaded successfully [+]
[+] LinkedIn Login Success [+]

LinkedIn Timeout Error, session has expired and attempts to reestablish have failed

Bill Gates
    Facebook: https://www.facebook.com/gatesbill.mc?ref=br_rs
    Twitter: https://twitter.com/BillGates
    Instagram: https://instagram.com/thisisbillgates/

Steve Jobs
    Facebook: https://www.facebook.com/jobs.steve.12139862?ref=br_rs
    Twitter: https://twitter.com/forealstevejobs
    Instagram: https://instagram.com/stevejobsok/

Linus Torvalds
    Facebook: https://www.facebook.com/linus.torvalds.1671?ref=br_rs
    Twitter: https://twitter.com/TorvaldsChinese
    Instagram: https://instagram.com/linux_linus.t/

Results file: SM-Results/results-social-mapper.csv
HTML file: SM-Results/results-social-mapper.html

Task Duration: 0:07:20.087756
MakingL commented 5 years ago

@Greenwolf But it shows wrong when I run other command. Is Linkdin's API changed?

 python social_mapper.py -f company -i "SpiderLabs" -m accurate -a -t strict
[!] Could not authenticate to LinkedIn. 'NoneType' object has no attribute '__getitem__'

python social_mapper.py -f company -i "SpiderLabs" -m accurate -li -fb
[!] Could not authenticate to LinkedIn. 'NoneType' object has no attribute '__getitem__'
venturaEffect commented 5 years ago

Well I'm also playing with other bots for fun on -li and it opens window smaller and whithout option to scroll it untill sign up when it opens automattically.

Something is happening....

(On Linux with Chromium and puppeteer 1.13.0 and puppeteer 1.15.0)

Greenwolf commented 5 years ago

@MakingL is correct in that the API part has changed somehow, which is how it does the initial search by company. I'll try and get back to this next week to get it all working. I see that the manual linkedin searching isnt working either, which i thought i had fixed 🤔

venturaEffect commented 5 years ago

Hi @Greenwolf .Did you have a look on the linkedin searching & other the stuff?

Thanks a lot!

Hope you're doing well :)

venturaEffect commented 5 years ago

Hi again @Greenwolf .Now it is working perfectly! I've installed all again. I'm not sure if it's because of this or maybe you have done some changes.

Anyway thank you!!

Greenwolf commented 5 years ago

Ah thats good to hear, I made some changes a few weeks ago but i've been swamped with other things. So does everything work currently? Can you verify with the company based search, and does it find linkedin, instagram, facebook and twitter using the test people search?

venturaEffect commented 5 years ago

I'm going to try with company after lunch. I test it with image folder adding images. And for twitter, linkedin, Facebook and Instagram. All success

venturaEffect commented 5 years ago 
user@user:~/social_mapper$ python social_mapper.py -f company -i "spiderlabs" -m 
accurate -fb -tw -li -ig -t strict

[!] Could not authenticate to LinkedIn. 'NoneType' object has no attribute '__getitem__'

seems it doesn't work

Greenwolf commented 5 years ago

Ah that might be because the SpiderLabs isn’t a company. Try it on google or apple or microsoft etc so it finds something? Does that work?

venturaEffect commented 5 years ago

I tried with official LinkedIn companies. Got same result. Will try more times and we'll see if it works.... :sweat_smile:

Greenwolf commented 5 years ago

Should be all fixed now, @zaesarius care to help me verify 😃?

venturaEffect commented 5 years ago

Hi @Greenwolf ,

Tried with companies...

~/social_mapper$` python social_mapper.py -f company -i "international-academy" -m accurate -fb -tw -li -ig -t strict
[!] Could not authenticate to LinkedIn. 'NoneType' object has no attribute '__getitem__'

Still the same issue.

I'm also trying through a csv file that has fullname and image url columns. I've put it inside Input-Examples but it doesn't work.

This is the code with the error:

~/social_mapper$ python social_mapper.py -f csv -i /Input-Examples/test.csv -m accurate -fb -tw -li -ig -t strict
Traceback (most recent call last):
  File "social_mapper.py", line 899, in <module>
    file = open(args.input, 'rb')
IOError: [Errno 2] No such file or directory: '/Input-Examples/test.csv'

Any suggestion?

Thank you!

Greenwolf commented 5 years ago

Hey @zaesarius, it's all working for me on my end. The second problem is because you have put a / at the start of Input-Examples, try:

python social_mapper.py -f csv -i Input-Examples/test.csv -m accurate -fb -tw -li -ig -t strict

I think any other problems you are having now may be either due to locale setting on your accounts or installation problems. Otherwise you're going to have to delve into the code if you want to fix it. This is what it should all look like for both a image folder run, and a company search run. 

$ python social_mapper.py -f imagefolder -i Input-Examples/imagefolder/ -m fast -s -vv -fb -li -tw -ig

[+] Facebook Login Page loaded successfully [+]
[+] Facebook Login Success [+]

Facebook Check 1/3 : Bill Gates
cannot identify image file 'potential_target_image.jpg'
Facebook Check 2/3 : Linus Torvalds
        Match found: Linus Torvalds
        Facebook: https://www.facebook.com/linus.torvalds.1671?ref=br_r
Facebook Check 3/3 : Steve Jobs
        Match found: Steve Jobs
        Facebook: https://www.facebook.com/kaka.muda.543?ref=br_rs_WyJrZXl3b3Jkc191c2VycyJd

[+] Twitter Login Page loaded successfully [+]
[-] Twitter Login Failed [-]

Twitter Check 1/3 : Bill Gates
        Match found: Bill Gates
        Twitter: https://twitter.com/BillGates
Twitter Check 2/3 : Linus Torvalds
        Match found: Linus Torvalds
        Twitter: https://twitter.com/TorvaldsChinese
Twitter Check 3/3 : Steve Jobs
        Match found: Steve Jobs
        Twitter: https://twitter.com/forealstevejobs

[+] Instagram Login Page loaded successfully [+]
[+] Instagram Login Success [+]

Instagram Check 1/3 : Bill Gates
        Match found: Bill Gates
        Instagram: https://instagram.com/billgates_daddy/
Instagram Check 2/3 : Linus Torvalds
        Match found: Linus Torvalds
        Instagram: https://instagram.com/roland.yandx/
Instagram Check 3/3 : Steve Jobs
        Match found: Steve Jobs
        Instagram: https://instagram.com/frases.stevejobs/

[+] LinkedIn Login Page loaded successfully [+]
[+] LinkedIn Login Success [+]

LinkedIn Check 1/3 : Bill Gates
        Match found: Bill Gates
        LinkedIn: https://linkedin.com/in/williamhgates/
LinkedIn Check 2/3 : Linus Torvalds
        Match found: Linus Torvalds
        LinkedIn: https://linkedin.com/in/linustorvalds/
LinkedIn Check 3/3 : Steve Jobs
        Match found: Steve Jobs
        LinkedIn: https://linkedin.com/in/steve-jobs-47358646/

Bill Gates
        LinkedIn: https://linkedin.com/in/williamhgates/
        Twitter: https://twitter.com/BillGates
        Instagram: https://instagram.com/billgates_daddy/

Linus Torvalds
        LinkedIn: https://linkedin.com/in/linustorvalds/
        Facebook: https://www.facebook.com/linus.torvalds.1671?ref=br_rs
        Twitter: https://twitter.com/TorvaldsChinese
        Instagram: https://instagram.com/roland.yandx/

Steve Jobs
        LinkedIn: https://linkedin.com/in/steve-jobs-47358646/
        Facebook: https://www.facebook.com/kaka.muda.543?ref=br_rs_WyJrZXl3b3Jkc191c2VycyJd
        Twitter: https://twitter.com/forealstevejobs
        Instagram: https://instagram.com/frases.stevejobs/

Results file: SM-Results/results-social-mapper.csv
HTML file: SM-Results/results-social-mapper.html

Task Duration: 0:03:48.793424

$ python social_mapper.py -f company -i "Evil Corp LLC" -m fast -s -vv -fb -li -tw -ig
[!] Cannot load main LinkedIn page
[*] Obtained new session: xxx
[Notice] Found company ID: 19091732
[*] Using company ID: 19091732
[*] 7 Results Found
[*] Fetching 1 Pages

[*] Fetching page 0/1 with 7 results for Evil Corp LLC
[+] Facebook Login Page loaded successfully [+]
[+] Facebook Login Success [+]

[...]

[+] Twitter Login Page loaded successfully [+]
[-] Twitter Login Failed [-]

[...]

[+] Instagram Login Page loaded successfully [+]
[+] Instagram Login Success [+]

[...]

Results file: SM-Results/Evil Corp LLC-social-mapper.csv
HTML file: SM-Results/Evil Corp LLC-social-mapper.html
venturaEffect commented 5 years ago

Hi @Greenwolf

Again and like often, You're right!

I was just wondering after making it work I stop it and did manually a proxy on de Firefox settings and also on my Linux configuration.

Because of this I got an error. For me it looks everything fine. Anyway, took out the manual proxy setting and now it is running through twitter perfectly.

But, should it not be better to use proxies for this?

Do you know if there are, for face searching, some daily limits for each social account?

Again, thank you a lot @Greenwolf

Greenwolf commented 5 years ago

Hi @zaesarius, I would recommend using a VPN instead of a proxy just to avoid these problems. In each of the modules that uses the firefox web browser at the top you can see i've added some code to tweak firefox settings to block DOM notifications. You could try adding proxy code here if you really need it.

I'm glad we've finally got it working, what an epic issue thread! You really helped me get it all fixed and working again though, so thanks for all your input!

venturaEffect commented 5 years ago

Thank you so much for your time @Greenwolf !!

Following you on twitter! Would like to ask you some ideas that I came through but maybe is to much through here :smile:

Tell me if it's ok to DM

Cheers!