lanmaster53 / recon-ng

Open Source Intelligence gathering tool aimed at reducing the time spent harvesting information from open sources.
GNU General Public License v3.0
4.03k stars 637 forks source link

Problem on modules load recon/profiles-profiles/profiler #180

Closed gioguanuo closed 6 months ago

gioguanuo commented 1 year ago

Sorry if I am posting something of stupid. I have this problem on recon-ng [recon-ng][reconnaissance] > modules load recon/profiles-profiles/profiler [recon-ng][reconnaissance][profiler] > options set SOURCE MarkZuckerberg SOURCE => MarkZuckerberg [recon-ng][reconnaissance][profiler] > run

[*] Retrieving https://raw.githubusercontent.com/WebBreacher/WhatsMyName/master/web_accounts_list.json...

Looking Up Data For: Markzuckerberg

[!] Extra data: line 1 column 4 - line 1 column 15 (char 3 - 14). [!] Something broken? See https://github.com/lanmaster53/recon-ng/wiki/Troubleshooting#issue-reporting. [recon-ng][reconnaissance][profiler] >

rpgmaster280 commented 1 year ago

This issue was already reported (along with a fix) here: https://github.com/lanmaster53/recon-ng/issues/177

gioguanuo commented 1 year ago

Sorry sir, I have modified the python script as the instruction but the problem persist. Please what can I investigate? Thank you in advance

from recon.core.module import BaseModule from recon.mixins.threads import ThreadingMixin from urllib.parse import quote_plus

class Module(BaseModule, ThreadingMixin):

meta = {
    'name': 'OSINT HUMINT Profile Collector',
    'author': 'Micah Hoffman (@WebBreacher)',
    'version': '1.0',
    'description': 'Takes each username from the profiles table and searches a variety of web sites for those users. The list of valid sites comes from the parent project at https://git>
    'comments': (
        'Note: The global timeout option may need to be increased to support slower sites.',
        'Warning: Using this module behind a filtering proxy may cause false negatives as some of these sites may be blocked.',
    ),
    'query': 'SELECT DISTINCT username FROM profiles WHERE username IS NOT NULL',
}

def module_run(self, usernames):
    # retrieve list of sites
    # url = 'https://raw.githubusercontent.com/WebBreacher/WhatsMyName/master/web_accounts_list.json'
    url = 'https://raw.githubusercontent.com/WebBreacher/WhatsMyName/main/wmn-data.json'
    self.verbose(f"Retrieving {url}...")
    resp = self.request('GET', url)
    for user in usernames: 
        self.heading(f"Looking up data for: {user}")
        self.thread(resp.json()['sites'], user)

def module_thread(self, site, user):
    d = dict(site)
    if d['valid'] == True:
        self.verbose(f"Checking: {d['name']}")
        url = d['uri_check'].replace('{account}', quote_plus(user))
        resp = self.request('GET', url, allow_redirects=False)
        if resp.status_code == int(d['e_code']):
            self.debug(f"Codes matched {resp.status_code} {d['e_code']}")
            if d['e_string'] in resp.text or d['e_string'] in resp.headers:
                self.insert_profiles(username=user, url=url, resource=d['name'], cat=d['cat'])
                self.query('DELETE FROM profiles WHERE username = ? and url IS NULL', (user,))
lanmaster53 commented 6 months ago

All module related issues, including dependencies, should be raised in the module repository.

A fix for this problem has been merged.