search

saeeddhqan / Maryam

Maryam: Open-source Intelligence(OSINT) Framework
GNU General Public License v3.0
1.03k stars 166 forks source link

[New Feature] Checking the enumerated email assocaited with a domain in data leak dumps #74

Closed vikas-kundu closed 3 years ago

vikas-kundu commented 3 years ago

Emails enumerated while scanning can be checked from haveibeenpwned.com. Their API can be used for this purpose same. Reference: https://haveibeenpwned.com/API/v2

vikas-kundu commented 3 years ago

Can I work on this module. Where domain would be necessarily required to see if its been a part of a data breach and email part is optional?

saeeddhqan commented 3 years ago

Does it need an API key? We do not support sources that need an API key.

vikas-kundu commented 3 years ago

Unfortunately it does. I am looking for any free resource if available. Otherwise should I try scraping from haveibeenpwned ?

saeeddhqan commented 3 years ago

I'm afraid. But it isn't good. You can work on the Github Search module if you want to contribute. We need a Github search module, which interacts with Github by user cookie.

rachejazz commented 3 years ago

@saeeddhqan i thought @ktg-123 already is done with it? iirc, it was assigned to me. Waiting for the new core and the PRs merged. I will add them :)

vikas-kundu commented 3 years ago

@saeeddhqan So, interestingly while researching for the same, i found their xmlHttp request API. So managed to put together a script which can check emails from data breaches using their API via python's cloudscraper. This is the script . So, should I work on a module for the same?

saeeddhqan commented 3 years ago

Yes, you can. but don't use cloudscraper.

vikas-kundu commented 3 years ago

@saeeddhqan actually this endpoint is present within the haveibeenpwned website. So, cloudscraper is necessary because it is protected by Cloudflare like the rest of the website.

rachejazz commented 3 years ago

Did cloudscraper successfully bypass the check? I ran it last week, it said Cloudflare changed it's check params again

vikas-kundu commented 3 years ago

I wrote this script only a few hours ago. Was able to scrap successfully for me.

vikas-kundu commented 3 years ago

@saeeddhqan i have complete the email_pwned module. Should i issue a pull request ?

saeeddhqan commented 3 years ago

yes, please