search

kpcyrd / sn0int

Semi-automatic OSINT framework and package manager
https://sn0int.readthedocs.io/
GNU General Public License v3.0
1.98k stars 177 forks source link

Recursive unscoping #87

Open cybiere opened 5 years ago

cybiere commented 5 years ago

It would be great to be able to recursively unscope entries, meaning unscoping domain would also unscope related subdomains, IPs, etc...

kpcyrd commented 5 years ago

That's a good point! The tricky part about it is that some structs are not strictly hierarchical, like one subdomain always belongs to one domain, but an ip might belong to multiple subdomains.

If we follow all possible relations recursively you could easily remove a large part of your database from your scope. One possible solution is explicitly whitelisting relations that are safe to follow in recursion. This would be everything with ON DELETE CASCADE set in its relations, and many-to-many tables acting as a barrier. This means when unscoping a domain you would automatically unscope its subdomains and urls, but not the ip addresses since subdomains<->ipaddrs is a many-to-many relation acting as a boundary.

I would consider noscope -r for regular recursion and noscope -rr to cross those boundaries as well on some cases (for example I think this makes sense for the subdomains<->ipaddrs table but doesn't make sense for the breach<->email table).

Would that work for you? There's a full list of all structs here, _id fields are foreign keys and there's a section about many-to-many tables at the bottom. Let me know if you want any specific use-cases covered.

Thanks!

cybiere commented 5 years ago

That would be great this way ! I can't think of specific use cases right now, I just tried sn0int a few days ago and I felt this was missing. I also have a few more ideas, but since I don't know anything about rust I might just open feature suggestions in here ! Thanks, this tool is really nice !