sherlock-project / sherlock

Hunt down social media accounts by username across social networks
https://sherlockproject.xyz
MIT License
59.38k stars 6.81k forks source link

Adopt sqlite with proper caching #2219

Open ppfeister opened 3 months ago

ppfeister commented 3 months ago

Description

Code of Conduct

sdushantha commented 2 months ago

I like your suggestion @ppfeister, it prevents having several .txt files scattered across the file system.

With the results being stored in a sqlite DB, would we add a few extra flags Sherlock so that i can list and remove results from the DB?

ppfeister commented 2 months ago

Proper implementation in my mind would require results to be cached for a standard amount of time (TBD), allowing the result to be overwritten when expired (reduce rate limit hits), with a flag to skip caching new entries, and a flag to ignore presently cached entries.

Say, --no-cache and --force-check or similar.

I didn't see a need for any further options to manipulate the db, but if any should be added for better UX, they should be behind a sub-command of some type rather than even more flags than we already have. Say, sherlock cache --clear and sherlock cache --clear-site or similar. Only issue this specific example raises is ambiguity between cache the command and cache the username, but something similar should be workable.

ppfeister commented 2 months ago

sherlock-cache likely to be added.