edoardottt / cariddi

Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more
https://edoardoottavianelli.it
GNU General Public License v3.0
1.49k stars 152 forks source link

Example Regex File #118

Closed jaikishantulswani closed 1 year ago

jaikishantulswani commented 1 year ago

@edoardottt provide an example format for custom regex file.

edoardottt commented 1 year ago

Hi @jaikishantulswani !

The custom regex file can be something like this cariddi-test-regex.txt:

(?i)([a-zA-Z0-9_.+-]+@[a-zA-Z0-9]+[a-zA-Z0-9-]*\.[a-zA-Z0-9-.]*[a-zA-Z0-9]{2,})

This file contains the golang regex matching emails. Simple as that. I hope this clears your mind. Let me know if there are other doubts.

However in the next releases this is going to be updated with something better.

jaikishantulswani commented 1 year ago

@edoardottt It would be much more helpful if we use this for scanning secrets only with already crawled links like an argument to -only-scan

echo "https://example.com" | cariddi -only-scan

edoardottt commented 1 year ago

I'm not sure what you are trying to suggest... Can u be more specific?

jaikishantulswani commented 1 year ago

@edoardottt like we already have a file containing links crawled from other tool and now we use caridddi to only scan for secrets

echo "https://example.com/x.js" | cariddi -only-scan

cat "links.txt" | cariddi -onlyscan

edoardottt commented 1 year ago

ok... so cariddi not following/crawling anymore links found on the target but only scanning for things in the content. right? If yes, can you open a new issue with detailed description and examples? This will be developed for next releases :)

Thanks for the suggestion, appreciated!🎉

jaikishantulswani commented 1 year ago

and also @edoardottt If I create a regex file like the below it is not working

(?i)([a-zA-Z0-9_.+-]+@[a-zA-Z0-9]+[a-zA-Z0-9-]*\.[a-zA-Z0-9-.]*[a-zA-Z0-9]{2,})
(?i)([\"']?api[_-]?key[\"']?[^\\S\r\n]*[=:][^\\S\r\n]*[\"']?[\\w-]+[\"'])
edoardottt commented 1 year ago

what do you mean is not working? Regexes must be valid Golang regexes to be used, if I try to look for multiple regexes they are working fine. Do u mean the tool is panicking, not finding something that you're sure it should be found or what else?

jaikishantulswani commented 1 year ago

I am testing this on a local hosted file in which these should be found with name 'key'

edoardottt commented 1 year ago

ok, try to test also the regex matching the secret on https://regex101.com/ or similar tools. Just to be sure the regex is okay (remember to select golang regex). Or if you want share the file on pastebin, so I can test it too.

edoardottt commented 1 year ago

@jaikishantulswani have you tried using that website? Do you still have issues?
I plan to add an example file in the documentation, anyway it's basically a file containing golang regexes one per line..

edoardottt commented 1 year ago

@jaikishantulswani https://github.com/edoardottt/cariddi/wiki/External-regex-files