projectdiscovery / katana

A next-generation crawling and spidering framework.
MIT License
11.26k stars 593 forks source link

predefined fields to control output #47

Closed ehsandeep closed 2 years ago

ehsandeep commented 2 years ago

Please describe your feature request:

CLI Option:

   -f, -field           field to display in output (fqdn,rdn,url,rurl,path,file,key,value,kv) (default url)

Example:

Field Example
url (default) https://policies.google.com/terms/file.php?hl=en-IN&fg=1
rurl (root url) https://policies.google.com
path /terms/file.php?hl=en-IN&fg=1
file file.php
key hl,fg
value en-IN,1
kv hl=en-IN&fg=1
fqdn policies.google.com
rdn google.com

Example run:

echo https://example.com | ./katana -f path -silent

/domains
/protocols
/numbers
/about
/go/rfc2606
/go/rfc6761
/http://www.icann.org/topics/idn/
/http://www.icann.org/
/domains/root/db/xn--kgbechtv.html
/domains/root/db/xn--hgbk6aj7f53bba.html
/domains/root/db/xn--0zwm56d.html
/domains/root/db/xn--g6w251d.html
/domains/root/db/xn--80akhbyknj4f.html
/domains/root/db/xn--11b5bs3a9aj6g.html
/domains/root/db/xn--jxalpdlp.html
/domains/root/db/xn--9t4b11yi5a.html
/domains/root/db/xn--deba0ad.html
/domains/root/db/xn--zckzah.html
/domains/root/db/xn--hlcj6aya9esc7a.html
/assignments/special-use-domain-names
/domains/root
/domains/root/db
/domains/root/files
/domains/root/manage
/domains/root/help
/domains/root/servers
/domains/int
/domains/int/manage
/domains/int/policy
/domains/arpa
/domains/idn-tables
/procedures/idn-repository.html
/dnssec
/dnssec/files
/dnssec/ceremonies
/dnssec/procedures
/dnssec/tcrs
/dnssec/archive
/domains/reserved
/abuse
/time-zones
/about/presentations
/reports
/performance
/reviews
/about/excellence
/contact
/_js/jquery.js
/_js/iana.js

This will be similar to uncover implementation of uncover - https://github.com/projectdiscovery/uncover#field-format

Describe the use case of this feature:

parthmalhotra commented 2 years ago

Also a field for directories with a trailing slash :

Field Example
dir https://policies.google.com/terms/

Example run:

echo https://example.com | ./katana -f dir -silent

/go/
/assignments/
/domains/root/
/domains/int/
/procedures/
/dnssec/
/domains/
/about/
/_js/