projectdiscovery / katana

A next-generation crawling and spidering framework.
MIT License
11.1k stars 590 forks source link

headless mod doesn't work properly with SPA (Single Page Application) #502

Open lebik opened 1 year ago

lebik commented 1 year ago

It seems like katana doesn't work properly with SPA (Single Page Application) like AngularJS

katana version:

v1.0.2

Current Behavior:

Results (number of URLs) without headless mode and with headless mode are the same

Expected Behavior:

Results (number of URLs) in headless mode should contain more URLs compared to running without headless mode

Steps To Reproduce:

Experiment #1 - crawling SPA without headless mode

katana -d 6 -jc -timeout 40 -f url -H headers.txt -aff -ef 'css,gif,png' -u 'http://testspa.local'

Experiment #2 - crawling SPA in headless mode

# login and save session
/usr/bin/chromium --user-data-dir=/home/kali/testspa

# run katana in headless mode
katana -d 6 -jc -timeout 40 -f url -H headers.txt -aff -ef 'css,gif,png' -u 'http://testspa.local' -hl -noi -scp /usr/bin/chromium -cdd /home/kali/testspa 

Anything else:

In my example, I need to login first before I can start crawling, so without headless mode I pass the cookie file (-H headers.txt), while in headless mode I am logging in manually and then passing the -cdd option

Mzack9999 commented 9 months ago

Related https://github.com/projectdiscovery/katana/issues/579