Issue #10: Fix crawler with base tag

fcavallarin / htcap

htcap is a web application scanner able to crawl single page application (SPA) recursively by intercepting ajax calls and DOM changes.

GNU General Public License v2.0

610 stars 114 forks source link

Closed GuilloOme closed 7 years ago

GuilloOme commented 7 years ago

Big change are:

in core/crawl/lib/urlfinder.py:
- retrieving <base> href and using .urljoin() in UrlHTMLParser for link building
- adding test for UrlFinder
in core/crawl/probe/probe.js:
- using .href instead of .getAttribute('href') so the href will be interpreted by the browser (applying the <base> tag if needed)

Tested on WIVET (updated the score from 0% to 35%)