Make possible to resume a crawl with an existing database

fcavallarin / htcap

htcap is a web application scanner able to crawl single page application (SPA) recursively by intercepting ajax calls and DOM changes.

GNU General Public License v2.0

610 stars 114 forks source link

Make possible to resume a crawl with an existing database #16

Closed GuilloOme closed 4 years ago

GuilloOme commented 7 years ago

The actual behavior is:

if no .db file create a new one.
if there is an existing one, create a new one with rename.
if option -w, overwrite the existing one.

Benefit

start and stop a crawl without re-crawl the whole site
having a single db for multiple assessment

segment-srl commented 7 years ago

It's already possible to have multiple assessment on a single database. Every call to "htcap.py scan ..." will generate a new assessment on the same db. What is not possible is to resume the crawling using an existent database. This issue should be a feature request asking for functionality to resume an interrupted crawling.

GuilloOme commented 7 years ago

Cool, I mainly focus on the crawl feature, so I didn't notice that it was possible in scan mode.

I updated the issue name to "resume a crawl" and I started to work on a pull-request (since we need it in our workflow)

Thanks,

GuilloOme commented 7 years ago

I have a branch waiting to be turned in a Pull-Request, but since it depends on #17, I'll wait before summiting it.