search

fcavallarin / htcap

htcap is a web application scanner able to crawl single page application (SPA) recursively by intercepting ajax calls and DOM changes.
GNU General Public License v2.0
610 stars 114 forks source link

Maybe just some Warning! #9

Closed Indeserpen closed 7 years ago

Indeserpen commented 7 years ago

Hello!

First amazing script, lately got more familiar with it, and now use almost in every security assesment! It helps me to get very clear insight of web application and makes job more easy! Have some ideas and functions what maybe could be added! When ready gonna let you know you!

Second thing i noticed that on large scale crawl, I,m getting this error: Database karta.db initialized, crawl started with 10 threads [==== ] 966 of 7928 pages processed in 67 minutes Exception in thread Thread-9: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/usr/share/htcap/core/crawl/crawler_thread.py", line 62, in run self.crawl() File "/usr/share/htcap/core/crawl/crawler_thread.py", line 212, in crawl probe = self.send_probe(request, errors) File "/usr/share/htcap/core/crawl/crawler_thread.py", line 161, in send_probe probeArray = self.load_probe_json(jsn) File "/usr/share/htcap/core/crawl/crawler_thread.py", line 99, in load_probe_json return json.loads(jsn) File "/usr/lib/python2.7/json/init.py", line 339, in loads return _default_decoder.decode(s) File "/usr/lib/python2.7/json/decoder.py", line 367, in decode raise ValueError(errmsg("Extra data", s, end, len(s))) ValueError: Extra data: line 4 column 1 - line 4 column 249 (char 58 - 306) Maybe its just my system warning! I,m Using Kali Linux but its highly modified and build up for me and maybe because of this!

It seems that there is no impact on work flow, it continues crawling and after when working on database there is no error!

Anyways good script and good luck developing it!

segment-srl commented 7 years ago

Hello! Many thanks for your feedback! I'm always looking for good advices from users, so please feel free to contact me for any feature request, advice ecc.. !

About the warning, I believe that the json returned from phantomjs contains some garbage so the parser just rises an alert. I'm trying to figure out why it happens.

Btw I just pushed some changes so maybe the problem got fixed...

GuilloOme commented 7 years ago

It seems to be the same error as the one in #11

GuilloOme commented 7 years ago

@Indeserpen, what version of PhantomJS do you run in your Kali? Is it the one from the repo?

segment-srl commented 7 years ago

This issue is related to phantomjs build on some linux distros. Using the binary from the officail website should fix the problem. Since phantomjs is no more supported, htcap is now moving to headless chrome so issue similar to this one won't be fixed.

Tren commented 7 years ago

Ok wait for htcap update!