search

SSWConsulting / SSW.CodeAuditor

SSW Code Auditor V2
6 stars 6 forks source link

🐛 Scheduled Scan - Failing on SSW Website #874

Closed tombui99 closed 5 months ago

tombui99 commented 5 months ago

Cc: @william-liebenberg @tiagov8

Describe the Bug

In the past weeks, the scheduled CodeAuditor scan was failing on SSW Wesbite due to the below error. This was due to some links within SSW Website doesn't response to Head requests which returns empty value, and our Golang link scan engine misses the function to handle it.

Image

Figure: Error scanning on SSW Website

Tasks

More Information

Thanks!

tombui99 commented 5 months ago

@william-liebenberg @wicksipedia

Investigate and fix

Done - The reason of failure is from CodeAuditor running too many requests on SSW Website which brought down the site.

After working with the Website team, they have been able to fix the server and it is now back to work. The root cause was from CodeAuditor hitting an old broken v1 Website link that kept redirecting back to itself, therefore, it put a lot of pressure to the server (>30k requests in the period of 1 hour).

Throttle and delay setting has been added to ensure any website that CodeAuditor scans on can catch some breath. https://github.com/SSWConsulting/SSW.CodeAuditor/pull/877