search

mchaffe / cloudprefixes

Recon tool to query cloud prefixes for services associated with an IP address
GNU General Public License v3.0
24 stars 0 forks source link

403 Forbidden #4

Open Anthirian opened 4 weeks ago

Anthirian commented 4 weeks ago

It appears Microsoft uses some kind of automation protection that causes the tool to abort.

$ cloudprefixes -update
2024/10/27 11:45:01 INFO Updating prefixes: GitHub
2024/10/27 11:45:01 INFO Field contains CIDRs: field=Hooks
2024/10/27 11:45:01 INFO Field contains CIDRs: field=Web
2024/10/27 11:45:01 INFO Field contains CIDRs: field=API
2024/10/27 11:45:01 INFO Field contains CIDRs: field=Git
2024/10/27 11:45:01 INFO Field contains CIDRs: field=GithubEnterpriseImporter
2024/10/27 11:45:01 INFO Field contains CIDRs: field=Packages
2024/10/27 11:45:01 INFO Field contains CIDRs: field=Pages
2024/10/27 11:45:01 INFO Field contains CIDRs: field=Importer
2024/10/27 11:45:01 INFO Field contains CIDRs: field=Actions
2024/10/27 11:45:01 INFO Field contains CIDRs: field=ActionsMacos
2024/10/27 11:45:01 INFO Field contains CIDRs: field=Codespaces
2024/10/27 11:45:01 INFO Field contains CIDRs: field=Dependabot
2024/10/27 11:45:01 INFO Field contains CIDRs: field=Copilot
2024/10/27 11:45:01 INFO successfully inserted prefixes count=5173
2024/10/27 11:45:01 INFO Updating prefixes: Azure public
2024/10/27 11:45:01 INFO fetching HTML to find JSON url="https://www.microsoft.com/en-us/download/details.aspx?id=56519"
2024/10/27 11:47:01 status code error: 403 403 Forbidden

After some troubleshooting I found that it is related to fetching the URL displayed in the logs. When doing the same with Curl or in the browser, no 403 is returned but the page is shown normally.

mchaffe commented 2 weeks ago

It appears to be an issue with Akamai and likely bot mitigation. I'll take a look at a workaround