search

fwdcloudsec / known_aws_accounts

List of known AWS accounts
Apache License 2.0
150 stars 21 forks source link

add tooling to find dead links #8

Open z0ph opened 1 year ago

z0ph commented 1 year ago

Add tooling to identify dead links - First output:

Checking Cloudhealth
Checking SegmentIO
Checking StackDriver
Checking Zencoder
URL: https://support.brightcove.com/using-zencoder-s3
Response Code: 404
----------------------
Checking Datadog
Checking Cloudability
URL: https://developers.cloudability.com/docs/vendor-credentials-end-point
Response Code: 404
----------------------
Checking Rackspace
Checking New Relic
Checking Brightcove
URL: https://support.brightcove.com/using-dynamic-ingest-s3
Response Code: 404
----------------------
Checking CloudCheckr
URL: https://support.cloudcheckr.com/cloudcheckr-api-userguide/cloudcheckr-admin-api-reference-guide/
Response Code: 302
----------------------
Checking SignifAI
Error connecting to URL: https://docs.signifai.io/docs/amazon-web-services
Error message: HTTPSConnectionPool(host='docs.signifai.io', port=443): Max retries exceeded with url: /docs/amazon-web-services (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x1039afd50>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))
----------------------
Checking ParkMyCloud
Checking CenturyLinkCloud
Checking ELB logs
Checking Redshift logs
Checking Billing
Checking skeddly
Checking freshservice
URL: https://support.freshservice.com/support/solutions/articles/207515-creating-a-role-arn-for-integrating-amazon-web-services-aws-in-freshservice
Response Code: 302
----------------------
Checking signalfx
URL: https://signalfx-product-docs.readthedocs-hosted.com/en/latest/getting-started/send-data.html
Response Code: 302
----------------------
Checking cloudsploit
Checking globus
Checking dynatrace
Checking deepsecurity
Checking cloudbreak
Checking teraproc
Error connecting to URL: http://www.teraproc.com/awskey/
Error message: HTTPConnectionPool(host='www.teraproc.com', port=80): Max retries exceeded with url: /awskey/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x103b07550>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))
----------------------
Checking orbitera
Error connecting to URL: https://support.orbitera.com/support/solutions/articles/147040-add-new-aws-accounts
Error message: HTTPSConnectionPool(host='support.orbitera.com', port=443): Max retries exceeded with url: /support/solutions/articles/147040-add-new-aws-accounts (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x103b07550>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))
----------------------
Checking redline13
URL: https://www.redline13.com/blog/aws-setup/
Response Code: 404
----------------------
Checking kochava
Checking instaclustr
Checking CloudTrail
Checking Summit Route
Checking TrendMicro
URL: https://help.deepsecurity.trendmicro.com/Add-Computers/add-aws.html
Response Code: 404
----------------------
Checking Convox
URL: https://convox.com/docs/aws-integration
Response Code: 404
----------------------
Checking Spotinst
Checking Redlock
Checking Sumo Logic
URL: https://help.sumologic.com/03Send-Data/Sources/02Sources-for-Hosted-Collectors/Amazon-Web-Services/Grant-Access-to-an-AWS-Product
Response Code: 302
----------------------
Checking Bridgecrew
Checking Lacework
Checking Onelogin
Checking nOps
URL: https://help.nops.io/manual_setup
Response Code: 404
----------------------
Checking Fivetran
URL: https://fivetran.com/docs/logs/cloudwatch/setup-guide
Response Code: 302
----------------------
Checking Rapid7
Checking Databricks
Checking Threat Stack
Checking Cloudyn
Checking Lucidchart
Checking Workato
Checking Palo Alto Networks
Checking CloudZero
URL: https://www.cloudzero.com/hubfs/CloudZero%20Configuration%20Guide%20-%20Automated.pdf
Response Code: 404
----------------------
Checking Cloudinary
Checking Tenable
Checking Stitch
Checking Emnify
Checking Qualys Cloud View
Checking Auth0
Checking Altus
Checking AlertLogic
Checking CloudConformity
URL: https://github.com/cloudconformity/documentation-api/blob/master/Accounts.md#update-account
Response Code: 404
----------------------
Checking Nessus AWS Connector
Checking Dome9 Arc
Checking FortiCASB
Checking FortiCWP
Checking Azure Sentinel
No source for Azure Sentinel
----------------------
Checking Azure Billing Management
Checking ADC Application Deployment, ADM Delivery Management
Checking CloudManager for CloudVolumes
Checking Axonius.com
No source for Axonius.com
----------------------
Checking Fugue
No source for Fugue
----------------------
Checking CloudPhysics
Checking QRadar
No source for QRadar
----------------------
Checking LogicMonitor
No source for LogicMonitor
----------------------
Checking MVision ePO
Error connecting to URL: https://docs.mcafee.com/bundle/prod-name-n.n.x-guide-type/page/GUID-9B6E696A-78DA-4F41-A0FC-39699DD39639.html
Error message: HTTPSConnectionPool(host='docs.mcafee.com', port=443): Max retries exceeded with url: /bundle/prod-name-n.n.x-guide-type/page/GUID-9B6E696A-78DA-4F41-A0FC-39699DD39639.html (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1002)')))
----------------------
Checking Cloud Workload Protection
Checking Cloud Workload Protection
No source for Cloud Workload Protection
----------------------
Checking Logz.io
No source for Logz.io
----------------------
Checking Xi Frame
No source for Xi Frame
----------------------
Checking CloudWisdom, metricly
No source for CloudWisdom, metricly
----------------------
Checking Rockset
No source for Rockset
----------------------
Checking CloudHiro
Error connecting to URL: https://cloudhiro.com/AWS/AWSRegistrationGuide.php
Error message: HTTPSConnectionPool(host='cloudhiro.com', port=443): Max retries exceeded with url: /AWS/AWSRegistrationGuide.php (Caused by SSLError(CertificateError("hostname 'cloudhiro.com' doesn't match '*.cloudhiro.com'")))
----------------------
Checking Densify
URL: https://www.densify.com/docs/Content/Data_Collection_for_Public_Cloud_Systems/AWS_Data_Collection_Prerequisites_for_an_IAM_Role.htm
Response Code: 404
----------------------
Checking Armor Anywhere
URL: https://docs.armor.com/pages/viewpage.action?pageId=20709565
Response Code: 302
----------------------
Checking Genys
Checking site24x7
No source for site24x7
----------------------
Checking ylastic
Checking qubole
Checking Cloudability
URL: https://developers.cloudability.com/docs/vendor-credentials-end-point
Response Code: 404
----------------------
Checking VManage
Checking Cloud Applicatoin Manager
Checking Cloudaware
No source for Cloudaware
----------------------
Checking Cloud Ranger
No source for Cloud Ranger
----------------------
Checking FoxPass
Checking Nirmata
URL: https://nirmata-documentation.readthedocs.io/en/latest/CloudProviders.html
Response Code: 404
----------------------
Checking GitLab
No source for GitLab
----------------------
Checking Snyk
Checking CloudCraft
No source for CloudCraft
----------------------
Checking JupiterOne
Checking rev.com
URL: https://www.rev.com/api/s3bucketpolicy
Response Code: 404
----------------------
Checking Funnel
Checking Domo
URL: https://knowledge.domo.com/Connect/Connecting_to_Data_with_Connectors/Configuring_Each_Connector/Connectors_for_File_Retrieval/Amazon_S3_AssumeRole_Connector
Response Code: 302
----------------------
Checking Atlas DataLake
No source for Atlas DataLake
----------------------
Checking Upsolver
No source for Upsolver
----------------------
Checking Weave Cloud
Checking ChaosSearch
Checking EDB Postgres
No source for EDB Postgres
----------------------
Checking TheGlobalSolutions.net
No source for TheGlobalSolutions.net
----------------------
Checking wpengine
Checking cloudsqueeze
No source for cloudsqueeze
----------------------
Checking ThingSpace
URL: https://thingspace.verizon.com/resources/documentation/cloudconnector/Getting_Started/Streaming_to_AWS/
Response Code: 302
----------------------
Checking Anodot
Checking MediaMath
URL: https://apidocs.mediamath.com/reporting/log-level-data-service/overview#data-security-and-authorization
Response Code: 404
----------------------
Checking Presidio
Invalid URL: customer IAM policy
Checking Checkpoint Cloudguard
Checking Cisco Umbrella
Checking Cloudflare
Checking [Deprecated] AWS Log delivery Service
Checking Epsagon
URL: https://docs.epsagon.com/docs/faq
Response Code: 404
----------------------
Checking Turbot
URL: https://turbot.com/v5/docs/integrations/aws/import-aws-account
Response Code: 302
----------------------
Checking Qualys AWS EC2 Connector
Checking API Gateway
Checking Slack EKM
Checking SSLMate
Checking SSLMate (Sandbox Site)
ramimac commented 1 year ago

@z0ph Thanks for proposing this!

What action would you imagine us taking when a link is dead?

I don't think we'd want to "age out" accounts, so the link dying likely doesn't change validity of inclusion If we always want a valid link, I would generally prefer proactive automation (for example, archive.org'ing any link that is submitted) vs. reactive

z0ph commented 1 year ago

Hey Rami,

Thanks for reviewing my PR.

My goal was to help maintainers of this repository/initiative with awareness of dead links that reference a specific AccountId.

We could probably construct an archive.org link to replace dead links on the fly. Let us decide this collectively. But at least we are now aware that some links are not working anymore.

0xdabbad00 commented 1 year ago

I do not want to remove items because links no longer work, as folks probably still have trust policies in their environment for the related account IDs, and the purpose of this repo is to let people know what those are. There may be a need for us to identify some accounts as known but no longer valid or various other categories (ex. known malicious), but for now, I think we should just leave these. Some accounts may have new references, while others may be dead (ex. no one should have a trust relationship from my old Summit Route consulting business anymore). Mostly the links have been to ensure that at some point in time there was an admission by a company that they owned the account ID. As account IDs are not re-used, there shouldn't be any reason not to leave these. So for now, I think we should just ignore these.

sdemjanenko commented 1 year ago

@0xdabbad00 a few of the entries that you removed from the Permiso data-set had links, but those links no longer have the account ID. At one point they did, otherwise the link wouldn't have been in the original data-set. Do we want to include a crawled-at time indicating the time when the URL did have the account ID information?