google / gcp_scanner

A comprehensive scanner for Google Cloud
Apache License 2.0
304 stars 95 forks source link

bug: Fixes parallelization and improves output logic #284

Closed mshudrak closed 10 months ago

mshudrak commented 10 months ago
  1. Fixes #267 when gcp_scanner would hang indefinetely
  2. Output file does not contain projects parent node anymore
  3. Minor code formating using pyformat

Fixes #271. GCP Scanner does not rely on multiprocessing and future frameworks anymore. We use threads and implement our own logic to handle number workers count.

Added new option to control number of threads.

mshudrak commented 10 months ago

@yahia3200 FYI: I changed the output file format a bit by removing projects {}. The old one did not make sense. Could you please reflect that in the visualization tool (It will likely fail to parse the new one). Thanks!

The new format is like this:

{
  "project_info": {
    "projectNumber": "1076721540910",
    "projectId": "test-gcp-scanner-2",
    "lifecycleState": "ACTIVE",
    "name": "test-gcp-scanner-2",
    "createTime": "2023-04-26T18:09:13.431Z"
  },
  "service_account_chain": [],
  "current_service_account": "mshudrak@google.com",
  "token_scopes": [
    "openid",
    "https://www.googleapis.com/auth/userinfo.email",
    "https://www.googleapis.com/auth/cloud-platform",
    "https://www.googleapis.com/auth/appengine.admin",
    "https://www.googleapis.com/auth/compute",
    "https://www.googleapis.com/auth/accounts.reauth"
  ],
  "service_account_edges": {},
  "dns_policies": [
    {
      "id": "2473340462499062447",
      "name": "test-policy",
      "enableInboundForwarding": true,
      "description": "A test policy",
      "networks": [
        {
          "networkUrl": "https://compute.googleapis.com/compute/v1/projects/test-gcp-scanner-2/global/networks/test-vpc",
          "kind": "dns#policyNetwork"
        }
      ],
      "enableLogging": false,
      "kind": "dns#policy"
    }
  ],
  ...
}

Basically, we are not anymore nesting into projects. Let me know if this is a major change. I can revert it back if needed.

yahia3200 commented 10 months ago

@yahia3200 FYI: I changed the output file format a bit by removing projects {}. The old one did not make sense. Could you please reflect that in the visualization tool (It will likely fail to parse the new one). Thanks!

The new format is like this:

{
  "project_info": {
    "projectNumber": "1076721540910",
    "projectId": "test-gcp-scanner-2",
    "lifecycleState": "ACTIVE",
    "name": "test-gcp-scanner-2",
    "createTime": "2023-04-26T18:09:13.431Z"
  },
  "service_account_chain": [],
  "current_service_account": "mshudrak@google.com",
  "token_scopes": [
    "openid",
    "https://www.googleapis.com/auth/userinfo.email",
    "https://www.googleapis.com/auth/cloud-platform",
    "https://www.googleapis.com/auth/appengine.admin",
    "https://www.googleapis.com/auth/compute",
    "https://www.googleapis.com/auth/accounts.reauth"
  ],
  "service_account_edges": {},
  "dns_policies": [
    {
      "id": "2473340462499062447",
      "name": "test-policy",
      "enableInboundForwarding": true,
      "description": "A test policy",
      "networks": [
        {
          "networkUrl": "https://compute.googleapis.com/compute/v1/projects/test-gcp-scanner-2/global/networks/test-vpc",
          "kind": "dns#policyNetwork"
        }
      ],
      "enableLogging": false,
      "kind": "dns#policy"
    }
  ],
  ...
}

Basically, we are not anymore nesting into projects. Let me know if this is a major change. I can revert it back if needed.

fixed with #285