✨ FEAT: Machine Images Resources

[peb-peb]

Description

Added support for Machine Image Resources in the tool.

Changes Made

Changes have been made in the following files:

• [x] crawl.py
• [x] scanner.py

Checklist

• [x] I have read and followed the contributing guidelines.
• [x] I have tested my changes thoroughly and they work as expected.
• [x] I have added necessary tests for the changes made.
• [x] I have updated the documentation to reflect the changes made.
• [x] My code follows the project's coding style and standards.
• [x] I have added appropriate commit messages and comments for my changes.
• [ ] mark fields as VOLATILE accordingly

[peb-peb]

The output for the Machine Images would look something like:

"machine_images": [
      {
        "kind": "compute#machineImage",
        "id": "3228948560770099620",
        "creationTimestamp": "2023-03-20T05:53:32.008-07:00",
        "name": "test-mi-1",
        "selfLink": "https://www.googleapis.com/compute/v1/projects/qwiklabs-gcp-01-4ac901cbfcdf/global/machineImages/test-mi-1",
        "sourceInstance": "https://www.googleapis.com/compute/v1/projects/qwiklabs-gcp-01-4ac901cbfcdf/zones/us-central1-a/instances/test-vm-1",
        "status": "READY",
        "sourceInstanceProperties": {
          "tags": {
            "items": [
              "http-server",
              "https-server"
            ],
            "fingerprint": "6smc4R4d39I="
          },
          "machineType": "e2-medium",
          "canIpForward": false,
          "networkInterfaces": [
            {
              "kind": "compute#networkInterface",
              "network": "https://www.googleapis.com/compute/v1/projects/qwiklabs-gcp-01-4ac901cbfcdf/global/networks/default",
              "subnetwork": "https://www.googleapis.com/compute/v1/projects/qwiklabs-gcp-01-4ac901cbfcdf/regions/us-central1/subnetworks/default",
              "networkIP": "10.128.0.2",
              "name": "nic0",
              "accessConfigs": [
                {
                  "kind": "compute#accessConfig",
                  "type": "ONE_TO_ONE_NAT",
                  "name": "External NAT",
                  "natIP": "34.30.56.213",
                  "networkTier": "PREMIUM"
                }
              ],
              "fingerprint": "ebuqTuSddxs=",
              "stackType": "IPV4_ONLY"
            }
          ],
          "disks": [
            {
              "kind": "compute#savedAttachedDisk",
              "type": "PERSISTENT",
              "mode": "READ_WRITE",
              "source": "projects/qwiklabs-gcp-01-4ac901cbfcdf/zones/us-central1-a/disks/test-vm-1",
              "deviceName": "test-vm-1",
              "index": 0,
              "boot": true,
              "autoDelete": true,
              "licenses": [
                "projects/707281592825/global/licenses/debian-11-bullseye"
              ],
              "interface": "SCSI",
              "guestOsFeatures": [
                {
                  "type": "UEFI_COMPATIBLE"
                },
                {
                  "type": "VIRTIO_SCSI_MULTIQUEUE"
                },
                {
                  "type": "GVNIC"
                }
              ],
              "diskSizeGb": "10",
              "storageBytes": "559765696",
              "storageBytesStatus": "UP_TO_DATE",
              "diskType": "pd-balanced"
            }
          ],
          "metadata": {
            "kind": "compute#metadata",
            "fingerprint": "1K-9I6bDxc4=",
            "items": [
              {
                "key": "enable-oslogin",
                "value": "true"
              }
            ]
          },
          "serviceAccounts": [
            {
              "email": "118241340375-compute@developer.gserviceaccount.com",
              "scopes": [
                "https://www.googleapis.com/auth/devstorage.read_only",
                "https://www.googleapis.com/auth/logging.write",
                "https://www.googleapis.com/auth/monitoring.write",
                "https://www.googleapis.com/auth/servicecontrol",
                "https://www.googleapis.com/auth/service.management.readonly",
                "https://www.googleapis.com/auth/trace.append"
              ]
            }
          ],
          "scheduling": {
            "onHostMaintenance": "MIGRATE",
            "automaticRestart": true,
            "preemptible": false,
            "provisioningModel": "STANDARD"
          },
          "deletionProtection": false,
          "keyRevocationActionType": "NONE"
        },
        "instanceProperties": {
          "description": "",
          "tags": {
            "items": [
              "http-server",
              "https-server"
            ],
            "fingerprint": "6smc4R4d39I="
          },
          "machineType": "e2-medium",
          "canIpForward": false,
          "networkInterfaces": [
            {
              "kind": "compute#networkInterface",
              "network": "https://www.googleapis.com/compute/v1/projects/qwiklabs-gcp-01-4ac901cbfcdf/global/networks/default",
              "subnetwork": "https://www.googleapis.com/compute/v1/projects/qwiklabs-gcp-01-4ac901cbfcdf/regions/us-central1/subnetworks/default",
              "networkIP": "10.128.0.2",
              "name": "nic0",
              "accessConfigs": [
                {
                  "kind": "compute#accessConfig",
                  "type": "ONE_TO_ONE_NAT",
                  "name": "External NAT",
                  "natIP": "34.30.56.213",
                  "networkTier": "PREMIUM"
                }
              ],
              "fingerprint": "ebuqTuSddxs=",
              "stackType": "IPV4_ONLY"
            }
          ],
          "disks": [
            {
              "kind": "compute#attachedDisk",
              "type": "PERSISTENT",
              "mode": "READ_WRITE",
              "source": "https://www.googleapis.com/compute/v1/projects/qwiklabs-gcp-01-4ac901cbfcdf/zones/us-central1-a/disks/test-vm-1",
              "deviceName": "test-vm-1",
              "index": 0,
              "boot": true,
              "autoDelete": true,
              "licenses": [
                "https://www.googleapis.com/compute/v1/projects/debian-cloud/global/licenses/debian-11-bullseye"
              ],
              "interface": "SCSI",
              "guestOsFeatures": [
                {
                  "type": "UEFI_COMPATIBLE"
                },
                {
                  "type": "VIRTIO_SCSI_MULTIQUEUE"
                },
                {
                  "type": "GVNIC"
                }
              ],
              "diskSizeGb": "10"
            }
          ],
          "metadata": {
            "kind": "compute#metadata",
            "fingerprint": "1K-9I6bDxc4=",
            "items": [
              {
                "key": "enable-oslogin",
                "value": "true"
              }
            ]
          },
          "serviceAccounts": [
            {
              "email": "118241340375-compute@developer.gserviceaccount.com",
              "scopes": [
                "https://www.googleapis.com/auth/devstorage.read_only",
                "https://www.googleapis.com/auth/logging.write",
                "https://www.googleapis.com/auth/monitoring.write",
                "https://www.googleapis.com/auth/servicecontrol",
                "https://www.googleapis.com/auth/service.management.readonly",
                "https://www.googleapis.com/auth/trace.append"
              ]
            }
          ],
          "scheduling": {
            "onHostMaintenance": "MIGRATE",
            "automaticRestart": true,
            "preemptible": false,
            "provisioningModel": "STANDARD"
          },
          "minCpuPlatform": ""
        },
        "savedDisks": [
          {
            "kind": "compute#savedDisk",
            "sourceDisk": "projects/qwiklabs-gcp-01-4ac901cbfcdf/zones/us-central1-a/disks/test-vm-1",
            "storageBytes": "559765696",
            "storageBytesStatus": "UP_TO_DATE"
          }
        ],
        "storageLocations": [
          "us"
        ],
        "totalStorageBytes": "559765696"
      }
    ]

[peb-peb]

Also, are there any other places where the required changes should be made?

• [x] scanner.py
• [x] crawl.py

[peb-peb]

The tests shouldn't be failing because of the above changes. Tests are yet to be added for the given feature.

Can you please give a brief steps on what adding tests for a particular "GCP Scanner Resource feature" should look like? :)

[mshudrak]

It is likely happening because of the change in GCP. Nothing to do with your changes.

[mshudrak]

That's nice. Could you add test for this new resource? You can take a look at #40 already merged.

[peb-peb]

@mshudrak Help Wanted! Can't figure out why the tests are failing? Does it have to do something with how the testing environment is set up?

[mshudrak]

It is failing because GCP is now returning data that we do not expect in tests. It is fine. I will fix it. I will review the commit, don't worry about the tests for now.

[peb-peb]

Ok ;P

Also, would have to mark some fields as VOLATILE.