google / gcp_scanner

A comprehensive scanner for Google Cloud
Apache License 2.0
305 stars 95 forks source link

feat: compute GCP projects parallelly #265

Closed peb-peb closed 1 year ago

peb-peb commented 1 year ago

Description

Changes Made

Checklist

Related Issues

242 #245

Additional Notes

To run the above changes - python3 scanner.py -o ../output -k ../output/sakeys -l INFO -wc 8

Screenshot for above implementation- image

mshudrak commented 1 year ago

This is great. Thanks for the PR. I will review it closer to weekend. Could you please fix tests in meanwhile? It seems like it fails acceptance test.

peb-peb commented 1 year ago

@mshudrak I've transferred the main loop to scanner.main(). The tests should be passing, but the error FAILED src/gcp_scanner/test_acceptance.py::test_acceptance - AssertionError: assert 30 == 31 comes.

peb-peb commented 1 year ago

@mshudrak Ready for Review.

I have implemented the impersonation and it works as expected. But, the tests are still failing.

mshudrak commented 1 year ago

I also noticed this error when I try to work with GCP SA key

023-08-15 21:37:53 - INFO - >> current service account: <redacted>
2023-08-15 21:37:53 - INFO - Retrieving projects list
Traceback (most recent call last):
  File "/usr/lib/python3.11/multiprocessing/queues.py", line 244, in _feed
    obj = _ForkingPickler.dumps(obj)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: cannot pickle '_cffi_backend.FFI' object
Traceback (most recent call last):
  File "/usr/lib/python3.11/multiprocessing/queues.py", line 244, in _feed
    obj = _ForkingPickler.dumps(obj)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: cannot pickle '_cffi_backend.FFI' object