google / gcp_scanner

A comprehensive scanner for Google Cloud
Apache License 2.0
304 stars 95 forks source link

feat : Add functionality in test file to store crawler results #311

Closed shravankshenoy closed 7 months ago

shravankshenoy commented 7 months ago

Description

In the current approach, the test_res file is over written while testing all the crawlers. While this is an efficient approach, it becomes hard to debug at times when the tests fail as developer does not have access to the test_res file which is compared with the sample output.

This PR adds an additional parameter in verify function called store_results. If this parameter is set to True by developer during function call, it stores the result of the test crawler, which can be useful to analyze if test results fail.

Below is an example of how the new verify function can be called.

def test_compute_instance_name(self):
    """Test compute instance name."""
    self.assertTrue(
      verify(
        CrawlerFactory.create_crawler(
          "compute_instances",
        ).crawl(
          PROJECT_NAME,
          ClientFactory.get_client("compute").get_service(self.credentials),
        ),
        "compute_instances",
        True,
        True,
      )
    )

NOTE : In this PR, store_results has been set to True for 2 resources - compute_instances and services

shravankshenoy commented 7 months ago

Hi @mshudrak, based on the experience in #305 where it was difficult to figure out why the tests were failing, I thought of coming with this PR to modify the verify function, so that it is easier for the developer to debug and compare files.

Do let me know your thoughts on this PR, on whether it adds any value or not.

mshudrak commented 7 months ago

Hi, thanks for the PR. We currently just print to console when the test is failing. No need to store that in the fail.

shravankshenoy commented 7 months ago

Got it. Closing this PR.