CycodeLabs / raven

CI/CD Security Analyzer
Apache License 2.0
624 stars 33 forks source link

refactor(tests): Make Integration tests dynamic and Json Snapshot Based. #120

Closed oreenlivnicode closed 1 year ago

oreenlivnicode commented 1 year ago

Created organization - RavenIntegrationTests. It will contain only the repositories we want to test.

Refactor the way we test integrations. We will have a snapshot json file under tests/integration/structures_json/ of the neo4j structure. To test all the relationships and node properties as expected, we will convert graph objects to json structure using utilities I've added and compare it to the snapshot object. It will assert every node/relationship remains the same.

This allows us to scale up our tests, and to test complex scenarios with no effort. This is an example of snapshot file

{
    "nodes": [{
        "path": "actions/checkout",
        "using": "node20",
        "inputs": ["repository", "ref", "token", "ssh-key", "ssh-known-hosts", "ssh-strict", "persist-credentials", "path", "clean", "filter", "sparse-checkout", "sparse-checkout-cone-mode", "fetch-depth", "fetch-tags", "show-progress", "lfs", "submodules", "set-safe-directory", "github-server-url"],
        "name": "Checkout",
        "is_public": true,
        "_id": "d35e7df441120da9624b8c11e36151be",
        "url": "https://github.com/actions/checkout/tree/main/action.yml",
        "labels": ["CompositeAction"]
    },
],
    "relationships": [{
        "start_node": "83b34c58d9a3220059a67e3671ff97f2",
        "type": "ACTION",
        "end_node": "d35e7df441120da9624b8c11e36151be"
    },
 ]
}
oreenlivnicode commented 1 year ago

@alex-ilgayev @elad-pticha - Check out new integration_consts.py structure. The test_graph_structures.py file is set up to have a single, generic test that can be used across various integration tests.

Alternatively, we could break down the configuration dictionaries and include each specific configuration in its corresponding JSON structure file. This way, the configuration for a given test would be retrieved directly from its associated JSON file.