Some scanners not configured in initial Test_Type fixture

[valentijnscholten]

I've noticed that there are some scanners available in defect dojo dropdown or settings.dist.yp, but not in Test_Type.json which gets loaded on initial install. I think we should put them all in there so people are not confused after installing Defect Dojo?

Example:

DEDUPLICATION_ALGORITHM_PER_PARSER = {
    'Checkmarx Scan detailed': DEDUPE_ALGO_UNIQUE_ID_FROM_TOOL,
    'Checkmarx Scan': DEDUPE_ALGO_HASH_CODE,
    'SonarQube Scan detailed': DEDUPE_ALGO_UNIQUE_ID_FROM_TOOL,
    'SonarQube Scan': DEDUPE_ALGO_HASH_CODE,
    'Dependency Check Scan': DEDUPE_ALGO_HASH_CODE,
    'NPM Audit Scan': DEDUPE_ALGO_HASH_CODE,
    'Yarn Audit Scan': DEDUPE_ALGO_HASH_CODE,
    'Whitesource Scan': DEDUPE_ALGO_HASH_CODE,
    'ZAP Scan': DEDUPE_ALGO_HASH_CODE,
    'Qualys Scan': DEDUPE_ALGO_HASH_CODE,
    'PHP Symfony Security Check': DEDUPE_ALGO_HASH_CODE,
    'Clair Scan': DEDUPE_ALGO_HASH_CODE,
    'Clair Klar Scan': DEDUPE_ALGO_HASH_CODE,
    'Veracode Scan': DEDUPE_ALGO_UNIQUE_ID_FROM_TOOL_OR_HASH_CODE,
    # for backwards compatibility because someone decided to rename this scanner:
    'Symfony Security Check': DEDUPE_ALGO_HASH_CODE,
    'DSOP Scan': DEDUPE_ALGO_HASH_CODE,
    'Trivy Scan': DEDUPE_ALGO_HASH_CODE,
    'HackerOne Cases': DEDUPE_ALGO_UNIQUE_ID_FROM_TOOL_OR_HASH_CODE,
}

But possible also some of these:

    SCAN_TYPE_CHOICES = (("", "Please Select a Scan Type"),
                         ("Netsparker Scan", "Netsparker Scan"),
                         ("Burp Scan", "Burp Scan"),
                         ("Nessus Scan", "Nessus Scan"),
                         ("Nmap Scan", "Nmap Scan"),
                         ("Nexpose Scan", "Nexpose Scan"),
                         ("AppSpider Scan", "AppSpider Scan"),
                         ("Veracode Scan", "Veracode Scan"),
                         ("Checkmarx Scan", "Checkmarx Scan"),
                         ("Checkmarx Scan detailed", "Checkmarx Scan detailed"),
                         ("Crashtest Security JSON File", "Crashtest Security JSON File"),
                         ("Crashtest Security XML File", "Crashtest Security XML File"),
                         ("ZAP Scan", "ZAP Scan"),
                         ("Arachni Scan", "Arachni Scan"),
                         ("VCG Scan", "VCG Scan"),
                         ("Dependency Check Scan", "Dependency Check Scan"),
                         ("Dependency Track Finding Packaging Format (FPF) Export", "Dependency Track Finding Packaging Format (FPF) Export"),
                         ("Retire.js Scan", "Retire.js Scan"),
                         ("Node Security Platform Scan", "Node Security Platform Scan"),
                         ("NPM Audit Scan", "NPM Audit Scan"),
                         ("Qualys Scan", "Qualys Scan"),
                         ("Qualys Infrastructure Scan (WebGUI XML)", "Qualys Infrastructure Scan (WebGUI XML)"),
                         ("Qualys Webapp Scan", "Qualys Webapp Scan"),
                         ("OpenVAS CSV", "OpenVAS CSV"),
                         ("Snyk Scan", "Snyk Scan"),
                         ("Generic Findings Import", "Generic Findings Import"),
                         ("Trustwave Scan (CSV)", "Trustwave Scan (CSV)"),
                         ("SKF Scan", "SKF Scan"),
                         ("Clair Klar Scan", "Clair Klar Scan"),
                         ("Bandit Scan", "Bandit Scan"),
                         ("ESLint Scan", "ESLint Scan"),
                         ("SSL Labs Scan", "SSL Labs Scan"),
                         ("Acunetix Scan", "Acunetix Scan"),
                         ("Fortify Scan", "Fortify Scan"),
                         ("Gosec Scanner", "Gosec Scanner"),
                         ("SonarQube Scan", "SonarQube Scan"),
                         ("SonarQube Scan detailed", "SonarQube Scan detailed"),
                         (SCAN_SONARQUBE_API, SCAN_SONARQUBE_API),
                         ("MobSF Scan", "MobSF Scan"),
                         ("Trufflehog Scan", "Trufflehog Scan"),
                         ("Nikto Scan", "Nikto Scan"),
                         ("Clair Scan", "Clair Scan"),
                         ("Brakeman Scan", "Brakeman Scan"),
                         ("SpotBugs Scan", "SpotBugs Scan"),
                         ("AWS Scout2 Scan", "AWS Scout2 Scan"),
                         ("AWS Prowler Scan", "AWS Prowler Scan"),
                         ("IBM AppScan DAST", "IBM AppScan DAST"),
                         ("PHP Security Audit v2", "PHP Security Audit v2"),
                         ("PHP Symfony Security Check", "PHP Symfony Security Check"),
                         ("Safety Scan", "Safety Scan"),
                         ("DawnScanner Scan", "DawnScanner Scan"),
                         ("Anchore Engine Scan", "Anchore Engine Scan"),
                         ("Bundler-Audit Scan", "Bundler-Audit Scan"),
                         ("Twistlock Image Scan", "Twistlock Image Scan"),
                         ("Kiuwan Scan", "Kiuwan Scan"),
                         ("Blackduck Hub Scan", "Blackduck Hub Scan"),
                         ("Blackduck Component Risk", "Blackduck Component Risk"),
                         ("Openscap Vulnerability Scan", "Openscap Vulnerability Scan"),
                         ("Wapiti Scan", "Wapiti Scan"),
                         ("Immuniweb Scan", "Immuniweb Scan"),
                         ("Sonatype Application Scan", "Sonatype Application Scan"),
                         ("Cobalt.io Scan", "Cobalt.io Scan"),
                         ("Mozilla Observatory Scan", "Mozilla Observatory Scan"),
                         ("Whitesource Scan", "Whitesource Scan"),
                         ("Contrast Scan", "Contrast Scan"),
                         ("Microfocus Webinspect Scan", "Microfocus Webinspect Scan"),
                         ("Wpscan", "Wpscan"),
                         ("Sslscan", "Sslscan"),
                         ("JFrog Xray Scan", "JFrog Xray Scan"),
                         ("Sslyze Scan", "Sslyze Scan"),
                         ("Testssl Scan", "Testssl Scan"),
                         ("Hadolint Dockerfile check", "Hadolint Dockerfile check"),
                         ("Aqua Scan", "Aqua Scan"),
                         ("HackerOne Cases", "HackerOne Cases"),
                         ("Xanitizer Scan", "Xanitizer Scan"),
                         ("Outpost24 Scan", "Outpost24 Scan"),
                         ("Burp Enterprise Scan", "Burp Enterprise Scan"),
                         ("DSOP Scan", "DSOP Scan"),
                         ("Trivy Scan", "Trivy Scan"),
                         ("Anchore Enterprise Policy Check", "Anchore Enterprise Policy Check"),
                         ("Gitleaks Scan", "Gitleaks Scan"),
                         ("Choctaw Hog Scan", "Choctaw Hog Scan"),
                         ("Harbor Vulnerability Scan", "Harbor Vulnerability Scan"),
                         ("Github Vulnerability Scan", "Github Vulnerability Scan"),
                         ("Yarn Audit Scan", "Yarn Audit Scan"),
                         ("BugCrowd Scan", "BugCrowd Scan"),
                         ("GitLab SAST Report", "GitLab SAST Report"),
                         ("AWS Security Hub Scan", "AWS Security Hub Scan"),
                         ("GitLab SAST Report", "GitLab SAST Report"),
                         ("HuskyCI Report", "HuskyCI Report"),
                         ("Risk Recon API Importer", "Risk Recon API Importer"),
                         ("DrHeader JSON Importer", "DrHeader JSON Importer"),
                         ("Checkov Scan", "Checkov Scan"),
                         ("kube-bench Scan", "Kube-Bench Scan"),
                         ("CCVS Report", "CCVS Report"))

or these

def import_parser_factory(file, test, active, verified, scan_type=None):
    if scan_type is None:
        scan_type = test.test_type.name
    if scan_type == "Burp Scan":
        parser = BurpXmlParser(file, test)
    elif scan_type == "Burp Enterprise Scan":
        parser = BurpEnterpriseHtmlParser(file, test)
    elif scan_type == "Nessus Scan":
        filename = file.name.lower()
        if filename.endswith("csv"):
            parser = NessusCSVParser(file, test)
        elif filename.endswith("xml") or filename.endswith("nessus"):
            parser = NessusXMLParser(file, test)
    elif scan_type == "Clair Scan":
        parser = ClairParser(file, test)
    elif scan_type == "Nmap Scan":
        parser = NmapXMLParser(file, test)
    elif scan_type == "Nikto Scan":
        parser = NiktoXMLParser(file, test)
    elif scan_type == "Nexpose Scan":
        parser = NexposeFullXmlParser(file, test)
    elif scan_type == "Veracode Scan":
        parser = VeracodeXMLParser(file, test)
    elif scan_type == "Checkmarx Scan":
        parser = CheckmarxXMLParser(file, test)
    elif scan_type == "Checkmarx Scan detailed":
        parser = CheckmarxXMLParser(file, test, 'detailed')
    elif scan_type == "Contrast Scan":
        parser = ContrastCSVParser(file, test)
    elif scan_type == "Crashtest Security JSON File":
        parser = CrashtestSecurityJsonParser(file, test)
    elif scan_type == "Crashtest Security XML File":
        parser = CrashtestSecurityXmlParser(file, test)
    elif scan_type == "Bandit Scan":
        parser = BanditParser(file, test)
    elif scan_type == "ESLint Scan":
        parser = ESLintParser(file, test)
    elif scan_type == "ZAP Scan":
        parser = ZapXmlParser(file, test)
    elif scan_type == "AppSpider Scan":
        parser = AppSpiderXMLParser(file, test)
    elif scan_type == "Arachni Scan":
        parser = ArachniJSONParser(file, test)
    elif scan_type == 'VCG Scan':
        parser = VCGParser(file, test)
    elif scan_type == 'Dependency Check Scan':
        parser = DependencyCheckParser(file, test)
    elif scan_type == 'Dependency Track Finding Packaging Format (FPF) Export':
        parser = DependencyTrackParser(file, test)
    elif scan_type == 'Retire.js Scan':
        parser = RetireJsParser(file, test)
    elif scan_type == 'Node Security Platform Scan':
        parser = NspParser(file, test)
    elif scan_type == 'NPM Audit Scan':
        parser = NpmAuditParser(file, test)
    elif scan_type == 'PHP Symfony Security Check':
        parser = PhpSymfonySecurityCheckParser(file, test)
    elif scan_type == 'Generic Findings Import':
        parser = GenericFindingUploadCsvParser(file, test, active, verified)
    elif scan_type == 'Qualys Scan':
        parser = QualysParser(file, test)
    elif scan_type == 'Qualys Infrastructure Scan (WebGUI XML)':
        parser = QualysInfraScanParser(file, test)
    elif scan_type == 'Qualys Webapp Scan':
        parser = QualysWebAppParser(file, test)
    elif scan_type == "OpenVAS CSV":
        parser = OpenVASUploadCsvParser(file, test)
    elif scan_type == 'Snyk Scan':
        parser = SnykParser(file, test)
    elif scan_type == 'SKF Scan':
        parser = SKFCsvParser(file, test)
    elif scan_type == 'SSL Labs Scan':
        parser = SSLlabsParser(file, test)
    elif scan_type == 'Trufflehog Scan':
        parser = TruffleHogJSONParser(file, test)
    elif scan_type == 'Clair Klar Scan':
        parser = ClairKlarParser(file, test)
    elif scan_type == 'Gosec Scanner':
        parser = GosecScannerParser(file, test)
    elif scan_type == 'Trustwave Scan (CSV)':
        parser = TrustwaveUploadCsvParser(file, test)
    elif scan_type == 'Netsparker Scan':
        parser = NetsparkerParser(file, test)
    elif scan_type == 'PHP Security Audit v2':
        parser = PhpSecurityAuditV2(file, test)
    elif scan_type == 'Acunetix Scan':
        parser = AcunetixScannerParser(file, test)
    elif scan_type == 'Fortify Scan':
        parser = FortifyXMLParser(file, test)
    elif scan_type == 'SonarQube Scan':
        parser = SonarQubeHtmlParser(file, test)
    elif scan_type == 'SonarQube Scan detailed':
        parser = SonarQubeHtmlParser(file, test, 'detailed')
    elif scan_type == SCAN_SONARQUBE_API:
        parser = SonarQubeApiImporter(test)
    elif scan_type == 'MobSF Scan':
        parser = MobSFParser(file, test)
    elif scan_type == 'AWS Scout2 Scan':
        parser = AWSScout2Parser(file, test)
    elif scan_type == 'AWS Prowler Scan':
        parser = AWSProwlerParser(file, test)
    elif scan_type == 'Brakeman Scan':
        parser = BrakemanScanParser(file, test)
    elif scan_type == 'SpotBugs Scan':
        parser = SpotbugsXMLParser(file, test)
    elif scan_type == 'Safety Scan':
        parser = SafetyParser(file, test)
    elif scan_type == 'DawnScanner Scan':
        parser = DawnScannerParser(file, test)
    elif scan_type == 'Anchore Engine Scan':
        parser = AnchoreEngineScanParser(file, test)
    elif scan_type == 'Bundler-Audit Scan':
        parser = BundlerAuditParser(file, test)
    elif scan_type == 'Twistlock Image Scan':
        parser = TwistlockParser(file, test)
    elif scan_type == 'IBM AppScan DAST':
        parser = IbmAppScanDASTXMLParser(file, test)
    elif scan_type == 'Kiuwan Scan':
        parser = KiuwanCSVParser(file, test)
    elif scan_type == 'Blackduck Hub Scan':
        parser = BlackduckHubCSVParser(file, test)
    elif scan_type == 'Blackduck Component Risk':
        parser = BlackduckHubParser(file, test)
    elif scan_type == 'Sonatype Application Scan':
        parser = SonatypeJSONParser(file, test)
    elif scan_type == 'Openscap Vulnerability Scan':
        parser = OpenscapXMLParser(file, test)
    elif scan_type == 'Immuniweb Scan':
        parser = ImmuniwebXMLParser(file, test)
    elif scan_type == 'Wapiti Scan':
        parser = WapitiXMLParser(file, test)
    elif scan_type == 'Cobalt.io Scan':
        parser = CobaltCSVParser(file, test)
    elif scan_type == 'Mozilla Observatory Scan':
        parser = MozillaObservatoryJSONParser(file, test)
    elif scan_type == 'Whitesource Scan':
        parser = WhitesourceJSONParser(file, test)
    elif scan_type == 'Microfocus Webinspect Scan':
        parser = MicrofocusWebinspectXMLParser(file, test)
    elif scan_type == 'Wpscan':
        parser = WpscanJSONParser(file, test)
    elif scan_type == 'Sslscan':
        parser = SslscanXMLParser(file, test)
    elif scan_type == 'JFrog Xray Scan':
        parser = XrayJSONParser(file, test)
    elif scan_type == 'Sslyze Scan':
        parser = SslyzeXmlParser(file, test)
    elif scan_type == 'Testssl Scan':
        parser = TestsslCSVParser(file, test)
    elif scan_type == 'Hadolint Dockerfile check':
        parser = HadolintParser(file, test)
    elif scan_type == 'Aqua Scan':
        parser = AquaJSONParser(file, test)
    elif scan_type == 'HackerOne Cases':
        parser = HackerOneJSONParser(file, test)
    elif scan_type == 'Xanitizer Scan':
        parser = XanitizerXMLParser(file, test)
    elif scan_type == 'Trivy Scan':
        parser = TrivyParser(file, test)
    elif scan_type == 'Outpost24 Scan':
        parser = Outpost24Parser(file, test)
    elif scan_type == 'DSOP Scan':
        parser = DsopParser(file, test)
    elif scan_type == 'Anchore Enterprise Policy Check':
        parser = AnchoreEnterprisePolicyCheckParser(file, test)
    elif scan_type == 'Gitleaks Scan':
        parser = GitleaksJSONParser(file, test)
    elif scan_type == 'Harbor Vulnerability Scan':
        parser = HarborVulnerabilityParser(file, test)
    elif scan_type == 'Github Vulnerability Scan':
        parser = GithubVulnerabilityParser(file, test)
    elif scan_type == 'Choctaw Hog Scan':
        parser = ChoctawhogParser(file, test)
    elif scan_type == 'GitLab SAST Report':
        parser = GitlabSastReportParser(file, test)
    elif scan_type == 'Yarn Audit Scan':
        parser = YarnAuditParser(file, test)
    elif scan_type == 'BugCrowd Scan':
        parser = BugCrowdCSVParser(file, test)
    elif scan_type == 'HuskyCI Report':
        parser = HuskyCIReportParser(file, test)
    elif scan_type == 'CCVS Report':
        parser = CCVSReportParser(file, test)
    elif scan_type == 'AWS Security Hub Scan':
        parser = AwsSecurityFindingFormatParser(file, test)
    elif scan_type == 'Risk Recon API Importer':
        parser = RiskReconParser(file, test)
    elif scan_type == 'DrHeader JSON Importer':
        parser = DrHeaderJSONParser(file, test)
    elif scan_type == 'Checkov Scan':
        parser = CheckovParser(file, test)
    elif scan_type == 'kube-bench Scan':
        parser = KubeBenchParser(file, test)
    else:
        raise ValueError('Unknown Test Type')

[valentijnscholten]

@dsever if you decide to work on something related to enabling/disabling scanners per instance, could you look at maybe tying this a bit more together so they don't get out of sync?

[dsever]

I will see what can I do...

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[valentijnscholten]

@damiencarol is this still relevant? I think the entries get autocreated these days?

[damiencarol]

@valentijnscholten no. and we should remove the fixture also.