edoardottt / cariddi

Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more
https://edoardoottavianelli.it
GNU General Public License v3.0
1.49k stars 152 forks source link

JSON lines aggregate results #115

Closed edoardottt closed 11 months ago

edoardottt commented 1 year ago

When crawling a target and searching for info, when multiple matches are in the same URL the JSON struct holds two distinct elements instead of one element using an array:

Now:

{
  "url": "http://testphp.vulnweb.com/index.php",
  "method": "GET",
  "status_code": 200,
  "words": 388,
  "lines": 110,
  "content_type": "text/html",
  "matches": {
    "filetype": {
      "extension": "php",
      "severity": 5
    },
    "infos": [
      {
        "name": "Email address",
        "match": "wvs@acunetix.com"
      },
      {
        "name": "HTML comment",
        "match": "<!-- InstanceEndEditable -->"
      },
      {
        "name": "HTML comment",
        "match": "<!-- here goes headers headers -->"
      },
      {
        "name": "HTML comment",
        "match": "<!-- end masthead -->"
      },
      {
        "name": "HTML comment",
        "match": "<!-- begin content -->"
      },
      {
        "name": "HTML comment",
        "match": "<!--end content -->"
      },
      {
        "name": "HTML comment",
        "match": "<!--end navbar -->"
      },
      {
        "name": "HTML comment",
        "match": "<!-- InstanceEnd -->"
      }
    ]
  }
}

Desired output:

{
  "url": "http://testphp.vulnweb.com/index.php",
  "method": "GET",
  "status_code": 200,
  "words": 388,
  "lines": 110,
  "content_type": "text/html",
  "matches": {
    "filetype": {
      "extension": "php",
      "severity": 5
    },
    "infos": [
      {
        "name": "Email address",
        "match": [
          "wvs@acunetix.com"
        ]
      },
      {
        "name": "HTML comment",
        "match": [
          "<!-- InstanceEndEditable -->",
          "<!-- here goes headers headers -->",
          "<!-- end masthead -->",
          "<!-- begin content -->",
          "<!--end content -->",
          "<!--end navbar -->",
          "<!-- InstanceEnd -->"
        ]
      }
    ]
  }
}

cc @ocervell what do you think?

ocervell commented 1 year ago

Yes, it would be a good improvement indeed !

ocervell commented 1 year ago

@edoardottt would you mind releasing a tag with the JSON lines support we've added ? I need it for integration with an upcoming security tool that will be released in the near future :)

edoardottt commented 1 year ago

@edoardottt would you mind releasing a tag with the JSON lines support we've added ? I need it for integration with an upcoming security tool that will be released in the near future :)

released 2 weeks ago :) https://github.com/edoardottt/cariddi/releases/tag/v1.3.1

ocervell commented 1 year ago

Not sure how I missed it ! Thanks a lot !