projectdiscovery / katana

A next-generation crawling and spidering framework.
MIT License
11.22k stars 595 forks source link

Can we record the source of the URL in the results #1007

Closed Autumn-27 closed 2 months ago

Autumn-27 commented 2 months ago

Can we record the source of the URL, such as' example. com/app/test ', which was obtained from' example. com/main. js', and record the address of 'example. com/main. js' in the results.

ehsandeep commented 2 months ago

@Autumn-27 it's in the jsonl output, you need to use -j option.

{
  "timestamp": "2024-08-28T13:11:59.381242+05:30",
  "request": {
    "method": "GET",
    "endpoint": "https://myaccount.google.com/yourdata/search?utm_source=googlemenu&fg=1&cctld=com",
    "tag": "header",
    "attribute": "location",
    "source": "https://www.google.com/history/privacyadvisor/search/unauth?utm_source=googlemenu&fg=1&cctld=com"
  },
  "response": {
    "status_code": 200,
    "headers": {
      "x_xss_protection": "0",
      "accept_ch": "Sec-CH-UA-Arch, Sec-CH-UA-Bitness, Sec-CH-UA-Full-Version, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Model, Sec-CH-UA-WoW64, Sec-CH-UA-Form-Factors, Sec-CH-UA-Platform, Sec-CH-UA-Platform-Version",
      "server": "ESF",
      "set_cookie": "NID=517=LQeh710ndeL4LH9wnIC3aGgO99CXRd5J_-AvWo4u1Pq-GPP-R3dkY3wL9qorR8cKl1IIzczWX2RpVp-H52kwWexlEc_5rUT0t4vK6sm7KCayTrQuGmFUL7MrlHvZmWuPmXx2seVSZfwxW1Cy0lmbohEj0mISGK5HKJl5Ty6JblQ; expires=Thu, 27-Feb-2025 07:41:58 GMT; path=/; domain=.google.com; Secure; HttpOnly; SameSite=none",
      "strict_transport_security": "max-age=31536000",
      "cross_origin_opener_policy": "same-origin-allow-popups",
      "x_frame_options": "SAMEORIGIN",
      "content_security_policy": "script-src 'report-sample' 'nonce-Qax3MvjACLOM5eNGjuCxBA' 'unsafe-inline';object-src 'none';base-uri 'self';report-uri /_/PrivacyAdvisorUi/cspreport;worker-src 'self';script-src 'unsafe-inline' 'unsafe-eval' blob: data: 'self' https://apis.google.com https://ssl.gstatic.com https://www.google.com https://www.googletagmanager.com https://www.gstatic.com https://www.google-analytics.com https://ssl.gstatic.com/external_hosted/lottie/lottie_light.js https://support.google.com/inapp/ https://www.google.com/tools/feedback/ https://www.gstatic.com/inproduct_help/ https://www.gstatic.com/support/content/ https://youtube.googleapis.com https://www.youtube.com https://maps.googleapis.com https://www.googleapis.com/appsmarket/v2/installedApps/;report-uri /_/PrivacyAdvisorUi/cspreport/allowlist;require-trusted-types-for 'script';report-uri /_/PrivacyAdvisorUi/cspreport",
      "x_ua_compatible": "IE=edge",
      "alt_svc": "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000",
      "cache_control": "no-cache, no-store, max-age=0, must-revalidate",
      "pragma": "no-cache",