Expose header `key: value` as protocol response variable in headless protocol

[ehsandeep]

Similar to http engine, we can expose header key: value as a variable to use with dsl matchers / extractors.

Here is an example from existing http engine -

id: basic-example

info:
  name: Test HTTP Template
  author: pdteam
  severity: info

http:
  - method: GET
    path:
      - "{{BaseURL}}"

    extractors:
      - type: dsl
        dsl:
          - content_type

echo scanme.sh | ./nuclei -t aa.yaml -v -svd
                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v2.9.6

        projectdiscovery.io

[INF] Current nuclei version: v2.9.6 (latest)
[INF] Current nuclei-templates version: v9.5.2 (latest)
[INF] New templates added in latest release: 50
[INF] Templates loaded for current scan: 1
[INF] Targets loaded for current scan: 1
[INF] Running httpx on input host
[INF] Found 1 URL from httpx
[DBG] Final Protocol request variables: 
    1. File => 
    2. BaseURL => https://scanme.sh
    3. Host => scanme.sh
    4. RootURL => https://scanme.sh
    5. DN => scanme
    6. TLD => sh
    7. Input => https://scanme.sh
    8. SD => 
    9. Hostname => scanme.sh
    10. ip => 
    11. Scheme => https
    12. Path => 
    13. Port => 443
    14. RDN => scanme.sh
    15. FQDN => scanme.sh

[VER] [basic-example] Sent HTTP request to https://scanme.sh
[DBG] Protocol response variables: 
    1. BaseURL => https://scanme.sh
    2. Port => 443
    3. RDN => scanme.sh
    4. curl-command => curl -X 'GET' -d '' -H 'A .... 7.36' 'https://scanme.sh'
    5. content_type => text/plain; charset=utf-8
    6. template-info => {Test HTTP Template pdteam    {info} map[] <nil> }
    7. Host => scanme.sh
    8. File => 
    9. template-path => /Users/geekboy/Github/nuclei/v2/cmd/nuclei/aa.yaml
    10. host => https://scanme.sh
    11. Path => 
    12. RootURL => https://scanme.sh
    13. header => HTTP/1.1 200 OK  Connecti .... Jun 2023 09:55:38 GMT    
    14. request => GET / HTTP/1.1  Host: sca .... Accept-Encoding: gzip    
    15. SD => 
    16. Scheme => https
    17. Input => https://scanme.sh
    18. FQDN => scanme.sh
    19. status_code => 200
    20. duration => 0.2552055
    21. interactsh-server => 
    22. TLD => sh
    23. date => Thu, 08 Jun 2023 09:55:38 GMT
    24. type => http
    25. matched => https://scanme.sh
    26. content_length => 2
    27. template-id => basic-example
    28. response => HTTP/1.1 200 OK  Connecti .... n 2023 09:55:38 GMT    ok
    29. all_headers => HTTP/1.1 200 OK  Connecti .... Jun 2023 09:55:38 GMT    
    30. DN => scanme
    31. Hostname => scanme.sh
    32. ip => 128.199.158.128
    33. body => ok

[basic-example] [http] [info] https://scanme.sh [text/plain; charset=utf-8]

Originally posted by @ehsandeep in https://github.com/projectdiscovery/nuclei/pull/3794#pullrequestreview-1469468441

[ehsandeep]

Example headless template:

id: headless-header-key

info:
  name: headless header key
  author: pdteam
  severity: info

headless:
  - steps:
      - args:
          url: "{{BaseURL}}"
        action: navigate
      - action: waitload

    extractors:
      - type: dsl
        dsl:
          - content_type

Expected result:

echo scanme.sh | ./nuclei -t test.yaml -headless -silent

[headless-header-key] [headless] [info] https://scanme.sh [text/plain; charset=utf-8]

[Mzack9999]

Depends on https://github.com/projectdiscovery/nuclei/issues/3846