[BUG] json outputs and exports missing encoded templates

blockisec commented 1 month ago

Is there an existing issue for this?

[X] I have searched the existing issues.

Current Behavior

There is the -omit-template flag which should disable adding encoded template to the JSON-based outputs. However, even without using this no encoded templates are available anymore.

Expected Behavior

have encoded templates

Steps To Reproduce

run nuclei with -jsonl or json export. the that encoded template is missing

Relevant log output

No response

Environment

- OS: Blackarch
- Nuclei: v3.3.2 
- Go: 1.23.1

Anything else?

No response

blockisec commented 1 month ago

just commented out the classification key and this brings the encoded template back. Tested with wordpress-detect.

RamanaReddy0M commented 1 month ago

✗ nuclei -t basic.yaml -u example.com -silent -j -o out.jsonl
{"template-id":"basic-check","template-path":"/Users/ramana/projectdiscovery-workspace/nuclei/cmd/nuclei/basic.yaml","template-encoded":"aWQ6IGJhc2ljLWNoZWNrCgppbmZvOgogIG5hbWU6IENoZWNrIGZvciAyMDAgT0sgc3RhdHVzCiAgYXV0aG9yOiByYW1hbmEKICBzZXZlcml0eTogaW5mbwogIGRlc2NyaXB0aW9uOiBDaGVja3MgaWYgdGhlIHRhcmdldCByZXR1cm5zIGEgMjAwIE9LIHN0YXR1cyBjb2RlLgogIHRhZ3M6IGh0dHAsaW5mbwoKaHR0cDoKICAtIG1ldGhvZDogR0VUCiAgICBzdG9wLWF0LWZpcnN0LW1hdGNoOiB0cnVlCiAgICBwYXRoOgogICAgICAtICJ7e0Jhc2VVUkx9fSIKCiAgIyAtIHJhdzoKICAjICAgLSB8CiAgIyAgICAgR0VUIGh0dHBzOi8ve3tIb3N0fX0/PT0mJiBIVFRQLzEuMQogICMgICAgIEhvc3Q6IHt7SG9zdH19CiAgIyAgIHVuc2FmZTogdHJ1ZQogICAgbWF0Y2hlcnM6CiAgICAgICMgLSB0eXBlOiBkc2wKICAgICAgIyAgIGRzbDoKICAgICAgIyAgICAgLSBjb250YWlucyhib2R5LCAiXCJlXCIiKQogICAgICAjIC0gdHlwZTogd29yZAogICAgICAjICAgcGFydDogaGVhZGVyCiAgICAgICMgICB3b3JkczoKICAgICAgIyAgICAgLSAiWC1DYWNoZXMiCiAgICAgICMgICAgIC0gImV4cGlyZXNzIgogICAgICAjICAgbmVnYXRpdmU6IHRydWUKICAgICAgIyAgIGNhc2UtaW5zZW5zaXRpdmU6IHRydWUKICAgICAgIyAgIGNvbmRpdGlvbjogYW5kCiAgICAgIC0gdHlwZTogc3RhdHVzCiAgICAgICAgc3RhdHVzOgogICAgICAgICAgLSAyMDAK","info":{"name":"Check for 200 OK status","author":["ramana"],"tags":["http","info"],"description":"Checks if the target returns a 200 OK status code.","severity":"info"},"type":"http","host":"example.com","port":"443","scheme":"https","url":"https://example.com","matched-at":"https://example.com","request":"GET / HTTP/1.1\r\nHost: example.com\r\nUser-Agent: Mozilla/5.0 (Windows NT 6.1; rv:109.0) Gecko/20100101 Firefox/115.0\r\nConnection: close\r\nAccept: */*\r\nAccept-Language: en\r\nAccept-Encoding: gzip\r\n\r\n","response":"HTTP/1.1 200 OK\r\nConnection: close\r\nAccept-Ranges: bytes\r\nAge: 423041\r\nCache-Control: max-age=604800\r\nContent-Type: text/html; charset=UTF-8\r\nDate: Tue, 17 Sep 2024 16:12:59 GMT\r\nEtag: \"3147526947+gzip\"\r\nExpires: Tue, 24 Sep 2024 16:12:59 GMT\r\nLast-Modified: Thu, 17 Oct 2019 07:18:26 GMT\r\nServer: ECAcc (nyd/D16F)\r\nVary: Accept-Encoding\r\nX-Cache: HIT\r\n\r\n\u003c!doctype html\u003e\n\u003chtml\u003e\n\u003chead\u003e\n    \u003ctitle\u003eExample Domain\u003c/title\u003e\n\n    \u003cmeta charset=\"utf-8\" /\u003e\n    \u003cmeta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" /\u003e\n    \u003cmeta name=\"viewport\" content=\"width=device-width, initial-scale=1\" /\u003e\n    \u003cstyle type=\"text/css\"\u003e\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    \u003c/style\u003e    \n\u003c/head\u003e\n\n\u003cbody\u003e\n\u003cdiv\u003e\n    \u003ch1\u003eExample Domain\u003c/h1\u003e\n    \u003cp\u003eThis domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.\u003c/p\u003e\n    \u003cp\u003e\u003ca href=\"https://www.iana.org/domains/example\"\u003eMore information...\u003c/a\u003e\u003c/p\u003e\n\u003c/div\u003e\n\u003c/body\u003e\n\u003c/html\u003e\n","ip":"2606:2800:21f:cb07:6820:80da:af6b:8b2c","timestamp":"2024-09-17T21:42:59.972354+05:30","curl-command":"curl -X 'GET' -H 'Accept: */*' -H 'Accept-Language: en' -H 'User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:109.0) Gecko/20100101 Firefox/115.0' 'https://example.com'","matcher-status":true}

I see template-encoded here! Have an example to reproduce in your case?

blockisec commented 1 month ago

It seems to be related to the cpe field and the :* sequence. I used wordpress-detect and this one does not contain the template-encoded field. But if I remove the above mentioned sequence (or the complete classification tag) , the template-encoded is back.

Edit: Add screenshot with json output and missing encoded template

2024-09-17-1726590273

Edit2: hm. it seems that it doesnt matter what I am changing in the template. Even adding a 2 to the template name brings the encoded-template back. Maybe its related to signing.

RamanaReddy0M commented 1 month ago

@blockisec for the signed templatetemplate_url will be provided instead of template_encoded. When running unsigned template template_encoded will be provided since it's a custom template.

In this case the template_url is missing even if you're executing signed template. Applied fix here.

ref: https://github.com/projectdiscovery/nuclei/issues/4218

projectdiscovery / nuclei