Warn - open /tmp/nuclei_scanner3696466119/000002.log: no such file or directory, while using nuclei library

[AlexS778]

Nuclei version:

github.com/projectdiscovery/nuclei/v3 v3.2.2

Current Behavior:

While scanning with nuclei library, we are encountering such warnings with any template/host, and there are no results coming (no vulns are being found):

May 08 20:57:51 hostname nuclei_scanner[4013248]: [WRN] [CVE-2024-0204] Could not execute request for https://samplehost [:RUNTIME] got err while executing https://samplehost <- GET https://samplehost giving up after 2 attempts: Get "https://samplehost": open /tmp/nuclei_scanner3696466119/000002.log: no such file or directory
May 08 20:57:51 hostname nuclei_scanner[4013248]: [WRN] [CVE-2024-0204] Could not execute request for https://samplehost [:RUNTIME] got err while executing https://samplehost <- GET https://samplehost giving up after 2 attempts: Get "https://samplehost": [:RUNTIME] ztls fallback failed <- tls: oversized record received with length 20527

Expected Behavior:

No warnings

Steps To Reproduce:

We are using nuclei engine with such settings:

ne, err := nuclei.NewNucleiEngine(
        nuclei.EnableCodeTemplates(),
        nuclei.WithTemplateFilters(nuclei.TemplateFilters{IDs: templateIDs}),
        nuclei.WithTemplatesOrWorkflows(nuclei.TemplateSources{Templates: []string{templatesPath}}),
        nuclei.WithVerbosity(nuclei.VerbosityOptions{
            Verbose:       true,
            Silent:        false,
            Debug:         false,
            DebugRequest:  false,
            DebugResponse: false,
            ShowVarDump:   false,
        }),
        nuclei.WithHeaders([]string{userAgent}),
        nuclei.WithConcurrency(nuclei.Concurrency{
            TemplateConcurrency: 20000,
            HostConcurrency:     20000,
        }),
        nuclei.WithScanStrategy("auto"),
        nuclei.WithGlobalRateLimit(100, time.Second),
    )
    if err != nil 

Then we simply are following the example usage posted here - https://github.com/projectdiscovery/nuclei/blob/dev/lib/README.md We also have checked - there are no restrictions on who can write to /tmp folder and there are enough space on our machine.

Anything else:

We were using nuclei quite successfully for a while, but something has changed and now it doesn't work, but we didn't update anything (nuclei library version staying the same) nor changed our machine configuration. We would appreciate any help on why this could happen.

[Mzack9999]

Potentially related too https://github.com/projectdiscovery/nuclei/issues/5165

[AlexS778]

Hello, also it feels like our problem can be related to this one: https://github.com/projectdiscovery/nuclei/issues/3693

While reviewing directories which were created by nuclei, we found same files that were mentioned in issue above:

ls -a /tmp/nuclei_scanner1134558272
.  ..  000001.log  CURRENT  LOCK  LOG  MANIFEST-000000

Currently, I'm struggling to understand why those files are being created and when. I'm monitoring /tmp directory with fswatch and I don't see any new directories/files being created by nuclei.

So quick summary: nuclei is looking for 0000002.log file in the directories called /tmp/nuclei_scannerXXXXXX, without that file nuclei cannot make a request (not quite sure why). I don't see any pattern why those directories are being created nor every template that we run is requiring that file.

[Mzack9999]

@AlexS778 After investigating it seems like this is related to https://github.com/projectdiscovery/nuclei/issues/5165 - The issue is yet another internal memory reference preventing gc from recycling objects (similarly to https://github.com/projectdiscovery/nuclei/issues/4836). In this case the faulty component is httpclientpool.Dialer that remains hanging around. I'll implement a fix soon as part of https://github.com/projectdiscovery/nuclei/issues/4808

[AlexS778]

Hello @Mzack9999. Thanks for replying and investigating our issue. Just out of curiosity, what seems to be the issue with httpclientpool.Dialer? Why does it need 00002.log to perform the request and how is that related to gc cleaning trouble? I tried to find the correlating code in nuclei library and struggled to find it.

[Mzack9999]

@AlexS778 When using nuclei as SDK, probably with V3 introduction or the more recent https://github.com/projectdiscovery/nuclei/pull/5112, when invoking NucleiEngine.Close(...) the internal global instances of levelDB (used for various shared tasks) are closed. Normally the memory objects holding these instances should be recycled and new ones created upon nuclei.NewNucleiEngine(...), but the httpclientpool.Dialer holds a reference to the closed instance (which got closed) and attempts to read non-existing files as upon closing, they are removed. This is what i believe can be the root cause.