Issue with `outputWriter.WriteCallback`

[randolphcyg]

When Nuclei is called multiple times as a go service, outputWriter.WriteCallback only outputs the result of the request after the first service start

I refer to the v2/examples/simple.go file, encapsulate the logic of calling the nuclei engine, and use the NewMockOutputWriter of the testutils package, but only after the service starts, the first call will write the scan result to outputWriter.WriteCallback. Through debug , each call run the engine. execute, but the second call will not output the result, I do not know how to solve it.

Nuclei version:

latest version: 2.9.8

Code:

func Nuclei(targets []string, templatePaths []string, debug bool, crossTags []string, tags, excludeTags goflags.StringSlice, templateThreads int, severities severity.Severities) (results []*output.ResultEvent, err error) {
    cache := hosterrorscache.New(30, hosterrorscache.DefaultMaxHostsCount, nil)
    defer cache.Close()

    progressImpl, _ := progress.NewStatsTicker(0, false, false, false, false, 0)
    reportingClient, _ := reporting.New(&reporting.Options{}, "")
    defer reportingClient.Close()

    results = make([]*output.ResultEvent, 0)
    outputWriter := testutils.NewMockOutputWriter()
    //defer outputWriter.Close()

    outputWriter.WriteCallback = func(event *output.ResultEvent) {
        if len(event.Response) > 10240 {
            event.Response = event.Response[:10240]
        }

        results = append(results, event)
    }

    defaultOpts := types.DefaultOptions()
    protocolstate.Init(defaultOpts)
    protocolinit.Init(defaultOpts)

    defer protocolinit.Close()

    if len(templatePaths) > 0 {
        defaultOpts.Templates = templatePaths
    }
    if templateThreads > 0 {
        defaultOpts.TemplateThreads = templateThreads
    }
    if severities != nil && len(severities) > 0 {
        defaultOpts.Severities = severities
    }
    defaultOpts.JSONL = true
    defaultOpts.Debug = debug
    defaultOpts.Validate = true
    defaultOpts.UpdateTemplates = false
    defaultOpts.Verbose = true
    defaultOpts.EnableProgressBar = true
    if len(excludeTags) > 0 {
        defaultOpts.ExcludeTags = excludeTags
    }
    if len(tags) > 0 {
        defaultOpts.Tags = tags
    }

    interactOpts := interactsh.DefaultOptions(outputWriter, reportingClient, progressImpl)
    interactClient, err := interactsh.New(interactOpts)
    if err != nil {
        err = errors.Wrap(err, "Could not create interact client")
    }
    defer interactClient.Close()

    catalog := disk.NewCatalog(config.GetNucleiTemplatesHome())
    executorOpts := protocols.ExecutorOptions{
        Output:          outputWriter,
        Options:         defaultOpts,
        Progress:        progressImpl,
        Catalog:         catalog,
        IssuesClient:    reportingClient,
        RateLimiter:     ratelimit.New(context.Background(), 150, time.Second),
        Interactsh:      interactClient,
        HostErrorsCache: cache,
        Colorizer:       aurora.NewAurora(true),
        ResumeCfg:       types.NewResumeCfg(),
    }

    defer executorOpts.RateLimiter.Stop()
    engine := core.New(defaultOpts)
    engine.SetExecuterOptions(executorOpts)

    workflowLoader, err := parsers.NewLoader(&executorOpts)
    if err != nil {
        err = errors.Wrap(err, "Could not create workflow loader")
    }
    executorOpts.WorkflowLoader = workflowLoader

    store, err := loader.New(loader.NewConfig(defaultOpts, catalog, executorOpts))
    if err != nil {
        err = errors.Wrap(err, "Could not create loader client")
    }
    store.Load()

    if len(store.Templates()) == 0 {
        err = errors.New("There is no POC template that meets the requirements")
        return
    }

    allTemplates := store.Templates()
    // custom filter fot templates
    filterTemplates := make([]*templates.Template, 0)
    for _, crossTag := range crossTags {
        tmpTemplates, _ := CustomTemplateFilter(allTemplates, crossTag)
        filterTemplates = append(filterTemplates, tmpTemplates...)
    }

    if len(filterTemplates) == 0 {
        err = errors.New("there is no match templates！")
        return
    }
    log.Info("# Matched templates num：", len(filterTemplates))

    inputArgs := make([]*contextargs.MetaInput, 0)
    for _, target := range targets {
        inputArgs = append(inputArgs, &contextargs.MetaInput{Input: target})
    }

    _ = engine.Execute(filterTemplates, &inputs.SimpleInputProvider{Inputs: inputArgs})
    engine.WorkPool().Wait() // Wait for the scan to finish

    return
}

Running log:

// ### first call
2023-07-12 10:57:02.175187      [INFO]  netx/vulnerabilityScan.go:73    [task ID： 2023000018 targets: 192.168.11.199]
2023-07-12 10:57:02.176182      [INFO]  netx/vulnerabilityScan.go:128   [ =============== call Nuclei ===============]
[http://192.168.11.199:22 http://192.168.11.199:8080]
2023-07-12 10:57:04.937856      [INFO]  nuclei-plus/nucleiplus.go:137   [# Matched templates num： 1 ]
[INF] Using Interactsh Server: oast.live
############## scan result num：  1
######## macth scan result： 0 CVE-2022-22963
@@@@ fetched info：   TaskID   2023000018 192.168.11.199 8080

// ### second call
2023-07-12 10:57:25.211144      [INFO]  netx/vulnerabilityScan.go:73    [task ID： 2023000019 targets: 192.168.11.199]
2023-07-12 10:57:25.211144      [INFO]  netx/vulnerabilityScan.go:128   [ =============== call Nuclei ===============]
[http://192.168.11.199:22 http://192.168.11.199:8080]
2023-07-12 10:57:25.303964      [INFO]  nuclei-plus/nucleiplus.go:137   [# Matched templates num： 1 ]

// ### 3rd call
2023-07-12 10:57:42.563658      [INFO]  netx/vulnerabilityScan.go:73    [task ID： 2023000020 targets: 192.168.11.199]
2023-07-12 10:57:42.563658      [INFO]  netx/vulnerabilityScan.go:128   [ =============== call Nuclei ===============]
[http://192.168.11.199:22 http://192.168.11.199:8080]                                                                 
2023-07-12 10:57:42.656469      [INFO]  nuclei-plus/nucleiplus.go:137   [# Matched templates num： 1 ]

Debug:

only the first call print nuclei log [[INF] Using Interactsh Server: oast.live], after debug, confirm each call will run into engine.Execute, now i have no idea..

[randolphcyg]

If call defer interactClient.Close()，results []*output.ResultEvent will be empty when the second sacan ends; If not call defer interactClient.Close()，the upper function will not wait for all the results output; I do not known what factor affects.

I just want to get the ummary output of a scan, so I can record the vulnerability with it's taskID field.

[mnd5756242]

Has this problem been solved?