projectdiscovery / nuclei

Fast and customizable vulnerability scanner based on simple YAML based DSL.
https://docs.projectdiscovery.io/tools/nuclei
MIT License
20.2k stars 2.47k forks source link

when use -rc, lost results #2603

Closed hktalent closed 1 year ago

hktalent commented 2 years ago

@ehsandeep 1、create web server like router.POST("/:id/_doc", lib.SaveRsult) 2、nuclei -rc set ReportingConfig

3、now found lost resut

image

not found the result

image

I have observed that my configuration is correct as only a small number of results are missing, not sure if I can investigate

other is ok

image
hktalent commented 2 years ago
image

leveldb: closed

hktalent commented 2 years ago
image
hktalent commented 1 year ago

@ehsandeep -rc can not work, nuclei -v
2.7.9

cmd

 nuclei -rate-limit 300  -bulk-size 64 -concurrency 64   -rc  $PWD/config/nuclei_esConfig.yaml -duc -t $PWD/config/nuclei-templates/network/detection/weblogic-iiop-detect.yaml  -tags weblogic -u test.com -vv -debug

cat $PWD/config/nuclei_esConfig.yaml 

config: $PWD/config/nuclei_esConfig.yaml

elasticsearch:
  # IP for elasticsearch instance
  ip: 127.0.0.1
  # Port is the port of elasticsearch instance
  port: 8081
  # IndexName is the name of the elasticsearch index
  index-name: nuclei_index
  # SSL enables ssl for elasticsearch connection
  ssl: true
  # SSLVerification disables SSL verification for elasticsearch
  ssl-verification: false
  # Username for the elasticsearch instance
  username: elastic
  # Password is the password for elasticsearch instance
  password: testnmanp                                      

curl https://127.0.0.1:8081/ is http2

* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200 
< accept-ranges: bytes
< content-type: text/html; charset=utf-8
< last-modified: Sat, 26 Nov 2022 07:43:29 GMT
< x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
< content-length: 3548
< date: Fri, 02 Dec 2022 08:03:38 GMT

router.POST("/:type/_doc/:id", util1.AuthHandler, lib.SaveRsult)

ehsandeep commented 1 year ago

@hktalent could you clarify issue and share the details / nuclei cmds, in a manner that issue can be reproduced?

hktalent commented 1 year ago

@ehsandeep -rc $PWD/config/nuclei_esConfig.yaml(The configuration file is as described above)

I specified the http server with this parameter Use gin to realize the reception of simulated es

Then I don't get any request from http server The current http server is http2.0, I don’t know if this is the reason

but, if I integrate from the code and set reportconfig, it can work normally

hktalent commented 1 year ago

@ehsandeep i found the reason It turned out to be my anti-crawler, the interceptor identified it as an unsafe request, and intercepted it Recommendations Recommendations Set a regular user-agent instead of using the default Go-http-client

 "POST /nuclei_index/_doc HTTP/1.1 404 24.935µs "Go-http-client/1.1" "
Screenshot 2022-12-03 at 14 48 29
jimen0 commented 1 year ago

The correct way to fix this is to allow-list the nuclei exporter on the elasticsearch side, not to hide the Go client.

hktalent commented 1 year ago

@jimen0 But that's not friendly If in a complex cluster network environment Traffic will pass through multiple layers of firewalls. In this case, adjusting the user-agent of nuclei is the least expensive