unpoller / unpoller

Application: Collect ALL UniFi Controller, Site, Device & Client Data - Export to InfluxDB or Prometheus
https://unpoller.com
MIT License
2.06k stars 146 forks source link

save_ids not working #235

Closed nchlsgh1 closed 4 years ago

nchlsgh1 commented 4 years ago

In my default configuration with save_ids = true, the events are not captured in InfluxDB, it seems like the table is also not created. Any ideas?

 # UniFi Poller v2 primary configuration file. TOML FORMAT #

###########################################################

[poller]
  # Turns on line numbers, microsecond logging, and a per-device log.
  # The default is false, but I personally leave this on at home (four devices).
  # This may be noisy if you have a lot of devices. It adds one line per device.
  debug = false

  # Turns off per-interval logs. Only startup and error logs will be emitted.
  # Recommend enabling debug with this setting for better error logging.
  quiet = false

  # Load dynamic plugins. Advanced use; only sample mysql plugin provided by default.
  plugins = []

#### OUTPUTS

    # If you don't use an output, you can disable it.

[prometheus]
  disable = true
  # This controls on which ip and port /metrics is exported when mode is "prometheus".
  # This has no effect in other modes. Must contain a colon and port.
  http_listen = "0.0.0.0:9130"
  report_errors = false

[influxdb]
  disable = false
  # InfluxDB does not require auth by default, so the user/password are probably unimportant.
  url  = "http://192.168.0.50:8086"
  #user = "unifipoller"
  #pass = "unifipoller"
  # Be sure to create this database.
  db = "UniFi"
  # If your InfluxDB uses a valid SSL cert, set this to true.
  verify_ssl = false
  # The UniFi Controller only updates traffic stats about every 30 seconds.
  # Setting this to something lower may lead to "zeros" in your data.
  # If you're getting zeros now, set this to "1m"
  interval = "30s"

#### INPUTS

[unifi]
  # Setting this to true and providing default credentials allows you to skip
  # configuring controllers in this config file. Instead you configure them in
  # your prometheus.yml config. Prometheus then sends the controller URL to
  # unifi-poller when it performs the scrape. This is useful if you have many,
  # or changing controllers. Most people can leave this off. See wiki for more.
  dynamic = false

# The following section contains the default credentials/configuration for any
# dynamic controller (see above section), or the primary controller if you do not
# provide one and dynamic is disabled. In other words, you can just add your
# controller here and delete the following section.
[unifi.defaults]
  role       = "Home Controller"
  url        = "https://192.168.0.50:8443"
  user       = "{removed}"
  pass       = "{removed}"
  sites      = ["all"]
  save_ids   = true
  save_dpi   = false
  save_sites = true
  verify_ssl = false

# The following is optional and used for configurations with multiple controllers.

# You may repeat the following section to poll multiple controllers.
#[[unifi.controller]]
  # Friendly name used in dashboards. Uses URL if left empty; which is fine.
  # Avoid changing this later because it will live forever in your database.
  # Multiple controllers may share a role. This allows grouping during scrapes.
  #role = ""
  #url = "https://127.0.0.1:8443"

  # Make a read-only user in the UniFi Admin Settings, allow it access to all sites.
  #user = "unifipoller"
  #pass = "4BB9345C-2341-48D7-99F5-E01B583FF77F"

  # If the controller has more than one site, specify which sites to poll here.
  # Set this to ["default"] to poll only the first site on the controller.
  # A setting of ["all"] will poll all sites; this works if you only have 1 site too.
  #sites = ["all"]

  # Enable collection of Intrusion Detection System Data (InfluxDB only).
  # Only useful if IDS or IPS are enabled on one of the sites.
  #save_ids = false

  # Enable collection of Deep Packet Inspection data. This data breaks down traffic
  # types for each client and site, it powers a dedicated DPI dashboard.
  # Enabling this adds roughly 150 data points per client.  That's 6000 metrics for
  # 40 clients.  This adds a little bit of poller run time per interval and causes
  # more API requests to your controller(s). Don't let these "cons" sway you:
  # it's cool data. Please provide feedback on your experience with this feature.
  #save_dpi = false

  # Enable collection of site data. This data powers the Network Sites dashboard.
  # It's not valuable to everyone and setting this to false will save resources.
  #save_sites = true

  # If your UniFi controller has a valid SSL certificate (like lets encrypt),
  # you can enable this option to validate it. Otherwise, any SSL certificate is
  # valid. If you don't know if you have a valid SSL cert, then you don't have one.
  #verify_ssl = false
davidnewhall commented 4 years ago

The IDS feature in poller doesn't produce any useful data. It will either be fixed or removed in the future.

davidnewhall commented 4 years ago

I'll look into this further when I look into #220. (reminder to me)

davidnewhall commented 4 years ago

I made some changes to this data, and I'm now just waiting for a new IDS event to happen. If you want to try it yourself the docker tag is events.

nchlsgh1 commented 4 years ago

Thanks David, let me test it on my end too

nchlsgh1 commented 4 years ago

Tested it out by manually triggering IPS events, excellent work done and looks great! Is the category type available as well(DNS,CIArmy,Dshield)? Some screenshots of it in action 👍

Will this be merged into Latest? Or just in the events branch(Then i'll migrate over my setup from latest to the events branch) UnifiCtrl_Chronograf Grafana Grafana Worldmap

Grafana_Test
davidnewhall commented 4 years ago

Still not complete, but thank you for trying it out! This will eventually be merged into master and produced into the latest tag on docker hub. I'm still trying to figure out why DPI collection stopped working (a bug).

As for the category name, it is there, it's just not in the same format as seen on the UniFi UI. The column name is catname.