viewmithun commented 6 years ago

Hi Team,

I am using Nessusbeat-6.0.0-alpha and Elasticsearch 6.0. After running, ./nessusbeat -c nessusbeat.yml -e -d "*" I get the following error,

2018/06/16 06:04:19.764832 client.go:462: WARN Can not index event (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"illegal_argument_exception","reason":"field name cannot be an empty string"}} 2018/06/16 06:04:19.764881 client.go:462: WARN Can not index event (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"illegal_argument_exception","reason":"field name cannot be an empty string"}}

Here is my nessusbeat.yml file content in brief,

nessusbeat: report_path: /opt/nessus/var/nessus/users/admin/reports

cacert_path:

api_url:

api_username:

api_password:

timestamp_fields:

name: nessusbeat

output.elasticsearch:

Array of hosts to connect to.

hosts: ["localhost:9200"]

hosts: ["127.0.0.1:9200"]

Kindly assist in resolving the issue. Seems like Elasticsearch doesn't like to have dots, dash in Nessusbeat Index pattern.

viewmithun commented 6 years ago

Hi there,

Any updates WRT this issue?

Thanks, Mithun

viewmithun commented 6 years ago

Hi there,

Any updates WRT this issue?

Thanks, Mithun

darvid commented 6 years ago

Need more details and possibly additional logs. In your nessusbeat configuration, did you specify api_url, api_username, and api_password? These are required fields. Note that nessusbeat only works with Nessus v6, as the API was removed in v7.

viewmithun commented 6 years ago

Hi Darvid,

Thanks for the response. Here are the details you asked for,

Yes I did specify api_url & api_username in Nessusbeat configuration file, Here is the config.go file information,

`// Config is put into a different package to prevent cyclic imports in case // it is needed in several locations

package config

type Config struct { CaCertPath string config:"cacert_path" ReportPath string config:"report_path" NessusApiUrl string config:"api_url" NessusApiUsername string config:"api_username" NessusApiPassword string config:"api_password" TimestampFields string config:"timestamp_fields" }

var DefaultConfig = Config{ ReportPath: "/opt/nessus/var/nessus/users/xxxxxxx/reports", NessusApiUrl: "https://xxxxxxxxx:8834", NessusApiUsername: "xxxxxxxxxxx", NessusApiPassword: "xxxxxxxxxxx", }`

Version of Nessus is, ./nessuscli --version nessuscli (Nessus) 6.11.2 [build M20102] Copyright (C) 1998 - 2017 Tenable Network Security, Inc
*Additional logs while running, ./nessusbeat -c nessusbeat.yml -e -d ""**

` "host": (xxx.xx.xx.xxx) "name": "Ping the remote host", "plugin_id": "10180", "plugin_output": "The remote host (xxx.xx.xx.xxx) is considered as dead - not scanning\nThe remote host (xxx.xx.xx.xxx) did not respond to the following ping methods :\n- TCP ping\n- UDP ping\n- ICMP ping\n", "port": "0", "protocol": "tcp", "risk": "None", "see_also": "", "solution": "n/a", "synopsis": "It was possible to identify the status of the remote host (alive or\ndead).", "type": "nessusbeat" } 2018/07/11 15:16:55.800634 client.go:214: DBG Publish: { "": "2018-07-11T15:16:55.800Z", "@timestamp": "2018-07-11T15:16:55.800Z", "beat": { "hostname": "nessusbeat_hostname", "name": "nessusbeat", "version": "6.0.0-alpha3" }, "cve": "", "cvss": "", "description": "Nessus was able to determine if the remote host is alive using one or\nmore of the following ping types :\n\n - An ARP ping, provided the host is on the local subnet\n and Nessus is running over Ethernet.\n\n - An ICMP ping.\n\n - A TCP ping, in which the plugin sends to the remote host\n a packet with the flag SYN, and the host will reply with\n a RST or a SYN/ACK.\n\n - A UDP ping (e.g., DNS, RPC, and NTP).", "host": "xxxxxxxxxxx", "name": "Ping the remote host", "plugin_id": "10180", "plugin_output": "The remote host (xxx.xx.xx.xxx) is considered as dead - not scanning\nThe remote host (xxx.xx.xx.xxx) did not respond to the following ping methods :\n- TCP ping\n- UDP ping\n- ICMP ping\n", "port": "0", "protocol": "tcp", "risk": "None", "see_also": "", "solution": "n/a", "synopsis": "It was possible to identify the status of the remote host (alive or\ndead).", "type": "nessusbeat" } 2018/07/11 15:16:55.801010 client.go:214: DBG Publish: { "": "2018-07-11T15:16:55.800Z", "@timestamp": "2018-07-11T15:16:55.800Z", "beat": { "hostname": "nessusbeat_hostname", "name": "nessusbeat", "version": "6.0.0-alpha3" }, "cve": "", "cvss": "", "description": "Nessus was able to determine if the remote host is alive using one or\nmore of the following ping types :\n\n - An ARP ping, provided the host is on the local subnet\n and Nessus is running over Ethernet.\n\n - An ICMP ping.\n\n - A TCP ping, in which the plugin sends to the remote host\n a packet with the flag SYN, and the host will reply with\n a RST or a SYN/ACK.\n\n - A UDP ping (e.g., DNS, RPC, and NTP).", "host": "xxxxxxxxx", "name": "Ping the remote host", "plugin_id": "10180", "plugin_output": "The remote host (xxx.xx.xx.xxx) is considered as dead - not scanning\nThe remote host (xxx.xx.xx.xxx) did not respond to the following ping methods :\n- TCP ping\n- UDP ping\n- ICMP ping\n", "port": "0", "protocol": "tcp", "risk": "None", "see_also": "", "solution": "n/a", "synopsis": "It was possible to identify the status of the remote host (alive or\ndead).", "type": "nessusbeat" } 2018/07/11 15:16:55.845379 client.go:275: DBG PublishEvents: 50 events have been published to elasticsearch in 59.273476ms. 2018/07/11 15:16:55.845474 client.go:462: WARN Can not index event (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"illegal_argument_exception","reason":"field name cannot be an empty string"}} 2018/07/11 15:16:55.845546 client.go:462: WARN Can not index event (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"illegal_argument_exception","reason":"field name cannot be an empty string"}} 2018/07/11 15:16:55.846399 client.go:462: WARN Can not index event (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse","caused_by"{"type":"illegal_argument_exception","reason":"field name cannot be an empty string"}} 2018/07/11 15:16:55.916523 client.go:462: WARN Can not index event (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"illegal_argument_exception","reason":"field name cannot be an empty string"}} 2018/07/11 15:16:55.916562 single.go:150: DBG send completed 2018/07/11 15:16:55.916592 output.go:109: DBG output worker: publish 50 events 2018/07/11 15:16:55.982454 client.go:275: DBG PublishEvents: 50 events have been published to elasticsearch in 65.830136ms. 2018/07/11 15:16:55.982538 client.go:462: WARN Can not index event (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"illegal_argument_exception","reason":"field name cannot be an empty string"}} 2018/07/11 15:16:55.982571 client.go:462: WARN Can not index event (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"illegal_argument_exception","reason":"field name cannot be an empty string"}}

`

Request you to take a look into these logs and let me know your insights.

Thanks, Mithun

darvid commented 6 years ago

Ah, this looks like a bug introduced by a8b3b1d791162655f884446f9ada383f8d072c17. If timestamp_fields in the config is empty, it will create a field with an empty key and the @timestamp as the value.

Please set timestamp_fields in the config to something, literally anything (like ts or nessus_ts).

viewmithun commented 6 years ago

Thanks a lot darvid. Issue got resolved. Now, Nessusbeat is working as expected.

Regards, Mithun

darvid / nessusbeat

Error in implementing Nessusbeat #5

cacert_path:

api_url:

api_username:

api_password:

timestamp_fields:

Array of hosts to connect to.

hosts: ["localhost:9200"]