Feature request: support for services and nmap import

[honoki]

BBRF does not yet support services, which would be a nice addition to urls, ips and urls. The idea would be to store the results of network scans like nmap or masscan that discovered open services.

Inspiration for some supported attributes for services can be gathered from this nmap-to-mongo project:

• ip
• port
• state
• service
• hostname
• ostype
• ...

A useful unique key (_id) would likely be the concatenation of ip:port so that it can feed back into other scanners, e.g. like this:

# retrieve all known services (i.e. known open ports) and store URLs when the service is a valid HTTP service
bbrf services | httpx | bbrf url add - -t root:true

Thanks to @pry0cc for the valuable input!

[honoki]

To ingest services, I'm thinking to store only ip:port as the identifier, to also accept ip:port:service as input to store service names, and to leave all other properties to be manually specified by means of the tagging system, e.g.:

bbrf service add 127.0.0.1:8443 127.0.0.1:8888 -t hostname:localhost -t protocol:tcp
bbrf service add 127.0.0.1:80:http 127.0.0.1:21:ftp -t hostname:localhost -t protocol:tcp
bbrf service add 127.0.0.1:22:ssh 127.0.0.1:53:domain 127.0.0.1:80 -t scanned:$(date +%s)

This avoids the complexity of mapping domains to IPs (e.g. when an IP maps to more than one domain especially) while still allowing search operations supported by a mix of default properties and tags:

bbrf services where port is 8443 # this will always return all known services on port 8443
bbrf services where hostname is localhost # this will only return results when a tag has been manually provided

As a result, a document representing a service in BBRF might look as follows:

{
  "_id": "127.0.0.1:21",
  "type": "service",
  "port": "21",
  "ip": "127.0.0.1",
  "service": "ftp",
  "program": "example",
  "tags": {
    "protocol": "tcp",
    "hostname": "localhost",
    "scanned": "1611940306"
  }
}

[honoki]

The first iteration for this is available for use in the client! Note that existing servers need to be updated with the new views.json to support the new feature:

wget https://raw.githubusercontent.com/honoki/bbrf-server/main/views.json -O views.json
curl -X PUT http://yourserver:5984/bbrf/_design/bbrf" -u admin:password -d @views.json