Place config for hostnames in a separate config file

[dcthor]

Would like the option of having the config file's main.hostnames.[] array in a separate file. This would do a couple things:

1. Allow a different syntax, thereby supporting requests like #120 (device groupings and host aliases)
2. Let automated systems generate new hostname lists without disrupting/modifying the "main" configuration file. The poller would need to re-parse the [hostlist] configuration file on each loop to capture new hosts (this would also be an improvement on the current poller

[palisadoes]

I think we should:

1. differentiate the hostnames by zone. A zone being a location or some other management domain. We should not have subzones for simplicity
2. allow for hosts to have a descriptive alias as in #120

I'm working on switchmap-ng v2. I have a tracking issue here: https://github.com/PalisadoesFoundation/switchmap-ng/issues/129. This is where I'm putting my focus.

[dcthor]

OK re: no "subzones"

An idea (brainstorming) for a new "sites.yaml" file:

sitelist:
    - site: SITE-NAME-A
      hosts:
        - hostname1
        - hostname2
        - hostname3
    - site: SITE-B
      hosts:
        - hostnameA
        - hostnameB
        - hostnameC

Q's:

1. Should it be an error if a "host" exists in more than one site? I'd say "yes. It would definitely be an error if a "site" exists more than 1x
2. Should we allow for the idea of "includes" -- perhaps allow the current config.yaml to have a list of site-files which supplements the current main.hostnames.[] array? (see below)

main:
  hostnames:
  - hostX
  - hostY
  sitefiles:
  - sites.yaml
  - sites-2.yaml

[dcthor]

For aliases... would we put that in the same sitelist array, or a separate file which host/alias pairs?

[palisadoes]

The current code ingests all .yaml files in the configuration directory and combines them into a single configuration dict.

1. It works correctly assuming the files have the correct YAML fields.
2. So the use of includes is already built in.

I think zone is a better representative name, as there may be a logical distinction to consider (eg. multiple VLAN zones at a site with overlapping VLAN numbers)

I'm working on v2 of switchmap with a database backend. It will have three daemons, a poller, a database server running an API, and a web server. This could be the way we setup the configuration.

main:
  agent_subprocesses: 20
  system_directory: directory
  log_level: debug
  username: peter

poller:
  polling_interval: 21600
  zones:
    - zone: SITE-NAME-A
      hostnames:
        - hostname1
        - hostname2
        - hostname3
    - zone: SITE-B
      hostnames:
        - hostnameA
        - hostnameB
        - hostnameC
  snmp_groups:
    - group_name: CISCO
      snmp_authpassword: password
      snmp_authprotocol: sha
      snmp_community: null
      snmp_port: 161
      snmp_privpassword: password
      snmp_privprotocol: aes
      snmp_secname: name
      snmp_version: 3

server:
  bind_port: 7000
  listen_address: localhost
  db_host: localhost
  db_name: switchmap
  db_user: switchmap
  db_pass: password

dashboard:
  bind_port: 7001
  listen_address: localhost

[dcthor]

pulling in all files from the /etc folder ---> Awesome!

Zones instead of Sites --> Yup, totally makes sense.

I like your alternative yaml, especially the idea of having separate sections for main/poller/server/dashboard. Suggest "server" be renamed something like "dbserver" for clarity.

[dcthor]

Q about agent_subprocesses - shouldn't that be moved to the appropriate section (main/poller/server/dashboard) ?

[palisadoes]

In the new design all the daemons will use multiprocessing to increase performance. It needs to be in the core section for now.

For ease of configuration a single location is better than three.

I'm also thinking that we may need to give zones numbers so that we can configure a poller to only poll from specified zones. What do you think, too complex?