search

wlanslovenija / nodewatcher

A modular open networks growing platform.
http://nodewatcher.net/
Other
63 stars 20 forks source link

WIP: Integrate SNMP Daemon configuration #60

Closed robimarko closed 5 years ago

robimarko commented 7 years ago

Work in progress for adding SNMP Daemon configuration to Nodewatcher. That way we would be able to configure snmpd as an extra package instead of having to manually install and configure it later.

Currently, I think that most of the work is done, but database migrations need to be added as well some syntax fixes.

Note: Ignore parts about settings.py and docker-compose as well beta builders. Those builders are the same as current ones but with snmpd package. They are available here: https://hub.docker.com/r/otvorenamreza/lede-builder/tags/

kostko commented 7 years ago

Nice work! I've added some comments. I think that we should reuse as much information as possible from other modules (e.g., location and node maintainer), so you don't need to enter the same information multiple times.

kostko commented 7 years ago

Currently, I think that most of the work is done, but database migrations need to be added as well some syntax fixes.

Yes, please commit database migrations as well.

robimarko commented 7 years ago

Requested changes are made. I will now prepare database migrations.

robimarko commented 7 years ago

Those are good ideas but over complicate this currently. After those are added to Nodewatcher changes can be made to apply them to SNMP module as well. We wouldn't use project name as the community as we use our own community name for all of the nodes if it were to change I bet you that soon we would not know which node is in which community.

I have already explained why location does not suit us as well in chat

kostko commented 7 years ago

BTW, I just now noticed that you have this under monitor.sources, but actually you only provide reporting and not a SNMP source for nodewatcher. Are you planning to also implement SNMP as a source in nodewatcher? This would of course only make sense for including devices, which don't support nodewatcher-agent?

robimarko commented 7 years ago

@mitar Suggested that it should be placed here. This way in future it can be extended to be the source for Nodewatcher. I currently don't have plans for that.

mitar commented 7 years ago

Those are good ideas but over complicate this currently.

Why it complicate? It makes it easier to manage and change multiple nodes at once. If you will ever have to change anything you will be able to change it in one place.

We wouldn't use project name as the community as we use our own community name for all of the nodes if it were to change I bet you that soon we would not know which node is in which community.

Yes, different users of nodewatcher would have different needs. Some could configure it so that all nodes of nodewatcher is in one community, some could configure it so that each project is one community.

Now, if you are saying that there is no point in having multiple SNMP communities in the network, then I propose we do not allow maintainers to configure random community values here and this is something directly configured by nodewatcher for the whole network managed by nodewatcher for all nodes. In the case of wlan slovenija installation of nodewatcher, that would then be wlan-si.net or some other related name of the community. But because I assume you would want a name for your nodes to match your project, I would propose that we also allow projects to have their own SNMP communities.

I have already explained why location does not suit us as well in chat

Yes. I am not saying that you should use location, but that location could be one of options to use for users who do want it. For you a wider concept like "location area" might be a better one to choose.

You have to remember that nodewatcher is not a centralized web interface to UCI. So the idea is not to have simply what would run on one node through web interface to set those values, but on a central server. The idea is that you have a system to manage nodes. And managing means that you do not have to go around 100 nodes when you change one value, but you can do that in one place, and it updates values everywhere. This is why it is good to have values being defined only in one place, so that they can be reused around. Look at form defaults. Yes, you can disable defaults and enter everything manually, but the question is what happens when you want to update some value.

@kostko, what do you think, is this maybe something better resolved at the level of form defaults? So a community can define their own defaults for SNMP, and then they can in defaults say from where those values come from. In form defaults, if you change code which generates where form defaults are getting values, does this automatically update all configurations of nodes, or just the next time you are updating configuration using defaults?

BTW, what I am thinking here would be that it would be great if SNMP would work the same as nodewatcher agent. So that it could be a replacement for it. So that it is an integral part of nodewatcher, which can just work out of the box. So my question is, how much configuration do we even want to expose to the user. Or should be only to choose which protocol to use for getting data (nodewatcher pull, push, or snmp). And then the rest would simply be chosen by nodewatcher?

@robimarko: Why you want SNMP support? So that nodewatcher would be collecting information from nodes using SNMP? Or that some other programs to monitor could also monitor nodes? And then you want to expose information from nodewatcher like community and location to those systems? I think it the latter case it is then even more important to dynamically populate those values, so that it is also a form of syncing/federation between nodewatcher and that other system.

robimarko commented 7 years ago

Mitar I can mostly agree. But we differ in some views. Forcing people to use project name for SNMP community is not preferred. That way use case for SNMP is being limited. I think that it would be good to have project name being default SNMP community and that users can change it in node settings if it does not suit their needs. A better solution would be having the option to set per project community name that does not have to be equal to the project name.

For location, I think that good solution would be having the option to make groups withing Projects and that than group name or group location would be default location with having option to change it in node settings.

I agree that this would be better solved by having per project or per group default settings that still can be changed in node settings.

@mitar We want SNMP support because we use it every day for monitoring 92 nodes currently. We currently use LibreNMS for that purpose. It offers us email, rocket chat and various other alerts that are fully customizable. Also, offers integration with various applications and has a lot of extensions. And it enables us to monitor parts of our backbone that do not have Nodewatcher firmware because either LEDE/OpenWRT do not support them or performance penalty is too high. Mostly that is a lot of PtP links using Ubiquiti AC gear and plenty of POE switches as others like Mikrotik and everything else not running Nodewatcher firmware. And numerous other features.

It would be great that Nodewatcher could use SNMP as data source and that certainly can be something on TODO list for future because SNMP is a lot more difficult to implement than HTTP. Implementation in Nodewatcher can be done via using LibreNMS API or by simply reading measurements from InfluxDB since LibreNMS natively can export data to InfluxDB and we currently do that for our Grafana graphing. But having SNMP daemon properly configured is the first step to it.

I look forward to more discussion and further improvement of Nodewatcher as it looks like it is getting better and better every day

kostko commented 7 years ago

Forcing people to use project name for SNMP community is not preferred.

I don't think Mitar was suggesting that, he said that it would be configurable.

Implementation in Nodewatcher can be done via using LibreNMS API or by simply reading measurements from InfluxDB since LibreNMS natively can export data to InfluxDB and we currently do that for our Grafana graphing.

No, this would be a bad implementation. We don't need additional dependencies (e.g., we will not run LibreNMS just to have SNMP support), so the SNMP source should be stand-alone.

robimarko commented 7 years ago

Ok, then we agree on almost all. I know that it would be bad implementation, but otherwise writing standalone support would be a lot of work.

Something like PySNMP or Easy SNMP can be used but it would still require a lot of work. Surely something that can be done in future

kostko commented 7 years ago

@kostko, what do you think, is this maybe something better resolved at the level of form defaults? So a community can define their own defaults for SNMP, and then they can in defaults say from where those values come from. In form defaults, if you change code which generates where form defaults are getting values, does this automatically update all configurations of nodes, or just the next time you are updating configuration using defaults?

So defaults are the most flexible (e.g., you can combine multiple sources or make everything static) and this is something that could be handled there. And then we can introduce a defaults module, which configures this based on project or other factors and also allows overrides.

Something like PySNMP or Easy SNMP can be used but it would still require a lot of work. Surely something that can be done in future

Yes, so I think that adding SNMP source support to nodewatcher would be great so we can use nodewatcher also for devices, which cannot run our agent instead of requiring external systems like LibreNMS to be used.

robimarko commented 7 years ago

@kostko I agree completely

mitar commented 7 years ago

OK, so what are then next steps? Maybe:

robimarko commented 7 years ago

I think you pretty much summed it up. Most difficult part is the last one.

mitar commented 7 years ago

I am not sure why would it be too hard? We already have whole infrastructure for running long-lived processes available. You just instead of doing periodic HTTP requests, use PySNMP to fetch SNMP data from nodes. (Or you configure it to listen for SNMP data from nodes.) But yes, we would have to learn how to do it. :-)

robimarko commented 7 years ago

Because SNMP despite having simple in its name is nowhere near simple as current HTTP JSON data source. It is based on tables and on Linux as well as OpenWRT/LEDE by default a lot of stuff that you currently pull with HTTP is not pulled by default but requires the use of exec and custom tables.

If you want to see how data fetched from SNMP looks I can make you account to our LibreNMS monitoring

mitar commented 7 years ago

Maybe try to use this example to fetch any value from your existing nodes. You can also check this project to see various IDs to use to fetch data from AirOS. Once you have those IDs, it is pretty easy to fetch them and store them in database (the latter part is already there).

See this processor. It runs and uses HTTP, but similarly we could use SNMP.

robimarko commented 7 years ago

LibreNMS GitHub is an awesome place to look for examples. They pretty much support everything that has SNMP daemon.

robimarko commented 7 years ago

I am just trying out if this works. Unfortunately,it looks like a problem with writing to DB. Here is full log: https://gist.github.com/robimarko/bc6003771467ce5c0213e8ef908aa249