[RFC] [RFT] Move options to config file

TobleMiner commented 4 years ago

This PR moves the configuration of mesh-announce to a config file. The old commandline based config interface is a hot mess.

Currently this is a breaking change. The old command line options have been removed and replaced by a config file.

@AiyionPrime @mweinelt what are your thoughts on this change? Do you agree with the overall config concept?

AiyionPrime commented 4 years ago

Have seen this, but won't look into this until thursday, if my weekplan works out. Breaking change sounds scary :D But maybe it cleans up the code? Will read through the commits soon; but for my own amusement. Hanover appears to be fine with the current state, as it 'works' for us. Happy Easter and such things!

mweinelt commented 4 years ago

Thanks for making that happen, I'll also have to find time in the next few days to review and test this.

mweinelt commented 4 years ago

I like the config format and with minor nits got it up and running – but it does not seem to work properly.

Config: https://gist.github.com/mweinelt/696d5b33c08944a1f65ccb439b346cf2

It does open a socket and listen on it.

udp                UNCONN              0                    0                                                                        *:1001                   *:*            users:(("python3",pid=28615,fd=3))

But I don't see queries arriving at the process and thus no replies. The firewall rules are still in place as they were before, so they're not at fault.

# strace -p 28615                                                                                                                                                                                       [811/811]
strace: Process 28615 attached                       
restart_syscall(<... resuming interrupted read ...>) = 0
poll([{fd=3, events=POLLIN}], 1, 500)   = 0 (Timeout)
poll([{fd=3, events=POLLIN}], 1, 500)   = 0 (Timeout)
poll([{fd=3, events=POLLIN}], 1, 500)   = 0 (Timeout)
poll([{fd=3, events=POLLIN}], 1, 500)   = 0 (Timeout)
poll([{fd=3, events=POLLIN}], 1, 500)   = 0 (Timeout)
poll([{fd=3, events=POLLIN}], 1, 500)   = 0 (Timeout)
poll([{fd=3, events=POLLIN}], 1, 500)   = 0 (Timeout)
poll([{fd=3, events=POLLIN}], 1, 500strace: Process 28615 detached
 <detached ...>
^C

Meanwhile queries are incoming:

# tcpdump -ni dom1-br port 1001
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on dom1-br, link-type EN10MB (Ethernet), capture size 262144 bytes
01:15:31.157162 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34
01:16:31.157267 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34
01:17:31.161753 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34
01:18:31.157309 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34
01:19:31.157313 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34
01:20:31.157291 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34
01:21:31.157256 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34
01:22:31.157227 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34
01:23:31.161184 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34

And the firewall allows that:

-A INPUT -i dom1-br -p udp -m udp --dport 1001 -j ACCEPT

Puzzling.

mweinelt commented 4 years ago

Finally … Yanic seems to be unhappy about an empty node_id.

Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.546+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:13ff:fe00:104%dom13-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.588+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:17ff:fe00:104%dom17-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.589+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:3ff:fe00:104%dom3-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.606+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:4ff:fe00:104%dom4-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.610+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:5ff:fe00:104%dom5-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.633+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:14ff:fe00:104%dom14-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.652+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:ff:fe00:104%dom0-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.661+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:9ff:fe00:104%dom9-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.661+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:8ff:fe00:104%dom8-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.668+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:11ff:fe00:104%dom11-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.670+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:6ff:fe00:104%dom6-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.694+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:16ff:fe00:104%dom16-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.706+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:2ff:fe00:104%dom2-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.709+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:10ff:fe00:104%dom10-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.710+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:1ff:fe00:104%dom1-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.717+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:7ff:fe00:104%dom7-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.718+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:15ff:fe00:104%dom15-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.719+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:12ff:fe00:104%dom12-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"

TobleMiner commented 4 years ago

In Kiel the responses generated look perfectly normal to me

root@NightGlider:~# gluon-neighbour-info -r nodeinfo -p 1001 -d fda1:384a:74de:4242::ff00

{
  "software": {
    "fastd": {
      "enabled": true,
      "version": "v18"
    },
    "batman-adv": {
      "version": "2016.4"
    },
    "firmware": {
      "base": "Debian",
      "release": "9.11"
    }
  },
  "vpn": true,
  "node_id": "deadbeefff00",
  "hardware": {
    "nproc": 8
  },
  "network": {
    "mac": "de:ad:be:ef:ff:00",
    "mesh": {
      "bat-ffki": {
        "interfaces": {
          "tunnel": [
            "de:ad:be:ef:ff:00",
            "8a:18:67:f9:b8:08",
            "6e:f6:4e:2c:e2:6f",
            "3e:4b:3c:92:f5:20",
            "2a:f4:70:32:f9:f7",
            "92:62:15:4d:2f:8d"
          ]
        }
      }
    },
    "addresses": [
      "2a07:59c6:ec02::1",
      "2a03:2260:1017::1",
      "fe80::dcad:beff:feef:ff00",
      "fda1:384a:74de:4242::ff00"
    ]
  },
  "hostname": "vpn0"
}

I'd only expect the node id to be missing it the domain is either

not configred as a batman domain

or

the batman interface does not exist

mweinelt commented 4 years ago

# Default settings
[Defaults]
# Listen port, defaults to 1001
Port: 1001
# Default multicast listen addresses
MulticastLinkAddress: ff02::2:1001
MulticastSiteAddress: ff05::2:1001
# Default domain to use
#DefaultDomain: <domain code>
# Default domain type
DefaultDomainType: batadv

The DefaultDomainType does not take effect. Works if I set DomainType in the domain.

mweinelt commented 4 years ago

However still no domain code. Is that expected?

root@64283-darmstadtcccde:~# gluon-neighbour-info -r nodeinfo -d fd01:67c:2ed8:1001::1 | jq "."
{
  "network": {
    "addresses": [
      "fd01:67c:2ed8:1001::1",
      "fe80::d8ff:1ff:fe00:104",
      "2001:67c:2ed8:1001::1",
      "fe80::9c38:62ff:fe9d:7f7f"
    ],
    "mac": "da:ff:01:00:01:02",
    "mesh": {
      "dom1-bat": {
        "interfaces": {
          "tunnel": [
            "da:ff:01:00:01:05",
            "da:ff:01:00:01:03"
          ]
        }
      }
    }
  },
  "software": {
    "fastd": {
      "version": "v18",
      "enabled": true
    },
    "firmware": {
      "release": "10",
      "base": "Debian"
    },
    "batman-adv": {
      "version": "2019.5-1-g44fa6734"
    }
  },
  "hardware": {
    "nproc": 6
  },
  "vpn": true,
  "hostname": "gw01.darmstadt.freifunk.net",
  "node_id": "daff01000102"
}

TobleMiner commented 4 years ago

@mweinelt Ah, sorry. I changed the name of the default domain type setting to just DomainType in ce359cbff08bae00022cec3a90f99e61aaca904c. That is required for the inheritance feature of the config parser

The missing domain code is not expected, however I simply forgot to support that feature while converting to the config file based system

TobleMiner commented 4 years ago

Fixed in d3d587b10f58c310b13c7a5f8f5dc0b86ff35f3a

mweinelt commented 4 years ago

Yep, that makes everything work. Thanks!

herbetom commented 4 years ago

What's the holdup? In Freifunk Rhein-Neckar we started using the config file method yesterday and it seems to work.

You could of course discuss about continuing to support single domain setups via command line arguments (if there is any need), but i would tend not to do that.

AiyionPrime commented 4 years ago

Hey @herbetom , mind to share your config with me? Then I'll test it in Hannover as well. Time to get this into master :)

mweinelt commented 4 years ago

# Default settings
[Defaults]
# Listen port, defaults to 1001
Port: 1001
# Default multicast listen addresses
MulticastLinkAddress: ff02::2:1001
MulticastSiteAddress: ff05::2:1001
# Default domain type
DomainType: batadv

[dom0]
# Batman interface, mandatory
BatmanInterface: dom0-bat
# Other listen interfaces
Interfaces: dom0-br, dom0-vlan, dom0-vpn-1312

[dom1]
# Batman interface, mandatory
BatmanInterface: dom1-bat
# Other listen interfaces
Interfaces: dom1-br, dom1-vlan, dom1-vpn-1312

[dom2]
# Batman interface, mandatory
BatmanInterface: dom2-bat
# Other listen interfaces
Interfaces: dom2-br, dom2-vlan, dom2-vpn-1312

[dom3]
# Batman interface, mandatory
BatmanInterface: dom3-bat
# Other listen interfaces
Interfaces: dom3-br, dom3-vlan, dom3-vpn-1312

[dom4]
# Batman interface, mandatory
BatmanInterface: dom4-bat
# Other listen interfaces
Interfaces: dom4-br, dom4-vlan, dom4-vpn-1312

[dom5]
# Batman interface, mandatory
BatmanInterface: dom5-bat
# Other listen interfaces
Interfaces: dom5-br, dom5-vlan, dom5-vpn-1312

[dom6]
# Batman interface, mandatory
BatmanInterface: dom6-bat
# Other listen interfaces
Interfaces: dom6-br, dom6-vlan, dom6-vpn-1312

[dom7]
# Batman interface, mandatory
BatmanInterface: dom7-bat
# Other listen interfaces
Interfaces: dom7-br, dom7-vlan, dom7-vpn-1312

[dom8]
# Batman interface, mandatory
BatmanInterface: dom8-bat
# Other listen interfaces
Interfaces: dom8-br, dom8-vlan, dom8-vpn-1312

[dom9]
# Batman interface, mandatory
BatmanInterface: dom9-bat
# Other listen interfaces
Interfaces: dom9-br, dom9-vlan, dom9-vpn-1312

[dom10]
# Batman interface, mandatory
BatmanInterface: dom10-bat
# Other listen interfaces
Interfaces: dom10-br, dom10-vlan, dom10-vpn-1312

[dom11]
# Batman interface, mandatory
BatmanInterface: dom11-bat
# Other listen interfaces
Interfaces: dom11-br, dom11-vlan, dom11-vpn-1312

[dom12]
# Batman interface, mandatory
BatmanInterface: dom12-bat
# Other listen interfaces
Interfaces: dom12-br, dom12-vlan, dom12-vpn-1312

[dom13]
# Batman interface, mandatory
BatmanInterface: dom13-bat
# Other listen interfaces
Interfaces: dom13-br, dom13-vlan, dom13-vpn-1312

[dom14]
# Batman interface, mandatory
BatmanInterface: dom14-bat
# Other listen interfaces
Interfaces: dom14-br, dom14-vlan, dom14-vpn-1312

[dom15]
# Batman interface, mandatory
BatmanInterface: dom15-bat
# Other listen interfaces
Interfaces: dom15-br, dom15-vlan, dom15-vpn-1312

[dom16]
# Batman interface, mandatory
BatmanInterface: dom16-bat
# Other listen interfaces
Interfaces: dom16-br, dom16-vlan, dom16-vpn-1312

[dom17]
# Batman interface, mandatory
BatmanInterface: dom17-bat
# Other listen interfaces
Interfaces: dom17-br, dom17-vlan, dom17-vpn-1312

AiyionPrime commented 4 years ago

Thanks, I'll be in freifunkh's mumble at seven, if anyone wants to join. https://leineserver.leinelab.org/ Default config if you join using the mumbleclient.

AiyionPrime commented 4 years ago

This branch looks promising. Argc is drastically reduced, as intended; The configfile was a breeze to setup.

As soon as this branch gets merged, I'll put the altered configuration into our ansible repo.

I'd vote for merge.

btw, is there an option to override the hostname?

herbetom commented 4 years ago

btw, is there an option to override the hostname?

I don't think so, but with the config file it should be easilier possible to add such an option, since their is to need to add another argc. :smile:

TobleMiner commented 3 years ago

Merged

AiyionPrime commented 3 years ago

I just announced in hanover, that we'll upgrade as soon as possible an then reevaluate, whether this accidentally resolved #52 or parts of #57.

ffnord / mesh-announce

[RFC] [RFT] Move options to config file #60