search

kanniget / nebulamgr

Bulk config tool for generating nebula VPN configs
GNU Affero General Public License v3.0
8 stars 3 forks source link

Support Multiple lighthouse Hosts #2

Open kanniget opened 4 years ago

kanniget commented 4 years ago

Current approach only supports a single lighthouse instance.

Need to adjust the code to allow multiple.

kanniget commented 4 years ago

Functionality added to support multiple lighthouses. Needs testing as I dont have multiple.

luismanson commented 4 years ago

Needs testing as I dont have multiple.

It's not working, something is wrong in my definition, i think.

Generating configs
     f1.vpn.testingnet.xyz
Traceback (most recent call last):
  File "./nebulamgr.py", line 232, in <module>
    main()
  File "./nebulamgr.py", line 224, in main
    process(args)
  File "./nebulamgr.py", line 189, in process
    build_conf(entry, Config)
  File "./nebulamgr.py", line 141, in build_conf
    lighthouse["address"]=resolveLighthouseAddress(lighthouse["name"], config)
TypeError: list indices must be integers or slices, not str

Updated definition

cidr: "24"
lighthouse:
    - f1.vpn.testingnet.xyz:
        remote: "one.testingnet.xyz"
        remote_port: "4242"

    - f2.vpn.testingnet.xyz:
        remote: "two.testingnet.xyz"
        remote_port: "4242"

hosts:
    - f1.vpn.testingnet.xyz:
        address: "10.175.17.1"
        port: "4242"
    - f2.vpn.testingnet.xyz:
        address: "10.175.17.2"
        port: "4242"
    - endpoint1.vpn.testingnet.xyz:
        address: "10.175.17.100"
        #localnet: "192.168.0.0/24"
        port: "4242"
kanniget commented 4 years ago

I am not near my dev environment at the moment but I think the issue is I did not update the sample conf.

I think it should be like this....

cidr: "24" lighthouse:

hosts:

luismanson commented 4 years ago

Thanks. That did not do it. Don't you need to loop lighthouses near this line? lighthouse["address"]=resolveLighthouseAddress(lighthouse["name"], config)

kanniget commented 4 years ago

I will have to look at it, I setup a test config file and it generates everything ok using the approach I mentioned.

There is a loop on the line before that one to run through the hosts and inside the function it loops through the lighthouses.....

luismanson commented 4 years ago

I will have to look at it, I setup a test config file and it generates everything ok using the approach I mentioned.

Thanks! It's working....Fixed with a git pull... :\

I think I found a missing loop for hosts:

  # IMPORTANT2: THIS SHOULD BE LIGHTHOUSES' NEBULA IPs, NOT LIGHTHOUSES' REAL ROUTABLE IPs
  hosts:
    {% for entry in lighthouse %}{% if not is_lighthouse %}- "{{ entry.address }}"  # {{entry.remote}} {% endif %}
    {% endfor %}

One question, in "Security/nbound" how's 'destination' field used?

kanniget commented 4 years ago

I will take a lookat the loop tonight, currently late for work...

from memory the logic is... the destination field is the name of the group or host the rule is applied to..

so when you want to allow a host A to connect to host B you put host B in the destination.... if you want to host A to connect to Group C then C goes in the destination field...

On Tue, 21 Jul 2020 at 00:48, Luis notifications@github.com wrote:

I will have to look at it, I setup a test config file and it generates everything ok using the approach I mentioned. Fixed with a git pull... :\

I think I found a missing loop for hosts:

IMPORTANT2: THIS SHOULD BE LIGHTHOUSES' NEBULA IPs, NOT LIGHTHOUSES' REAL ROUTABLE IPs

hosts: {% for entry in lighthouse %}{% if not is_lighthouse %}- "{{ entry.address }}" # {{entry.remote}} {% endif %} {% endfor %}

One question, in "Security/nbound" how's 'destination' field used?

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/kanniget/nebulamgr/issues/2#issuecomment-661086688, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATRAC6CTUEGR6IBVWSK5WLR4RKL5ANCNFSM4OPKC5GA .

luismanson commented 4 years ago

In nebulamgr.yml i see a combination of destination, group and host fieldname used...Is this ok? I still don't see where/how destination is evaluated and where it ends up in the config file. I'm always afraid to be asking obvious stuff, but I still do not understand 'a destination' field in 'input chain'? And there is no mention of it in nebula's config file.

Thanks for your help

security:
    inbound:
      - port: "any"
        proto: "icmp"
        host:  "any"
        destination: "all"
        comment: " Allow any icmp between nodes "
      - port: "22"
        proto: "tcp"
        group: "management"
        destination: "all"
        comment: " Allow all management servers to connect via port 22 ( SSH )  "

Official config.yml

# The firewall is default deny. There is no way to write a deny rule.
  # Rules are comprised of a protocol, port, and one or more of host, group, or CIDR
  # Logical evaluation is roughly: port AND proto AND (ca_sha OR ca_name) AND (host OR group OR groups OR cidr)
  # - port: Takes `0` or `any` as any, a single number `80`, a range `200-901`, or `fragment` to match second and further fragments of fragmented packets (since there is no port available).
  #   code: same as port but makes more sense when talking about ICMP, TODO: this is not currently implemented in a way that works, use `any`
  #   proto: `any`, `tcp`, `udp`, or `icmp`
  #   host: `any` or a literal hostname, ie `test-host`
  #   group: `any` or a literal group name, ie `default-group`
  #   groups: Same as group but accepts a list of values. Multiple values are AND'd together and a certificate would have to contain all groups to pass
  #   cidr: a CIDR, `0.0.0.0/0` is any.
  #   ca_name: An issuing CA name
  #   ca_sha: An issuing CA shasum
kanniget commented 4 years ago

ok. if you look in nebulamgr.py there is a function,

def build_host(hostname, config):

if hostname == "lighthouse":

host = {"name": hostname, "groups": [], "outbound": [], "inbound": []}

else:

for host_entry in config.get_config(section="hosts"): for name in host_entry: if hostname == name: host = host_entry[hostname] host["name"] = hostname host["outbound"] = [] host["inbound"] = [] host["groups"] = [] for group in config.get_config(section="groups"): for groupname in group: for member in group[groupname]: if member == hostname: host["groups"].append(groupname) security = config.get_config(section="security") for ruleclass in security: if ruleclass == "outbound": for rule in security[ruleclass]: host["outbound"].append(rule) else: for rule in security[ruleclass]: if rule["destination"] == "any": host[ruleclass].append(rule) elif rule["destination"] == "all": host[ruleclass].append(rule) elif rule["destination"] == hostname: host[ruleclass].append(rule) return host

The last part of it shows you how it is worked out.

For each host it generates a rule to match by determining if the host matches the destination field. As an aside, Looking at it I think I missed the "group" membership part of the checks.

Instead of think of it as a "destination" maybe think of it as "which hosts does this rule apply to?"

Does this help?

On Tue, 21 Jul 2020 at 11:10, Luis notifications@github.com wrote:

In nebulamgr.yml i see a combination of destination, group and host fieldname used...Is this ok? I still don't see where/how destination is evaluated and where it ends up in the config file. I'm always afraid to be asking obvious stuff, but I still do not understand 'a destination' field in 'input chain'? And there is no mention of it in nebula's config file.

Thanks for your help

security: inbound:

  • port: "any" proto: "icmp" host: "any" destination: "all" comment: " Allow any icmp between nodes "
  • port: "22" proto: "tcp" group: "management" destination: "all" comment: " Allow all management servers to connect via port 22 ( SSH ) "

Official config.yml

The firewall is default deny. There is no way to write a deny rule.

Rules are comprised of a protocol, port, and one or more of host, group, or CIDR

Logical evaluation is roughly: port AND proto AND (ca_sha OR ca_name) AND (host OR group OR groups OR cidr)

- port: Takes 0 or any as any, a single number 80, a range 200-901, or fragment to match second and further fragments of fragmented packets (since there is no port available).

code: same as port but makes more sense when talking about ICMP, TODO: this is not currently implemented in a way that works, use any

proto: any, tcp, udp, or icmp

host: any or a literal hostname, ie test-host

group: any or a literal group name, ie default-group

groups: Same as group but accepts a list of values. Multiple values are AND'd together and a certificate would have to contain all groups to pass

cidr: a CIDR, 0.0.0.0/0 http://0.0.0.0/0 is any.

ca_name: An issuing CA name

ca_sha: An issuing CA shasum

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/kanniget/nebulamgr/issues/2#issuecomment-661520232, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATRAC2KHGPAH6IYEXD4TQDR4TTH7ANCNFSM4OPKC5GA .

luismanson commented 4 years ago

Yes! thanks, 'll play white rules.

Cheers

kanniget commented 4 years ago

I fixed the bit of code where I missed groups in the destination field so if you have the latest it's in there.

I am struggling to get a connection from my roaming laptop to a machine deep I side my network behind 2 NAT devices....

I think the issue is connectivity not the security rules..

Thinking about config file management now as it's one thing to generate a config file and push it out but continually doing so as you test Configs etc is painful.... And I am only doing a couple.

On Wed, 22 Jul 2020, 9:13 am Luis, notifications@github.com wrote:

Yes! thanks, 'll play white rules.

Cheers

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/kanniget/nebulamgr/issues/2#issuecomment-662153236, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATRAC7OEOWXZGVOMFP5GNTR4YOKVANCNFSM4OPKC5GA .

luismanson commented 4 years ago

Thanks for the fix. I'll try later today. is not punching hole? What about logs?

I agree with you about config push and volume of devices. Given certificates can be include in config file. What about adding some fileds into host definition and a command flag to call Ansible and deploy ?

kanniget commented 4 years ago

Was thinking of something like using a private git repo with deploy keys. Just need to work out how to do a fit pull of specific files and it should work.

Ansible is really good but struggles in the windows space and not sure about digging into a socket container.

On Wed, 22 Jul 2020, 11:18 pm Luis, notifications@github.com wrote:

Thanks for the fix. I'll try later today. is not punching hole? What about logs?

I agree with you about config push and volume of devices. Given certificates can be include in config file. What about adding some fileds into host definition and a command flag to call Ansible and deploy ?

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/kanniget/nebulamgr/issues/2#issuecomment-662447596, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATRACZWLUYHPOX3CABD3KDR43RJHANCNFSM4OPKC5GA .

luismanson commented 4 years ago

That sounds good it works on many platforms!

El mié., 22 jul. 2020 21:45, Kanniget notifications@github.com escribió:

Was thinking of something like using a private git repo with deploy keys. Just need to work out how to do a fit pull of specific files and it should work.

Ansible is really good but struggles in the windows space and not sure about digging into a socket container.

On Wed, 22 Jul 2020, 11:18 pm Luis, notifications@github.com wrote:

Thanks for the fix. I'll try later today. is not punching hole? What about logs?

I agree with you about config push and volume of devices. Given certificates can be include in config file. What about adding some fileds into host definition and a command flag to call Ansible and deploy ?

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/kanniget/nebulamgr/issues/2#issuecomment-662447596, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AATRACZWLUYHPOX3CABD3KDR43RJHANCNFSM4OPKC5GA

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kanniget/nebulamgr/issues/2#issuecomment-662767441, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABUDCJRWA3EGQKIDJLU3KLR46B3XANCNFSM4OPKC5GA .

kanniget commented 4 years ago

So... I screwed up the template for the multiple lighthouses and that was why my config wasnt working... I have adjusted it and pushed it out.

luismanson commented 4 years ago

Thanks! I've also modified template in very similar way and did not commit/push my changes. I apologize, it was part of my trial to include certificates inside the config file, the CA gave me issues and got stuck. Is that useful to you? I can open a PR, but i need help with the ca file thing.