search

networkop / k8s-topo

Topology builder for network simulations inside K8S
BSD 3-Clause "New" or "Revised" License
71 stars 18 forks source link

Questions on using NSM #9

Open vparames86 opened 4 years ago

vparames86 commented 4 years ago

Hi Michael,

I have several questions with using NSM. 1) Using NSM eliminates the requirement for meshnet CNI plugin as it is doing the same? 2) Using "replace_namespaced_custom_object" or "patch_namespaced_custom_object" update the links and recreate the topology? 3) I also see in the nsm document that "https://github.com/networkservicemesh/networkservicemesh/blob/master/docs/what-is-nsm.md" it supports connectivity to external devices. Have you tried that scenario?

If the answer to questions 2 & 3 are yes then I can work on them and create a PR.

networkop commented 4 years ago

1 - is a yes. for #2 you'd need to update the CRD (NetworkService) and recreate the sidecars with the new topology data. It shouldn't be that difficult I just don't have time to look into this yet.

  1. NSM does support external devices but they need to communicate with eNSM somehow via NSM SDK. I don't have a clear understanding of how eNSM will be integrating with the rest of the NSMs. It would be a good question to ask in NSM slack channel.
nickolaev commented 4 years ago

I will try to answer with my viewpoint here, from NSM point of view.

  1. yes
  2. I am not sure :) that is for Michael
  3. we technically do not have external devices yet. Still, we are in the works of adopting an "SR-IOV" forwarder where we'll get raw access to fast networking and eventually a way to consume HW (thint ToR switch services). The main PR is here https://github.com/networkservicemesh/networkservicemesh/pull/1925
networkop commented 4 years ago

thanks for chiming in @nickolaev . What about normal interfaces or ipvlan/macvlan? do you want to support them at some point?

vparames86 commented 4 years ago

Thanks for clarifying @networkop and @nickolaev . I have one more question. I see an ip address pool getting configured env_ip_addr = client.V1EnvVar( name="IP_ADDRESS", value=f"10.60.{self.idx}.0/24" ) and I see from nsm docs that IPAddress - [ IP_ADDRESS ], the IP network to initialize a prefix pool in the IPAM composite

why do we need this? Does this have to be a /24 block always? Is there any specific range that I should not use in the private address space?

networkop commented 4 years ago

We don't need this. I'm using this as a way for not having to configure individual ip addresses and rely on NSM's IPAM capabilities. Also this doesn't have to be a /24 block, it can be /31 but then calculating/allocating these blocks would have been more difficult. So, in short, this env variable configures NSE to serve IP addresses to NSCs, you don't have to use it, I've only used it for simplicity.

Hard-coding this into the script was the wrong idea and I thought that I'd need to factor it out at a later stage and make this whole IPAM business optional and configurable by external configuration files. The reason why I left it hard-coded were:

  1. It works for simpe demonstration purposes where I don't care about individual IPs (e.g. 700 OSPF router demo)
  2. For QEMU-embedded images (vEOS, vQFX), the IP on these interfaces don't really matter as they just serve as a way to interconnect linux bridges and vEOS/QFX will have their own IPs configured on this bridge. The obvious caveat with this latter approach is that you need to make sure you don't overlap your NSE IPAM addresses with the ones that you'll have in your virtual topology.
vparames86 commented 4 years ago

Sorry to bother you with more questions. 1) If I don't use IPAM capabilities, how can I configure the individual ip addresses later once the topology gets created? If I use a /31 block then will I be limited on the NSM URLs? 2) I also see that each nse is linked to an ENDPOINT_NETWORK_SERVICE. If that is the case how can I make a device to be part of multiple topologies (network service)?

networkop commented 4 years ago

Sorry for late response.

  1. If you don't use NSM's IPAM, you can still configure IPs manually after the fact. I don't think there's a way to make NSM assign user-defined IPs out of the box. However, I understand the SDK allows NSMs to exchange k/v pairs that clients can act on, so in theory, it should be possible.
  2. Do you want to create a multipoint interface, e.g. bridge? or do you want to have the same device connected to multiple peers? The former should be possible with VPP, i understand that kernel driver only does p2p (veth and vxlan) interfaces. If it's the latter, then each device can be both NSE and NSC for different links. Effectively the role (NSC/NSE) is unique per-link and not per-device.