Agility0493 commented 3 months ago

ISSUE TYPE

Documentation Report

COMPONENT NAME

OVS, openvswitch, Netplan, UI

CLOUDSTACK VERSION

4.19.0.1

CONFIGURATION

- Advanced Networking Mode
- KVM Hypervisor
- Openvswitch for network bridging
- We have 1 NIC to work with on this hypervisor host

OS / ENVIRONMENT

Ubuntu 22.04 LTS

SUMMARY

We are stuck and need your help with contributing to the docs (https://docs.cloudstack.apache.org/en/4.19.0.1/installguide/hypervisor/kvm.html#configure-the-network-using-openvswitch).

The OVS section doesn't cover configuring networks on Ubuntu with Netplan. Overall there is a lack of good documentation online. We are sharing our research and starting this thread in hopes we can get unstuck and all can contribute to the docs for future users as Debian based OS is very common.

STEPS TO REPRODUCE

Everything up to the network configuration section in the docs has been successfully configured and completed.

Our VLAN Schema:

Management: < 500
Public: 500 - 599
Guest (Private): 600 - 699
Guest (Public): 700 - 799
Project Specific: 900 - 999
Future Use: > 1000

Commands used:

Follow official OVS install docs: # https://docs.openvswitch.org/en/latest/intro/install/distributions/#debian-ubuntu The apt package includes all that's needed for Apache Cloudstack.

$ sudo apt install openvswitch-switch

Next disable native bridging to not interfere with OVS. To check status of Bridge module: lsmod | grep bridge

The "denylist" is called blacklist.conf on ubuntu, use this command to edit it: sudo nano /etc/modprobe.d/blacklist.conf

Add the lines:

#turn off native bridging to not interfere with openVswitch  
blacklist bridge  
#turn off native bridging br_netfilter module  
alias netfilter/bridge off

Apply the changes: $ sudo depmod -a
remove modules br_netfilter and bridge. Note that br_netfilter is dependent on bridge and has to be removed first. $ sudo rmmod br_netfilter $ sudo rmmod bridge
confirm removal: $ lsmod | grep bridge

introduction-linux-bridging-commands-and-features#basic_bridge_commands

Linux Ubuntu has the ifup-ovs and ifdown-ovs scripts in the directory /etc/network check to confirm they are there.

Our Current Network Strategy: Assign VLAN 100 to "Management traffic", VLAN 500 to "public traffic", VLAN 700 to "guest public" and VLAN 101 to "storage."

Now that OVS is installed use the following commands:

$ sudo ovs-vsctl add-br cloudbr  
$ sudo ovs-vsctl add-port cloudbr eth0  
$ sudo ovs-vsctl set port cloudbr trunks=100,101,500,600  
$ sudo ovs-vsctl add-br mgmt0 cloudbr 100  
$ sudo ovs-vsctl add-br stor0 cloudbr 101  
$ sudo ovs-vsctl add-br public0 cloudbr 500  
$ sudo ovs-vsctl add-br guestpub0 cloudbr 700

We're not sure if the above plan translates to the Apache Cloudstack GUI Zone configuration traffic labels or if our naming is best practice: mgmt0 = "Management" , stor0 = "Storage", public0 = "Public", guestpub0 = "Guest". Please confirm.

Once the configurations are in place for OVS you can confirm by using: $ sudo ovs-vsctl show

Next we need to configure linux netplan to work with OVS. We've put together the following yaml in /etc/netplan/01-netcfg.yaml.

network:
version: 2
renderer: networkd
ethernets:
eno1:
  addresses: [<ip-address>/24]
  openvswitch:
    external-ids:
      iface-id: <mac-address>
    other-config:
      disable-in-band: false
eno2: {}
bridges:
cloudbr:
  addresses: [<ip-address>/24]
  interfaces: [eno1]
  openvswitch:
    protocols: [OpenFlow10, OpenFlow11, OpenFlow12]
    controller:
      addresses: [unix:/var/run/openvswitch/ovs0.mgmt]
      connection-mode: out-of-band
    fail-mode: secure
    mcast-snooping: true
    external-ids:
      iface-id: <hostname> #not sure if hostname goes here.
    other-config:
      disable-in-band: true
mgmt0:
  addresses: []
  interfaces: [cloudbr 100]
  openvswitch: {}
stor0:
  addresses: []
  interfaces: [cloudbr 101]
  openvswitch: {}
public0:
  addresses: []
  interfaces: [cloudbr 500]
  openvswitch: {}
guestpub0:
  addresses: []
  interfaces: [cloudbr 700]
  openvswitch: {}

EXPECTED RESULTS

Receive no errors when running:
 sudo netplan try

Successfully reboot ubuntu and have all networking working.

Be able to finalize Zone creation in Apache Cloudstack using the configuration here.

ACTUAL RESULTS

We get the error:
$ sudo netplan try
ERROR:root:/etc/netplan/01-netcfg.yaml:53:11: Error in network definition: public0: interface 'cloudbr 500' is not defined.
-cloudbr 500
 ^

boring-cyborg[bot] commented 3 months ago

Thanks for opening your first issue here! Be sure to follow the issue template!

rohityadavcloud commented 3 months ago

OVS related docs are here - https://docs.cloudstack.apache.org/en/4.19.0.1/plugins/ovs-plugin.html You're welcome also to contribute docs changes if you figure out setup instructions on Ubuntu.

Agility0493 commented 2 months ago

We were able to remove all netplan errors with the following revisions to etc/netplan/01-netcfg.yaml, but networking is still not working after rebooting. These changes were based on: #https://askubuntu.com/questions/1333570/is-there-a-way-to-create-an-open-vswitch-internal-port-with-netplan

network:
  version: 2
  openvswitch:
    protocols: [OpenFlow13, OpenFlow14, OpenFlow15]
  ethernets:
    eno1:
      addresses: [<ip-address>/24]
      openvswitch:
        external-ids:
          iface-id: <mac-address>
        other-config:
          disable-in-band: false
  bridges:
    cloudbr:
      interfaces: [eno1]
      openvswitch:
        protocols: [OpenFlow13, OpenFlow14, OpenFlow15]
        controller:
          addresses: [unix:/var/run/openvswitch/ovs0.mgmt]
          connection-mode: out-of-band
        fail-mode: secure
        mcast-snooping: true
        external-ids:
          iface-id: <hostname>
        other-config:
          disable-in-band: true
  vlans:
    mgmt0:
      id:100
      link: cloudbr
      openvswitch: {}
    stor0:
      id:101
      link: cloudbr
      openvswitch: {}
    public0:
      id:500
      link: cloudbr
      openvswitch: {}
    guestpub0:
      id:700
      link: cloudbr
      openvswitch: {}

The output for sudo ovs-vsctl show:

Bridge cloudbr
    Controller: "unix:/var/run/openvswitch/ovs0.mgmt
    fail_mode: secure
    Port cloudbr
       trunks: [100, 101, 500, 600]
       Interface cloudbr
            type: internal
    Port eno1
            Interface eno1
    Port public0
            tag: 500
            Interface public0
                 type: internal
    Port mgmt0
            tag: 100
            Interface: mgtm0
                 type: internal
    Port guestpub0
            tag: 700
            Interface: guestpub0
                 type: internal
    Port stor0
            tag: 101
            Interface store0
                 type: internal
    ovs_version: "2.17.9"

apache / cloudstack

OVS Netplan yaml config Ubuntu 22.04 documentation #8983