canonical / microcloud

Automated private cloud based on LXD, Ceph and OVN
https://microcloud.is
GNU Affero General Public License v3.0
280 stars 43 forks source link

docs: Is this a typo or? #403

Closed kundeng closed 2 weeks ago

kundeng commented 3 weeks ago

Please describe the question or issue you're facing with "Get started with MicroCloud".

cat << EOF > /etc/netplan/99-microcloud.yaml
# MicroCloud requires a network interface that doesn't have an IP address
network:
    version: 2
    ethernets:
        enp6s0:
            accept-ra: false
            dhcp4: false
            link-local: []
EOF
chmod 0600 /etc/netplan/99-microcloud.yaml

Later in the document, it says enp5s0


Reported from: https://canonical-microcloud.readthedocs-hosted.com/en/latest/microcloud/tutorial/get_started/

kundeng commented 3 weeks ago

It seems illogical that if enp5s0 has no IP assigned, the micro VMs would still be able to execute the following commands:

snap install microovn --channel=22.03/stable --cohort="+"
snap install microcloud --channel=latest/stable --cohort="+"

How is this possible without an IP address?

kundeng commented 3 weeks ago
netplan apply

also generate error:

WARNING:root:Cannot call Open vSwitch: ovsdb-server.service is not running.

which seems to require:

 apt install openvswitch-switch-dpdk
kundeng commented 3 weeks ago

yet another error is:

root@micro2:~# snap refresh lxd --channel=5.21/stable --cohort="+"
2024-09-22T01:05:12Z INFO Waiting for "snap.lxd.daemon.service" to stop.
error: cannot perform the following tasks:
- Run configure hook of "lxd" snap if present (run hook "configure": error: cannot communicate with server: Post "http://localhost/v2/snapctl": dial unix /run/snapd-snap.socket: connect: no such file or directory)
kundeng commented 3 weeks ago

anyway, following the tutorial results in:

No dedicated uplink interfaces detected, skipping distributed networking
Initializing a new cluster
 Local MicroCloud is ready
 Local LXD is ready
 Local MicroOVN is ready
 Local MicroCeph is ready
Awaiting cluster formation ...
 Peer "micro2" has joined the cluster
 Peer "micro4" has joined the cluster
 Peer "micro3" has joined the cluster
Configuring cluster-wide devices ...
MicroCloud is ready

I'm confused, I think 2 interfaces need to be created for each VM, but I'm not seeing such in the doc.

kundeng commented 3 weeks ago

upon looking at host lxd, I think I know what is going on. lxd by default does give each VM an ethernet interface. So the issue is the documentation has typo. It should be a consistent enp5s0. Will give it another try.

Is there a way to fix this without having to restart the tutorial?

masnax commented 3 weeks ago

MicroCloud does require two network interfaces per system. This is described in the networking section of the docs: https://canonical-microcloud.readthedocs-hosted.com/en/latest/microcloud/explanation/microcloud/#networking which is linked in the tutorial here: https://canonical-microcloud.readthedocs-hosted.com/en/latest/microcloud/tutorial/get_started/#create-a-network

enp5s0 should have addresses assigned since it's used for the cluster management network, but enp6s0 should not have any addresses assigned as it will be handled by OVN for external connectivity. Setting up the OVN network was skipped in your setup because no interface was found matching the configuration of enp6s0 as described in the tutorial.

You can set up the OVN network afterward without restarting the tutorial by following the initial steps in the tutorial to set up enp6s0 and then run the following with MicroOVN and LXD:

# Get the northbound connection config from MicroOVN
microovn status
# add the above config to LXD by using a comma separated list of addresses in the form ssl:<ip>:6641
lxc config set network.ovn.northbound_connection=...
# create a pending uplink network on each system in the cluster. 
# The parent interface should correspond to `enp6s0` from the tutorial.
lxc network create UPLINK --type=physical --target <system_name> parent=<interface>
# finalize the uplink network
lxc network create UPLINK ipv4.gateway=<subnet> ipv4.ovn.ranges=<ip_start-ip_end> ipv6.gateway=<subnet> dns.nameservers=<subnet>
# create the default ovn network
lxc network create default network=UPLINK --type ovn
masnax commented 3 weeks ago

Also, just a note, but the doc links you've posted so far (and the ones I posted above) are for the development version of MicroCloud. For your question, this shouldn't have any difference but you may encounter some steps in the docs that are not supported in the stable release of MicroCloud (latest/stable).

Here's a link to the stable docs: https://canonical-microcloud.readthedocs-hosted.com/en/stable/

simondeziel commented 2 weeks ago

also generate error:

WARNING:root:Cannot call Open vSwitch: ovsdb-server.service is not running.

It's just unrelated noise which I agree is confusing as MicroCloud does use OVS too.

which seems to require:

 apt install openvswitch-switch-dpdk

That's not actually required with MicroCloud, the warning can just be ignored.

Please reopen the issue if masnax's explanation wasn't clear.

kundeng commented 2 weeks ago

The crux of the problem lies in a document, whether stable or not, that lacks accuracy, leading to wasted time for those attempting to use it.

It would be appropriate for the individuals who identify as maintainers to review the tutorial thoroughly and amend the documentation based on this bug report.

simondeziel commented 2 weeks ago

@kundeng I re-read the link your provided in the issue description and don't see what's unclear/ambiguous, so please point it out so we can improve it.

In https://canonical-microcloud.readthedocs-hosted.com/en/latest/microcloud/tutorial/get_started/#create-and-configure-your-vms, the VMs all get an extra NIC named eth1 which once in the VM itself appears as enp6s0. This eth1 vs enp6s0 is explained further down in a note.

After that point, the VMs each have 2 NICs, enp5s0 providing Internet connectivity (as connected to the default lxdbr0 bridge) and enp6s0 that is connected to the extra network bridge microbr0. Interfaces (eth1/enp6s0) connected to that microbr0 must not have IPs hence the netplan snippet to that effect.