search

canonical / cloud-init

Official upstream for the cloud-init: cloud instance initialization
https://cloud-init.io/
Other
2.88k stars 859 forks source link

cloud-init hard codes IPV6_AUTOCONF=no, which prevents EC2 instances from getting a v6 default gateway if NetworkManager is disabled #3302

Closed ubuntu-server-builder closed 1 year ago

ubuntu-server-builder commented 1 year ago

This bug was originally filed in Launchpad as LP: #1808647

Launchpad details 
affected_projects = []
assignee = harald-jensas
assignee_name = Harald Jensås
date_closed = 2019-12-18T17:55:18.103512+00:00
date_created = 2018-12-15T18:03:23.899693+00:00
date_fix_committed = 2019-11-20T17:56:15.376575+00:00
date_fix_released = 2019-12-18T17:55:18.103512+00:00
id = 1808647
importance = medium
is_complete = True
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1808647
milestone = None
owner = irving-popovetsky
owner_name = Irving Popovetsky
private = False
status = fix_released
submitter = irving-popovetsky
submitter_name = Irving Popovetsky
tags = ['ec2', 'ipv6', 'rhel']
duplicates = []

Launchpad user Irving Popovetsky(irving-popovetsky) wrote on 2018-12-15T18:03:23.899693+00:00

Hello,

On CentOS 7.6 (cloud-init 18.2) on EC2 with IPv6 enabled. If you disable NetworkManager, your instances do not acquire an IPv6 default gateway.

I can only seem to work around the issue by removing this line: https://git.launchpad.net/cloud-init/tree/cloudinit/net/sysconfig.py#n668

But I would love to find a more elegant solution to override this value.

ubuntu-server-builder commented 1 year ago

Launchpad user Ryan Harper(raharper) wrote on 2019-04-29T14:21:03.654919+00:00

Hello,

Thanks for filling a bug. Could you attach the file output from 'cloud-init collect-logs' ?

Thanks

ubuntu-server-builder commented 1 year ago

Launchpad user Irving Popovetsky(irving-popovetsky) wrote on 2019-04-29T15:42:13.408393+00:00

collect-logs attached, thank you for taking a look! Launchpad attachments: cloud-init.tar.gz

ubuntu-server-builder commented 1 year ago

Launchpad user Irving Popovetsky(irving-popovetsky) wrote on 2019-04-29T15:46:36.980009+00:00

If you need a repro case, here's how I generate the base centos AMI: https://github.com/irvingpop/packer-chef-highperf-centos7-ami/blob/master/create_base_ami/create_base_ami.sh

Right now I'm working around the issue by patching it here: https://github.com/irvingpop/packer-chef-highperf-centos7-ami/blob/master/create_base_ami/create_base_ami.sh#L120

and on updates: https://github.com/irvingpop/packer-chef-highperf-centos7-ami/blob/master/scripts/install_aws_compatibiliy.sh#L19

ubuntu-server-builder commented 1 year ago

Launchpad user Launchpad Janitor(janitor) wrote on 2019-06-29T04:17:32.100770+00:00

[Expired for cloud-init because there has been no activity for 60 days.]

ubuntu-server-builder commented 1 year ago

Launchpad user Irving Popovetsky(irving-popovetsky) wrote on 2019-07-01T14:54:53.995874+00:00

Hello since this issue is pending a look from a cloud-init maintainer, could we please reopen?

ubuntu-server-builder commented 1 year ago

Launchpad user Ryan Harper(raharper) wrote on 2019-07-01T16:20:17.551490+00:00

The log looked a bit strange, it shows cloud-init 0.7.9, not 18.2. In any case, it still reproduced on master. The provided network-config is to dhcp4 and dhcp6 on the primary interface.

Looking at what IPV6_AUTOCONF boolean controls, I see:

IPV6_AUTOCONF=answer where answer is one of the following: yes — Enable IPv6 autoconf configuration for this interface. no — Disable IPv6 autoconf configuration for this interface. If enabled, an IPv6 address will be requested using Neighbor Discovery (ND) from a router running the radvd daemon. Note that the default value of IPV6_AUTOCONF depends on IPV6FORWARDING as follows: If IPV6FORWARDING=yes, then IPV6_AUTOCONF will default to no. If IPV6FORWARDING=no, then IPV6_AUTOCONF will default to yes and IPV6_ROUTER has no effect.

We don't currently provide a way via network-config to control IPV6_AUTOCONF nor IPV6FORWARDING and it's not clear that there is a default value in sysconfig either. Historically the IPV6_AUTOCONF=no was set expecting that the provided networking would include any gateway values if needed.

It's not clear to me why dhcp6 does not get you a gateway? Can you provide the dhcp6 lease and 'ip -6 a' and 'ip -6 route show' output?

I;d like to understand what does and does not work with IPV6_AUTOCONF enabled/disabled.

ubuntu-server-builder commented 1 year ago

Launchpad user Irving Popovetsky(irving-popovetsky) wrote on 2019-07-07T00:21:44.094887+00:00

Thanks for the fast response Ryan!

Here's some information from a fully updated CentOS 7.6 as of today, unmodified cloud-init 18.2-1.el7.centos.2 but no NetworkManager on AWS with IPV6 enabled.

[root@ip-172-31-21-249 ~]# cat /etc/sysconfig/network
# Created by cloud-init on instance boot automatically, do not edit.
#
NETWORKING=yes
NETWORKING_IPV6=yes
IPV6_AUTOCONF=no

[root@ip-172-31-21-249 ~]# ps -ef |grep dhc
root      1347     1  0 00:14 ?        00:00:00 /sbin/dhclient -1 -q -lf /var/lib/dhclient/dhclient--ens5.lease -pf /var/run/dhclient-ens5.pid -H ip-172-31-21-249 ens5
root      1425     1  0 00:14 ?        00:00:00 /sbin/dhclient -6 -1 -lf /var/lib/dhclient/dhclient6--ens5.lease -pf /var/run/dhclient6-ens5.pid ens5 -H ip-172-31-21-249

[root@ip-172-31-21-249 ~]# cat /var/lib/dhclient/dhclient6--ens5.lease
default-duid "\000\001\000\001$\263\360\355\002\030CCi:";
lease6 {
  interface "ens5";
  ia-na "CCi:" {
    starts 1562458400;
    renew 75;
    rebind 120;
    iaaddr 2600:1f14:589:f00:1212:8330:638:9075 {
      starts 1562458400;
      preferred-life 150;
      max-life 450;
    }
  }
  option dhcp6.client-id 0:1:0:1:24:b3:f0:ed:2:18:43:43:69:3a;
  option dhcp6.server-id 0:3:0:1:2:42:8f:b:a5:8e;
}
lease6 {
  interface "ens5";
  ia-na "CCi:" {
    starts 1562458400;
    renew 75;
    rebind 120;
    iaaddr 2600:1f14:589:f00:1212:8330:638:9075 {
      starts 1562458400;
      preferred-life 150;
      max-life 450;
    }
  }
  option dhcp6.client-id 0:1:0:1:24:b3:f0:ed:2:18:43:43:69:3a;
  option dhcp6.server-id 0:3:0:1:2:42:8f:b:a5:8e;
}
lease6 {
  interface "ens5";
  ia-na "CCi:" {
    starts 1562458477;
    renew 75;
    rebind 120;
    iaaddr 2600:1f14:589:f00:1212:8330:638:9075 {
      starts 1562458477;
      preferred-life 150;
      max-life 450;
    }
  }
  option dhcp6.client-id 0:1:0:1:24:b3:f0:ed:2:18:43:43:69:3a;
  option dhcp6.server-id 0:3:0:1:2:42:8f:b:a5:8e;
}
lease6 {
  interface "ens5";
  ia-na "CCi:" {
    starts 1562458552;
    renew 75;
    rebind 120;
    iaaddr 2600:1f14:589:f00:1212:8330:638:9075 {
      starts 1562458552;
      preferred-life 150;
      max-life 450;
    }
  }
  option dhcp6.client-id 0:1:0:1:24:b3:f0:ed:2:18:43:43:69:3a;
  option dhcp6.server-id 0:3:0:1:2:42:8f:b:a5:8e;
}

[root@ip-172-31-21-249 ~]# ip -6 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 state UP qlen 1000
    inet6 2600:1f14:589:f00:1212:8330:638:9075/64 scope global dynamic
       valid_lft 442sec preferred_lft 142sec
    inet6 fe80::18:43ff:fe43:693a/64 scope link
       valid_lft forever preferred_lft forever

[root@ip-172-31-21-249 ~]# ip -6 route show
unreachable ::/96 dev lo metric 1024 error -113 pref medium
unreachable ::ffff:0.0.0.0/96 dev lo metric 1024 error -113 pref medium
unreachable 2002:a00::/24 dev lo metric 1024 error -113 pref medium
unreachable 2002:7f00::/24 dev lo metric 1024 error -113 pref medium
unreachable 2002:a9fe::/32 dev lo metric 1024 error -113 pref medium
unreachable 2002:ac10::/28 dev lo metric 1024 error -113 pref medium
unreachable 2002:c0a8::/32 dev lo metric 1024 error -113 pref medium
unreachable 2002:e000::/19 dev lo metric 1024 error -113 pref medium
2600:1f14:589:f00::/64 dev ens5 proto kernel metric 256 expires 432sec pref medium
unreachable 3ffe:ffff::/32 dev lo metric 1024 error -113 pref medium
fe80::/64 dev ens5 proto kernel metric 256 mtu 9001 pref medium

If you ever want to play around with this yourself, I periodically create and publish AMIs based on the template I reference: https://github.com/irvingpop/packer-chef-highperf-centos7-ami#latest-amis Launchpad attachments: Updated collect-logs output

ubuntu-server-builder commented 1 year ago

Launchpad user Paride Legovini(paride) wrote on 2019-07-08T16:21:47.230476+00:00

Thanks for the additional information. Could you also provide the output of the same commands this time with IPV6_AUTOCONF=yes?

Does setting IPV6_AUTOCONF=yes make the instance behave as when you apply your workaround?

ubuntu-server-builder commented 1 year ago

Launchpad user Irving Popovetsky(irving-popovetsky) wrote on 2019-07-10T23:35:19.012112+00:00

absolutely! the behavior is identical whether IPV6_AUTOCONF=yes is in /etc/sysconfig/networking or if the line is totally omitted (presumably because IPV6FORWARDING is not set to "yes"?)

I ran the same commands as comment #7 and the only one that had any different output was the route table, particularly the last line:

[root@ip-172-31-40-118 ~]# ip -6 route show
unreachable ::/96 dev lo metric 1024 error -113 pref medium
unreachable ::ffff:0.0.0.0/96 dev lo metric 1024 error -113 pref medium
unreachable 2002:a00::/24 dev lo metric 1024 error -113 pref medium
unreachable 2002:7f00::/24 dev lo metric 1024 error -113 pref medium
unreachable 2002:a9fe::/32 dev lo metric 1024 error -113 pref medium
unreachable 2002:ac10::/28 dev lo metric 1024 error -113 pref medium
unreachable 2002:c0a8::/32 dev lo metric 1024 error -113 pref medium
unreachable 2002:e000::/19 dev lo metric 1024 error -113 pref medium
2600:1f14:589:f01::/64 dev ens5 proto kernel metric 256 pref medium
unreachable 3ffe:ffff::/32 dev lo metric 1024 error -113 pref medium
fe80::/64 dev ens5 proto kernel metric 256 mtu 9001 pref medium
default via fe80::41c:cff:fe68:6810 dev ens5 proto ra metric 1024 expires 1798sec hoplimit 64 pref medium

Now the device is learning the default gw via ra (Router Advertisement) because the script /etc/sysconfig/network-scripts/ifup-ipv6

is doing:

# Set some proc switches depending on defines
if [ "$IPV6FORWARDING" = "yes" ]; then
    # Global forwarding should be enabled

    # Check, if global IPv6 forwarding was already set by global script
    if [ $ipv6_global_forwarding_current -ne 1 ]; then
        net_log $"Global IPv6 forwarding is enabled in configuration, but not currently enabled in kernel"
        net_log $"Please restart network with '/sbin/service network restart'"
    fi

    ipv6_local_forwarding=1
    ipv6_local_auto=0
    ipv6_local_accept_ra=0
    if [ "$IPV6_ROUTER" = "no" ]; then
        ipv6_local_forwarding=0
    fi
    if [ "$IPV6_AUTOCONF" = "yes" ]; then
        ipv6_local_auto=1
        ipv6_local_accept_ra=2
    fi
else
    # Global forwarding should be disabled

    # Check, if global IPv6 forwarding was already set by global script
    if [ $ipv6_global_forwarding_current -ne 0 ]; then
        net_log $"Global IPv6 forwarding is disabled in configuration, but not currently disabled in kernel"
        net_log $"Please restart network with '/sbin/service network restart'"
    fi

    ipv6_local_forwarding=0
    ipv6_local_auto=1
    ipv6_local_accept_ra=1
    if [ "$IPV6_AUTOCONF" = "no" ]; then
        ipv6_local_auto=0
        if [ ! "$IPV6_FORCE_ACCEPT_RA" = "yes" ]; then
            ipv6_local_accept_ra=0
        fi
    fi
fi

if [ ! "$IPV6_SET_SYSCTLS" = "no" ]; then
    /sbin/sysctl -e -w net.ipv6.conf.$SYSCTLDEVICE.forwarding=$ipv6_local_forwarding >/dev/null 2>&1
    /sbin/sysctl -e -w net.ipv6.conf.$SYSCTLDEVICE.accept_ra=$ipv6_local_accept_ra >/dev/null 2>&1
    /sbin/sysctl -e -w net.ipv6.conf.$SYSCTLDEVICE.accept_redirects=$ipv6_local_auto >/dev/null 2>&1
    /sbin/sysctl -e -w net.ipv6.conf.$SYSCTLDEVICE.autoconf=$ipv6_local_auto >/dev/null 2>&1
fi
ubuntu-server-builder commented 1 year ago

Launchpad user Launchpad Janitor(janitor) wrote on 2019-09-09T04:17:34.589449+00:00

[Expired for cloud-init because there has been no activity for 60 days.]

ubuntu-server-builder commented 1 year ago

Launchpad user Irving Popovetsky(irving-popovetsky) wrote on 2019-09-09T15:37:55.741399+00:00

bump to keep alive

ubuntu-server-builder commented 1 year ago

Launchpad user Ryan Harper(raharper) wrote on 2019-11-20T17:55:23.308871+00:00

Merge Proposal migrated to github:

https://github.com/canonical/cloud-init/pull/51

ubuntu-server-builder commented 1 year ago

Launchpad user Ryan Harper(raharper) wrote on 2019-11-20T17:56:14.729995+00:00

https://github.com/canonical/cloud-init/commit/62bbc262c3c7f633eac1d09ec78c055eef05166a

ubuntu-server-builder commented 1 year ago

Launchpad user Chad Smith(chad.smith) wrote on 2019-12-18T17:55:19.650481+00:00

This bug is believed to be fixed in cloud-init in version 19.4. If this is still a problem for you, please make a comment and set the state back to New

Thank you.