logsearch / logsearch-boshrelease

A BOSH-scalable Elasticsearch+Logstash+Kibana release
http://www.logsearch.io
Apache License 2.0
57 stars 46 forks source link

Verify/Ensure enhanced networking is enabled on 14.04 HVM stemcells once available #21

Closed sopel closed 10 years ago

sopel commented 10 years ago

Enhanced Networking on Linux Instances in a VPC is a highly desirable side effect when using HVM instances in a VPC:

With C3, R3, and I2 instances, you can enable enhanced networking capabilities. We support enhanced networking capabilities using single root I/O virtualization (SR-IOV). Enabling enhanced networking on your instance results in higher performance (packets per second), lower latency, and lower jitter.

To enable enhanced networking on your instance, you must ensure that its kernel has the required driver installed, and that you set the sriovNetSupport attribute for the instance.

As just mentioned in https://github.com/cityindex/logsearch/issues/359#issuecomment-42183628, the latter is the case for the current Amazon Linux 2014.03 and Ubuntu LTS 14.04 HVM images and should ideally be available automatically once BOSH offers 14.04 based stemcells too, but given the custom image creation process, this should be explicitly verified and requested upstream if need be.

sopel commented 10 years ago

:information_source: see Support for AWS Virtualization type 'hvm' ? for a related discussion in the BOSH developers group.

mrdavidlaing commented 10 years ago

Pivotal have started publishing Ubuntu 14.04 stemcells.

$ bosh public stemcells
+-------------------------------------------------------------+
| Name                                                        |
+-------------------------------------------------------------+
| bosh-stemcell-2524-aws-xen-ubuntu-trusty-go_agent.tgz       |
| light-bosh-stemcell-2524-aws-xen-ubuntu-trusty-go_agent.tgz |
+-------------------------------------------------------------+

However, these are not yet being promoted as "stable".

@sopel - do you know how we can check for:

To enable enhanced networking on your instance, you must ensure that its kernel has the required driver installed

sopel commented 10 years ago

Finally, glad to see (HVM images are missing still though, which are a requirement here - we should hopefully be able to simply do that ourselves though, if need be).

Testing Whether Enhanced Networking Is Enabled describes the two steps to verify that the driver is installed on your instance and that the sriovNetSupport attribute is set.

$ sudo ethtool -i eth0
driver: ixgbevf
version: 2.11.3
firmware-version: N/A
bus-info: 0000:00:03.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no

$ aws ec2 describe-instance-attribute --instance-id instance_id --attribute sriovNetSupport
    "SriovNetSupport": {
        "Value": "simple"
    },
mrdavidlaing commented 10 years ago

On an instance started from stemcell: bosh-stemcell-2524-aws-xen-ubuntu-trusty-go_agent.tgz - Ubuntu 14.04 LTS (GNU/Linux 3.13.0-24-generic x86_64)

# sudo ethtool -i eth0
driver: vif
version: 
firmware-version: 
bus-info: vif-0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

I think this means that it doesn't have the required driver

dpb587 commented 10 years ago

I ran this against elasticsearch_eu-west-1c/0 which is running as r3.2xlarge from bosh-aws-xen-ubuntu-trusty-go_agent-hvm@2549.

local$ aws ec2 describe-instance-attribute --instance-id i-5bef4b19 --attribute sriovNetSupport | jq -c -r '.SriovNetSupport'
{"Value":"simple"}
i-5bef4b19$ sudo apt-get install -y ethtool && sudo ethtool -i eth0
driver: ixgbevf
version: 2.11.3-k
firmware-version: 
bus-info: 0000:00:03.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no

@sopel, can you verify that these results confirm enhanced networking usage? Then I think this issue can be closed?

sopel commented 10 years ago

Excellent, thanks for integrating this :)

rchekaluk commented 10 years ago

I'm curious if anyone on this thread has succeeded in upgrading Ubuntu Trusty ixgbevf from version 2.11.3-k to the recommended most recent version 2.15.3.

In attempting to build ixgbevf from source, I've encountered this error, even after upgrading the kernel to 3.13.0-35-generic.

mrdavidlaing commented 10 years ago

@rchekaluk - I'm afraid we haven't tried, so can't comment