Cannot specify apt mirror and proxy

[stub42]

Required information

• Distribution: Ubuntu
• Distribution version: 16.04

  Issue description

There is no documented way to specify which apt mirror and apt proxy to use.

One of the first things you usually do after creating a container is install needed applications using apt. For this to be usable in countries without fast connections to the USA, you need to be able to specify the apt mirror and apt proxy to use.

The closest you can do at the moment seems to be to set the http_proxy environment variable, but this affects everything including apt and does not let you specify the local mirror to use.

Without this feature, users are forced to manually make the changes after instantiating a container, and applications using lxd need to each reimplement this behaviour (eg. snapcraft)

[jamesbeedy]

@stub42 I set these values by means of user-data in the lxd profile. Have you tried this?

[stgraber]

Correct, the expectation here is to use cloud-init through the user.user-data or user.vendor-data keys (vendor-data is overriden by user-data so perfect for host-wide settings).

[stub42]

@jamesbeedy No, I'd never heard of it (nor has Google). Which would make this a documentation issue. And maybe a UI issue, since I don't think we want to require in depth knowledge of cloud-init to drive lxd.

[stgraber]

https://github.com/lxc/lxd/blob/master/doc/configuration.md does mention the cloud-init keys.

LXD's design is not to care about what you're running inside the container, you can run any Linux based operating system and we really don't want to have to know how to configure all of them.

So that's why we use cloud-init as it's what's being used in the cloud for VM provisioning of a variety of Linux distributions and is done by passing simple text content through (can be accessed through the /dev/lxd interface).

[stub42]

No luck passing things though, having I think set the correct keys in the default profile and created a fresh container:

$ lxc config show emelda
name: emelda
profiles:
- default
[...]

$ lxc profile show default
name: default
config:
  user.user-data: |
    apt-mirror: http://th.archive.ubuntu.com/ubuntu/
    apt-http-proxy: http://10.0.4.1:8000
    apt-proxy: http://10.0.4.1:8000
    apt_mirror: http://th.archive.ubuntu.com/ubuntu/
    apt_http_proxy: http://10.0.4.1:8000
    apt_proxy: http://10.0.4.1:8000
description: Default LXD profile
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: lxdbr0
    type: nic
$ lxc exec emelda -- bash
root@emelda:~# apt update
Hit:1 http://archive.ubuntu.com/ubuntu xenial InRelease                    
Get:2 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [95.7 kB]                          
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [94.5 kB]            
Get:4 http://archive.ubuntu.com/ubuntu xenial-backports InRelease [92.2 kB]            
0% [4 InRelease 10.0 kB/92.2 kB 11%] [3 InRelease 34.5 kB/94.5 kB 37%]^C

[stgraber]

The cloud-init documentation at https://cloudinit.readthedocs.io/en/latest/ certainly could do with some improvements to make it easier to discover the various options.

I did find the option for the apt mirror here though: http://cloudinit.readthedocs.io/en/latest/topics/examples.html#add-apt-repositories

https://git.launchpad.net/cloud-init/tree/doc/examples/cloud-config.txt lists a whole bunch more too, including apt_proxy.

I do believe @smoser and the cloud-init team are actively working on enhancing documentation, contributions would certainly be welcome.

[stgraber]

@stub42 cloud-init is an initial provisioning tool, it won't trigger after initial container startup, which is why it's not working above.

[stgraber]

stgraber@castiana:~$ lxc profile edit default
stgraber@castiana:~$ lxc launch ubuntu:16.04 test
Creating test
Starting test
stgraber@castiana:~$ lxc profile show default
name: default
config:
  user.user-data: |
    #cloud-config
    apt_mirror: http://us.archive.ubuntu.com/ubuntu/
description: ""
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: lxdbr0
    type: nic
stgraber@castiana:~$ lxc exec test -- apt update
Hit:1 http://us.archive.ubuntu.com/ubuntu xenial InRelease                                                               
Get:2 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease [95.7 kB]                                             
Get:3 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease [92.2 kB]                                  
Get:4 http://us.archive.ubuntu.com/ubuntu xenial/main Sources [868 kB]                                          
Get:5 http://us.archive.ubuntu.com/ubuntu xenial/restricted Sources [4808 B]                          
Get:6 http://us.archive.ubuntu.com/ubuntu xenial/universe Sources [7728 kB]                                    
Get:7 http://security.ubuntu.com/ubuntu xenial-security InRelease [94.5 kB]                                                                                                                                                                   
Get:8 http://security.ubuntu.com/ubuntu xenial-security/main Sources [32.8 kB]                                                                                                                                                                
Get:9 http://security.ubuntu.com/ubuntu xenial-security/universe Sources [8948 B]                                                                                                                                                             
Get:10 http://security.ubuntu.com/ubuntu xenial-security/multiverse Sources [728 B]                                                                                                                                                           
Get:11 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [123 kB]                                                                                                                                                         
Get:12 http://us.archive.ubuntu.com/ubuntu xenial/multiverse Sources [179 kB]                                                                                                                                                                 
Get:13 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [36.6 kB]                                                                                                                                                    
Get:14 http://us.archive.ubuntu.com/ubuntu xenial-updates/main Sources [174 kB]                                                                                                                                                               
Get:15 http://security.ubuntu.com/ubuntu xenial-security/universe Translation-en [22.0 kB]                                                                                                                                                    
Get:16 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe Sources [85.2 kB]                                                                                                                                                          
Get:17 http://us.archive.ubuntu.com/ubuntu xenial-updates/multiverse Sources [3220 B]                                                                                                                                                         
Get:18 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [358 kB]                                                                                                                                                        
Get:19 http://us.archive.ubuntu.com/ubuntu xenial-updates/main Translation-en [134 kB]                                                                                                                                                        
Get:20 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [311 kB]                                                                                                                                                    
Get:21 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe Translation-en [104 kB]                                                                                                                                                    
Get:22 http://us.archive.ubuntu.com/ubuntu xenial-backports/main Sources [1400 B]                                                                                                                                                             
Get:23 http://us.archive.ubuntu.com/ubuntu xenial-backports/universe Sources [800 B]                                                                                                                                                          
Get:24 http://us.archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages [1540 B]                                                                                                                                                      
Get:25 http://us.archive.ubuntu.com/ubuntu xenial-backports/main Translation-en [1484 B]                                                                                                                                                      
Get:26 http://us.archive.ubuntu.com/ubuntu xenial-backports/universe amd64 Packages [1000 B]                                                                                                                                                  
Get:27 http://us.archive.ubuntu.com/ubuntu xenial-backports/universe Translation-en [584 B]                                                                                                                                                   
Fetched 10.5 MB in 9s (1114 kB/s)                                                                                                                                                                                                             
Reading package lists... Done
Building dependency tree       
Reading state information... Done
5 packages can be upgraded. Run 'apt list --upgradable' to see them.

[stub42]

@stgraber The container was created after changing the default profile. I'd missed the '#cloud-config' line, which I'd assumed was a yaml comment in the example I was working from.

I agree we can't configure every possibility of what is installed in a container. However, I do think the Ubuntu experience should rock and we should most certainly support Ubuntu in containers with a decent user interface. I'm sure other vendors will.

[stgraber]

The Ubuntu way to provision a system, on the cloud, on the physical servers (MAAS) and soon to be on desktop and IoT (through Ubuntu Core) is cloud-init, so that's why we implemented this. It also provides a very nice separation between what LXD cares about (everything outside the container) and what cloud-init cares about (everything inside it).

The "#cloud-config" line you were missing is actually the file type of that cloud-init data. cloud-init does support a number of other input types, including compress binary encoded as base64 which Juju and some other provisioning tools make use of it through LXD.