kzahel commented 11 years ago

I am wondering if it is possible to run LXC containers in my chroot. Has anyone tried this? I tried to create one in ubuntu precise but got an error "wrong fs type on /dev/shm"

I think running a container inside a chroot may not be a supported use case for lxc yet. But for chromeos it's sort of our only option, aside from doing a dev_install and some portage overlay magic?

I'm just kind of rambling here, but I couldn't find any information about LXC on chromeOS and this seemed like a reasonable enough place to find some.

drinkcat commented 11 years ago

I gave a quick try at LXC, but I didn't manage to get it to work on my Arch laptop, so I got bored of it and gave up (maybe I should try again)... Instead I went for a lower level approach (dirty branch here: https://github.com/drinkcat/chroagh/commits/container.tmp), and I managed to "boot" Arch in a container, directly from Chrome OS. My idea is that it would be neat to start the crouton chroots in their own containers, so we can use systemd services, shut down chroots in a cleaner way (processes belonging to the chroot are clearly separated), control hardware access, etc.

So, in short, kernel support is available, and it's something I'd really like to get working. The only issue I have now is that the kernel for Samsung ARM is still stuck on 3.4, so it lacks some features, like being able to rejoin an existing container (important for crouton), or create a new instance of /dev/pts (not so critical for crouton, more so if you care about container isolation...).

Now, back to LXC, I see 2 options:

Compile it statically in a chroot (no idea how difficult that may be), and move it to /usr/local/bin in Chrome OS. Chances are it may work...
Find out what prevents it to work from inside the chroot. It looks like it's complaining about /dev/shm being a bind mount, you could try to remount /dev/shm with something like mount -t tmpfs tmpfs /dev/shm, and see where it fails next...

DennisLfromGA commented 11 years ago

@Kyle - Hadn't heard of 'LXC Containers' but after Googling it I think this concept sounds pretty exciting and promising - 'a chroot on steroids' is one description. I'll play around with it on a desktop Ubuntu install and see what happens...

@drinkcat - Are 'croagh' 'c_h_roagh' the same thing? I had a ' croagh:031df18f' quantal chroot installed with kde, unity & xfce a while back but haven't played with it in a while, I'll tinker with it again too. If they are different, I'll get the 'chroagh' tarball and try it out.

On Thu, Sep 12, 2013 at 12:04 AM, drinkcat notifications@github.com wrote:

I did a quick try at LXC, but I didn't manage to get it to work on my Arch laptop, so I got bored of it and gave up (maybe I should try again)... Instead I went for a lower level approach (dirty branch here: https://github.com/drinkcat/chroagh/commits/container.tmp), and I managed to "boot" Arch in a container, directly from Chrome OS. My idea is that it would be neat to start the crouton chroots in their own containers, so we can use systemd services, shut down chroots in a cleaner way (processes belonging to the chroot are clearly separated), control hardware access, etc.

So, in short, kernel support is available, and it's something I'd really like to get working. The only issue I have now is that the kernel for Samsung ARM is still stuck on 3.4, so it lacks some features, like being able to rejoin an existing container (important for crouton), or create a new instance of /dev/pts (not so critical for crouton, more so if you care about container isolation...).

Now, back to LXC, I see 2 options:

Compile it statically in a chroot (no idea how difficult that may be), and move it to /usr/local/bin in Chrome OS. Chances are it may work...

Find out what prevents it to work from inside the chroot. It looks like it's complaining about /dev/shm being a bind mount, you could try to remount /dev/shm with something like mount -t tmpfs tmpfs /dev/shm, and see where it fails next...

— Reply to this email directly or view it on GitHubhttps://github.com/dnschneid/crouton/issues/364#issuecomment-24293903 .

DennyL@GMail

drinkcat commented 11 years ago

@DennisLfromGA :

croagh was the name of a branch of crouton, written by @dnschneid, that allows multiple distributions to be supported (currently Debian and Ubuntu). It was merged a few months back now.
chroagh is my own repository (https://github.com/drinkcat/chroagh), where I stage branches before pushing them here. The master branch is a fork of crouton, with Arch support, and a few other branches that are not pushed to crouton yet. It should be useable, but is a bit outdated currently as I haven't backported the latest modifications in crouton (I'm waiting for my audio branch to get in before I rebase again).

DennisLfromGA commented 11 years ago

@drinkcat, Thanx for clearing that up, I guess I didn't understand exactly what it was and what David had done with it.

I had downloaded your 'chroagh' repo (7a16b4b) but hadn't installed it and played with it yet. From reading your comment above '"boot" Arch in a container, directly from Chrome OS', it sounds a little different from a 'crouton' chroot, can you explain it a little more for me please? I'll wait until you've rebased again and then give it a whirl.

P.S. I'm due to get my 128GB SSD for my Acer C7 today so I'm excited!

On Fri, Sep 13, 2013 at 10:44 AM, drinkcat notifications@github.com wrote:

@DennisLfromGA https://github.com/DennisLfromGA :

croagh was the name of crouton's branch, written by @dnschneidhttps://github.com/dnschneid, that allows multiple distributions to be supported (currently Debian and Ubuntu). It was merged a few months back now.

chroagh is my own repository (https://github.com/drinkcat/chroagh), where I stage branches before pushing them here. The master branch is a fork of crouton, with Arch support, and a few other branchs that are not pushed to crouton yet. It should be useable, but is a bit outdated currently as I haven't backported the latest modifications in crouton (I'm waiting for my audio branch to get in before I rebase again).

— Reply to this email directly or view it on GitHubhttps://github.com/dnschneid/crouton/issues/364#issuecomment-24399405 .

DennyL@GMail

drinkcat commented 11 years ago

@DennisLfromGA. First off, the container thing will not be in my next rebase, there is still quite a bit of work to be done, and I need to wait for kernel 3.8 to be pushed on the ARM Chromebook to get some important features. On the other hand, you are still welcome to test ArchLinux ,-)

About containers vs chroot:

A normal chroot just changes the root filesystem of the current process. This is by no means secure (there are easy ways to escape), and does not really isolate processes you run in the chroot from the outside.

Namespaces are far more powerful. It's basically a set of kernel features that allows you to separate a set of process from another, very similar to a virtual machine, but without the overhead of that. You can choose at what level you want to isolate the processes: mount, PID, network, user, etc. (see this article for details: https://lwn.net/Articles/531114/). LXC, systemd-nspawn are tools using those features.

My main reason for trying this out is issue https://github.com/drinkcat/chroagh/issues/8 by @kdb424. systemd, the init system shipped with Archlinux, is not able to start services, like ssh, in a chroot (unlike upstart used by Ubuntu).

By using a PID namespace, you can start a "proper" init process, with PID 1, and you "boot" the system: you see init starting services, and then you get a login prompt: just like when you boot a regular Linux machine. When you are done with the container, you type "halt", and it shuts down, terminating all the processes in the container. So neat and satisfying ,-)

Using a mount namespace is also useful: no more polluting of the output of mount on Chrome OS with lots of bind mounts. And all the mounts are automatically removed when the container shuts down.

Finally, user namespace allows a normal user (chronos on Chrome OS) to be root inside the chroot. I haven't tested that yet (it needs kernel 3.8), but I think that would mean we would not need to add sudo in front of most crouton commands, and would make sure the the chroot could not change kernel settings (stuff in /proc/sys or /sys). This might not be wanted in all cases, though.

UTS namespace would allow a different hostname in chroot. And network namespace could isolate the chroot from the network. Not quite sure why we would want either of that, but we could do it ,-)

In short, quite a bit of potential I think. Still a lot of work to be done to see exactly what it can bring to crouton, and where are the limitations when using some of the namespaces.

DennisLfromGA commented 11 years ago

Thanx @drinkcat for the explanation, I'll read through the article you mentioned and learned a little more about it. Hopefully, the ARM is not too far away from 3.8, it looks like we non-ARM users are on 3.8.11. The namespaces and containers do look promising for crouton though, I tried to change the hostname in a chroot on crouton and it was a disaster.

On Sat, Sep 14, 2013 at 12:12 AM, drinkcat notifications@github.com wrote:

@DennisLfromGA https://github.com/DennisLfromGA. First off, the container thing will not be in my next rebase, there is still quite a bit of work to be done, and I need to wait for kernel 3.8 to be pushed on the ARM Chromebook to get some important features. On the other hand, you are still welcome to test ArchLinux ,-)

About containers vs chroot:

A normal chroot just changes the root filesystem of the current process. This is by no means secure (there are easy ways to escape), and does not really isolate processes you run in the chroot from the outside.

Namespaces are far more powerful. It's basically a set of kernel features that allows you to separate a set of process from another, very similar to a virtual machine, but without the overhead of that. You can choose at what level you want to isolate the processes: mount, PID, network, user, etc. (see this article for details: https://lwn.net/Articles/531114/). LXC, systemd-nspawn are tools using those features.

My main reason for trying this out is issue drinkcat#8https://github.com/drinkcat/chroagh/issues/8by @kdb424 https://github.com/kdb424. systemd, the init system shipped with Archlinux, is not able to start services, like ssh, in a chroot (unlike upstart used by Ubuntu).

By using a PID namespace, you can start a "proper" init process, with PID 1, and you "boot" the system: you see init starting services, and then you get a login prompt: just like when you boot a regular Linux machine. When you are done with the container, you type "halt", and it shuts down, terminating all the processes in the container. So neat and satisfying ,-)

Using a mount namespace is also useful: no more polluting of the output of mount on Chrome OS with lots of bind mounts. And all the mounts are automatically removed when the container shuts down.

Finally, user namespace allows a normal user (chronos on Chrome OS) to be root inside the chroot. I haven't tested that yet (it needs kernel 3.8), but I think that would mean we would not need to add sudo in front of most crouton commands, and would make sure the the chroot could not change kernel settings (stuff in /proc/sys or /sys). This might not be wanted in all cases, though.

UTS namespace would allow a different hostname in chroot. And network namespace could isolate the chroot from the network. Not quite sure why we would want either of that, but we could do it ,-)

In short, quite a bit of potential I think. Still a lot of work to be done to see exactly what it can bring to crouton, and where are the limitations when using some of the namespaces.

— Reply to this email directly or view it on GitHubhttps://github.com/dnschneid/crouton/issues/364#issuecomment-24436430 .

DennyL@GMail

alexanderkyte commented 11 years ago

Have you considered something like docker? It wraps the lxc subsystem in a pretty nice way.

TreverN commented 11 years ago

I'm trying to get docker going on my Google I/O Stumpy's.

Docker wants a 3.8 kernel. Looks like Pixel's and Samsung 550's run a 3.8 kernel as of 7/22/2013, but not Stumpy yet.

Anyone on a Pixel or Sammy 550 know if docker with crouton/Ubuntu "just works"? I'm very new to playing with Docker and LXC so I could be naive here...
Anyone know if Chrome OS Beta or Dev is running a 3.8 kernel on Stumpy? Or does anyone know when 3.8 kernel will come to hardware (if it will) other than Pixel's and 550s?

Thanks...

Later: Answered #2 myself, dev channel is using 3.8.11. Will report back when I've answered #1...

alexanderkyte commented 11 years ago

The Samsung arm chromebook still appears to be on 3.4.

On Sun, Sep 22, 2013 at 12:50 AM, TreverN notifications@github.com wrote:

I'm trying to get docker going on my Google I/O Stumpy's.

Docker wants a 3.8 kernel. Looks like Pixel's and Samsung 550's run a 3.8 kernel as of 7/22/2013, but not Stumpy yet.

1.

Anyone on a Pixel or Sammy 550 know if docker with crouton/Ubuntu "just works"? I'm very new to playing with Docker and LXC so I could be naive here... 2.

Anyone know if Chrome OS Beta or Dev is running a 3.8 kernel on Stumpy? Or does anyone know when 3.8 kernel will come to hardware (if it will) other than Pixel's and 550s?

Thanks...

— Reply to this email directly or view it on GitHubhttps://github.com/dnschneid/crouton/issues/364#issuecomment-24876085 .

dnschneid commented 11 years ago

Eventually all devices will be on 3.8, but it will be a while.

alexanderkyte commented 11 years ago

Is there an easy way to test out 3.8 on one of the older devices? On Sep 23, 2013 8:46 PM, "David Schneider" notifications@github.com wrote:

Eventually all devices will be on 3.8, but it will be a while.

— Reply to this email directly or view it on GitHubhttps://github.com/dnschneid/crouton/issues/364#issuecomment-24967388 .

dnschneid commented 11 years ago

Switching to dev channel is the easiest, assuming it has 3.8 enabled for dev channel. Otherwise you'd have to compile chromium yourself.

alexanderkyte commented 10 years ago

Docker no longer requires a kernel to have aufs compiled into it.

http://blog.docker.io/2013/11/docker-0-7-docker-now-runs-on-any-linux-distribution/

ccaapton commented 10 years ago

It seems that most chromebooks are using kernel >= 3.8. Is there any plan to utilize PID/user/mount namespaces?

BTW, I think the stock wrapper, /sbin/minijail0 is very capable. It seems to support mount(including bind)/network/pid namespaces. But I'm not shure if it can be used by non-root user.

jpw1116 commented 10 years ago

Hmm. I have been checking daily for weeks for any updates pushed down the pipe but my Samsung Snowy (ARM v7) is still at 3.4.0. I'm hesitant to go upstream with associated risk.

tedm commented 10 years ago

Yes, stable on the ARM is still 3.4. I've been battling the dev channel and crouton for awhile, so yesterday I power washed, and reverted to stable, so far crouton, with chromium, and xfce targets on default precise release are working very well.

drinkcat commented 10 years ago

It looks like ARM may get 3.8 soon, I see kernel 3.8.11 being built in the Chromium OS builders: http://build.chromium.org/p/chromiumos/builders/daisy%20full/builds/6104/steps/BuildPackages/logs/stdio So it looks like it's just a matter of time until 3.8 appears on the dev channel (if it's not already there).

tedm commented 10 years ago

Yes, I think you're right @drinkcat - I had to powerwash and revert to stable, as even the most basic crouton functions weren't working on the dev channel. After reverting to stable, crouton (xfce, chromium) on precise works well. I expect that the dev channel will be upgraded soon, as there were a multitude of other issues, such as extension flags not being preserved, and random crashes, (more than usual) ... I wonder what the beta channel is like ? I'm surprised at the lack of testing on ARM systems, as it's by far the most popular platform for the Chromebooks, from the data I"ve seen.

dnschneid commented 10 years ago

Dev channel is testing.

tedm commented 10 years ago

right. But basic functionality, IMHO, such as extensions and flags still working, ctrl-keys in crosh behaving, not crashing frequently, should generally be supported in a dev channel. Again, IMHO.

tedm commented 10 years ago

@dnschneid and all: I apologize for complaining about my issues with the Dev channel. I should be grateful that for almost a year, I was able to function ok with Crouton while running in it, and shouldn't have taken that for granted.

Thank you for retaining Crouton support for those of us on the Samsung ARM, whose stable channel is kernel 3.4.0 and may be so for a while.

Crouton has probably been the most useful software ever for Chromebook users. Thanks David, Drinkcat, and all Crouton developers.

dnschneid commented 10 years ago

@tedm it sounds like beta channel would be ideal for you; it's a pretty happy medium between stability and previewing new features.

tedm commented 10 years ago

Yeah, beta would probably be fine to move to for me, and also since USB sticks, and catching up, aren't needed to change channels anymore on the arm, just powerwashes, it's now simpler to change channels.

If the desktop environments, or the clipboard extensions are ARM ready, I'm ready to test those.

air commented 10 years ago

On my Acer C720 Chromebook (reports kernel 3.8.11 with uname -r), I have the docker 0.7.5 daemon running in a crouton chroot with Ubuntu raring. I'm using

sudo docker -d -b none

However I can't yet start a container. Current error is

$ sudo docker run -i -t ubuntu /bin/bash
WARNING: IPv4 forwarding is disabled.
lxc-start: No such file or directory - failed to create '/sys/fs/cgroup/cpu/lxc' directory
lxc-start: failed to spawn '6e7d451def231c2eb1f6ec84fbdac605f69d9a799d196135fb82f49ee187997c'
[error] commands.go:2458 Error getting size: bad file descriptor

air commented 10 years ago

The issue appears to be that ChromeOS can see the mounts under /sys/fs/cgroup, but the chroot OS cannot. In ChromeOS:

chronos@localhost / $ mount|grep cgroup
none on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,relatime,mode=755)
cgroup on /sys/fs/cgroup/cpu type cgroup (rw,nosuid,nodev,noexec,relatime,cpu)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
chronos@localhost / $ ls -l /sys/fs/cgroup
total 0
drwxr-xr-x 4 root root 0 Jan 14 14:29 cpu
drwxr-xr-x 2 root root 0 Jan 14 14:29 freeze

Whereas in Ubuntu raring chroot:

(raring)air@localhost:~/dev$ mount|grep cgroup
(raring)air@localhost:~/dev$ ls -l /sys/fs/cgroup
total 0

I see that /usr/local/chroots/raring/proc is empty, so I assume the chroot inherits the mounts from ChromeOS.

@dnschneid, sorry for my ignorance - could take a moment to explain why the cpu and freezer mounts under cgroup don't make it into the chroot? Hopefully this isn't a blocker for the whole Docker approach.

dnschneid commented 10 years ago

@air: /sys isn't mounted recursively, nor is it mounted shared such that new mounts will appear inside the chroot. Try doing something similar to the /media mount in enter-chroot. You'd probably want chroot /sys to be a slave mount of the system /sys, so you'd --make-rshared /sys, then --rbind it in, then --make-rslave the chroot's /sys.

air commented 10 years ago

Thanks @dnschneid, I patched my enter-chroot to set up a slaved /sys space:

# Bind-mount /sys recursively in order to get working cgroups
echo "Mounting /sys recursively"
mount --make-rshared /sys
mount --rbind /sys "$CHROOT/sys"
mount --make-rslave /sys "$CHROOT/sys"

This works - inside the chroot we can now see /sys/fs/cgroup/cpu and all the contents. We no longer fail on the filesystem error.

This brings us to the next error, a 'device busy' from lxc-start:

(raring)air@localhost:~/dev$ sudo docker run -i -t ubuntu /bin/bash 
WARNING: IPv4 forwarding is disabled.
lxc-start: Device or resource busy - failed to mount a new instance of '/dev/pts'
lxc-start: failed to setup the new pts instance
lxc-start: failed to setup the container
lxc-start: invalid sequence number 1. expected 2
lxc-start: failed to spawn '6e282af6df4d902fcf7bf26aed756d354f99a0ddcde718bb3a72593bf80da32b'
[error] commands.go:2458 Error getting size: bad file descriptor

I'd like to run lxc-checkconfig but this currently returns

Kernel configuration not found at /proc/config.gz; searching...
lxc-checkconfig: unable to retrieve kernel configuration
Try recompiling with IKCONFIG_PROC, installing the kernel headers,
or specifying the kernel configuration path with:
  CONFIG=<path> lxc-checkconfig

So next step is perhaps to sanity check that LXC is capable of running anything. Any advice appreciated : )

air commented 10 years ago

So lxc-checkconfig isn't going to work because it's looking for kernel headers. For example it attempts this file read (found with strace), /lib/modules/3.8.11/build/.config. If this was ordinary Ubuntu (not in a chroot) we would just apt-get install linux-headers-generic and the 3.8.11 package would get placed in that location.

However with ChromeOS as the host, we are running a non-Ubuntu kernel 3.8.11, from Google. Ubuntu does not have headers for non-Ubuntu kernels, so apt-get is not going to help.

It does appear possible to get ChromeOS kernel headers by building them, but it looks a bit hairy: http://superuser.com/questions/657845/cannot-install-3-4-linux-headers-on-acer-c7-chromebook-running-chrubuntu-12-04

I will give up on lxc-checkconfig for now.

air commented 10 years ago

OK, so I didn't give up quite yet : ) Here's the output of CONFIG=/usr/src/linux-headers-3.8.11/.config lxc-checkconfigon a fresh chroot (with no cgroup binding yet - observe 'required' below):

--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled
Multiple /dev/pts instances: missing

--- Control groups ---
Cgroup: enabled
Cgroup namespace: required
Cgroup device: missing
Cgroup sched: enabled
Cgroup cpu account: missing
Cgroup memory controller: missing
Cgroup cpuset: missing

--- Misc ---
Veth pair device: enabled
Macvlan: missing
Vlan: missing
File capabilities: enabled

To get the magic .config file I built the kernel headers as described in the crouton wiki. Note that I didn't bother making the linux-image, just the headers. This gets you the file linux-headers-3.8.11_3.8.11-10.00.Custom_amd64.deb which you install with dpkg.

air commented 10 years ago

You don't have to fully build the Linux headers to get the config file to check against. The only step you need is make oldconfig as described in the wiki (only takes a few seconds versus many minutes for the headers). This generates a .config at which you can point lxc-checkconfig.

It looks like the LXC tool is just checking flags in the config. So e.g. Multiple /dev/pts instances: missing is just reporting that in the config, # CONFIG_DEVPTS_MULTIPLE_INSTANCES is not set.

Could someone more expert comment on what this means, maybe @ccaapton or @dnschneid? My interpretation here is that features required for LXC are simply not compiled into the ChromeOS kernel (3.8 branch); so LXC cannot function on Chromebooks. Is that correct?

air commented 10 years ago

Here's the explanation of what CONFIG_DEVPTS_MULTIPLE_INSTANCES is for. It was added specifically for container support.

drinkcat commented 10 years ago

@air: Is there a way to get LXC to run without another instance of /dev/pts?

I understand this is desired for additional separation between the host and the container, but it is possible to start containers without it.

ZoomyCat commented 10 years ago

@air I have a basic chroot script on my blog located here: http://cuttingedgelinux.com/2013/03/10/command-line-interface-chroot-access-another-linux-from-your-current-linux/

In that script it requires a bindmount to /proc /sys /dev and /dev/pts

I would think that if you bindmounted as you did with /sys above for these four directories. Doing /dev before /dev/pts it should work in the most basic manner we could expect.

ccaapton commented 10 years ago

Hi, guys. I played with chromeos's own minijail, and I'm able to enter my crouton folder using minijail instead of enter-chroot. It also supports new PID namespace, so ps/top will only show stuff inside the chroot. The global mount points are not polluted in chromeos.

Here is the command: "sudo minijail0 -b /proc,/proc,1 -b /dev,/dev,1 -b /sys,/sys,1 -C /usr/local/chroots/precise/ -u 1000 -g 1000 -p -vr /bin/bash"

So it is very close to LXC, right?

UPDATE: On my current C720 stable channel. PID 1 is still occupied by minijail0, but this update https://chromium.googlesource.com/chromiumos/platform/minijail/+/88f76a7746083aaeff0d714571945306f6f06c2d will allow us to have the child process running as PID 1.

drinkcat commented 10 years ago

@ccaapton: Nice! Yes, that's, at least, a basic Linux container (LXC is just wrapper scripts).

-b /proc,/proc,1 should not be there: /proc must be mounted again within the container, not bind-mounted (since PIDs are different, /proc is different). I suspect minijail is overriding your command line parameter (probably because of -r), ps would not be working fine otherwise...

I'm curious if we can get systemd to "boot" within minijail...

ccaapton commented 10 years ago

@drinkcat I have successfully run bash as PID 1. Here are the steps:

Step 1: In the chromeos shell, run

sudo minijail0 -b /dev,/dev,1 -C /usr/local/chroots/precise -r /bin/bash

Step 2: Now we are in new mount namespace. Compile http://lwn.net/Articles/533492/ source code as "ns_child_exec", and run it:

ns_child_exec -p /bin/bash

Step 3: Now we are in new pid namespace with bash as the PID 1.

echo $$

1 :)

Possible Stage 4: At this stage, we can mount procfs to make it visible for the whole session:

mount -t proc proc /proc

Now you can "ps" to verify it

I can not test if systemd works, as currently I don't have any chroot containing systemd. Could anyone try systemd with this?

ZoomyCat commented 10 years ago

I will certainly be playing around with this the next few days during my weekend. I really want to get systemd working properly and if this works correctly and is easy to implement I am definitely down for some testing!

alexanderkyte commented 10 years ago

But will there be any barriers to sharing X this way? On Feb 17, 2014 5:24 AM, "ccaapton" notifications@github.com wrote:

@drinkcat https://github.com/drinkcat I have successfully run bash as PID 1. Here are the steps:

Step 1: In the chromeos shell, run sudo minijail0 -b /dev,/dev,1 -C /usr/local/chroots/precise -r /bin/bash

Step 2: Now we are in new mount namespace. Compile http://lwn.net/Articles/533492/ source code as "ns_child_exec", and run it:

ns_child_exec -p /bin/bash

Step 3: Now we are in new pid namespace with bash as the PID 1.

echo $$

1 :)

Possible Stage 4: At this stage, we can mount procfs to make it visible for the whole session:

mount -t proc proc /proc

Now you can "ps" to verify it

I can not test if systemd works, as currently I don't have any chroot containing systemd. Could anyone try systemd with this?

Reply to this email directly or view it on GitHubhttps://github.com/dnschneid/crouton/issues/364#issuecomment-35244514 .

drinkcat commented 10 years ago

@alexanderkyte As long as we don't create a separate IPC namespace, and bind mount /tmp, we should be fine.

ccaapton commented 10 years ago

Hi, folks! I've already got LXC running upstart as PID 1, and booting the crouton directory. Here are my steps:

Before entering crouton, bind mount /usr/local sudo minijail0 -b /dev,/dev,1 -b /usr/local,/usr/local,1 -C /usr/local/chroots/precise -r /bin/bash
Download newest LXC source file from http://linuxcontainers.org/downloads
./autogen.sh && ./configure && make && sudo make install Now lxc is installed in chromeos, which is outside crouton chroot.
Exit crouton, create a conf in home directory. Say delta.conf: lxc.utsname = delta lxc.autodev = 1 lxc.rootfs = /usr/local/chroots/precise lxc.arch = amd64 lxc.tty=8
sudo lxc-start -n delta -f ~/delta.conf -d
sudo lxc-attach -n delta /bin/bash Now we are inside the LXC :)

@drinkcat, @dnschneid Can't wait to see this been integrated into crouton!

ZoomyCat commented 10 years ago

@ccaapton I am a little confused at which layer you are running step 3 on. Is this inside the crouton? If so how does this install lxc onto chrome os?

DennisLfromGA commented 10 years ago

@edge226 - Step 1. puts you in a precise chroot It's handy to have previously downloaded the LXC source file and place it in /usr/local/src or similar before you do Step 3.

@ccaapton - Step 3. gives me an error -

./autogen.sh: 6: ./autogen.sh: aclocal: not found

I tried linking aclocal.m4 to aclocal but it still bombs out for me.

ZoomyCat commented 10 years ago

@ccaapton dont you need to modify something to tell it to compile to /usr/local instead of the standard directory or is /usr/local the standard directory?

DennisLfromGA commented 10 years ago

@ccaapton - Sorry to have to hold my hand through this but after installing m4, Automake, & Autoconfig, when I run 'Step 3.' the 'make' seems to fail -

(saucy-x64-unity-desktop)root@localhost:/usr/local/src/lxc-0.9.0# ./autogen.sh && make && make install
+ test -d autom4te.cache
+ rm -rf autom4te.cache
+ aclocal -I config
+ autoheader
+ autoconf
+ automake --add-missing --copy
make: *** No targets specified and no makefile found.  Stop.

When I ran it again after the 'autogen.sh' step, it looks like it can't find a 'make' file -

(saucy-x64-unity-desktop)root@localhost:/usr/local/src/lxc-0.9.0# make                                                                               
make: *** No targets specified and no makefile found.  Stop.

Also, 'make install' fails -

(saucy-x64-unity-desktop)root@localhost:/usr/local/src/lxc-0.9.0# make install                                                                       
make: *** No rule to make target `install'.  Stop.

ccaapton commented 10 years ago

@DennisLfromGA Sorry, this is my fault. You need "./configure" after "./autogen.sh". Already updated my previous post.

DennisLfromGA commented 10 years ago

Okay, got it figured out. Had to run './configure --disable-capabilities' first, then 'make' & 'make install' worked fine...

DennisLfromGA commented 10 years ago

Close but no cigar. After creating the delta.conf file it complains about GLIBC_2.17 -

sudo lxc-start -n delta-saucy-x64 -f ~/delta-saucy-x64.conf -d
lxc-start: /lib64/libc.so.6: version `GLIBC_2.17' not found (required by /usr/local/lib/liblxc.so.0)

sudo lxc-attach -n delta-saucy-x64 /bin/bash
lxc-attach: /lib64/libc.so.6: version `GLIBC_2.17' not found (required by /usr/local/lib/liblxc.so.0)

I checked and I do have the files mentioned -

ll /usr/local/lib/libxc*
lrwxrwxrwx  1 root root   11 Feb 18 18:21 liblxc.so -> liblxc.so.0*
lrwxrwxrwx  1 root root   15 Feb 18 18:21 liblxc.so.0 -> liblxc.so.0.9.0*

But I guess GLIBC_2.17 isn't a part of this 3.4.0 OS. Here's what I'm using -

Installed  : Sat Feb 15 19:10:11 EST 2014
Node name  : localhost
Machine    : x86_64
Rel. name  : Chrome OS
OS Track   : beta-channel
OS Release : Linux 3.4.0
CROS Root  : ROOT-B (/dev/sda5)
OS Board   : parrot-signed-mp-v3keys
OS Coder   : Googler
OS Vers.   : 5116.80.0
Browser Ver: Google Chrome 33.0.1750.93

The 'croutonversion' output of my saucy chroot is -

(saucy-x64-unity-desktop)user@localhost:~$ croutonversion
crouton: version 1-20131220111353~desktops:63b0426f
release: saucy
architecture: amd64
targets: unity-desktop,unity
host: version 5116.80.0 (Official Build) beta-channel parrot

This is no biggy for me, I'll place this on hold for now and wait 'til it's a little more ready-for-prime-time or at least testing.

Thanx all, -DennisL

ZoomyCat commented 10 years ago

I had locale issues in building with the manjaro mix so I switched it over to the precise mix and I get different errors.

It fails while doing ./automake

Looks like I had some missing dependencies. running ./configure complained about python3-dev. after sudo apt-get install python3-dev it still complains there is no python3-dev...

drinkcat commented 10 years ago

@DennisLfromGA This kind of library issues are not unexpected, as Chromium OS ships different library versions. If there is a need to compile an binary in the chroot, for use in Chromium OS, it needs to be compiled statically.

Now, to get containers working, we have a few options:

Use minijail0. That's the nicest as the tool is already provided by Chromium OS. But, AFAIK, the tool is not able to join an existing container. Not nice, as we probably only want to have 1 instance of each chroot running at a time.
Use unshare/nsenter (we'll need to fetch the latest util-linux, build the tools statically). We also need to check that the tools can do everything we need to do (I suspect they do).
Use a custom tool (that's the approach I've taken in my old branch: https://github.com/drinkcat/chroagh/tree/container.tmp).

Finally, Chromebooks on kernel 3.4 wil probably need to stay with the current approach, as kernel 3.4 lacks an important feature: being able to join an existing namespace.

DennisLfromGA commented 10 years ago

@drinkcat - Thanx for the info., I think I'll just cool my heels on this for a while until it's a little more settled. I'll spend a my time playing with the new 'multi-profiles' feature in the canary channel, it's really slick. I made a screencast that shows off some of it's features, I've just scratched the surface I'm sure.

dnschneid / crouton

LXC containers? #364

sudo minijail0 -b /dev,/dev,1 -C /usr/local/chroots/precise -r /bin/bash

ns_child_exec -p /bin/bash

echo $$

mount -t proc proc /proc

ns_child_exec -p /bin/bash

echo $$

mount -t proc proc /proc