search

clearcontainers / runtime

OCI (Open Containers Initiative) compatible runtime using Virtual Machines
Apache License 2.0
589 stars 70 forks source link

Fail to start "ros" image with CC #569

Open thiagomacieira opened 7 years ago

thiagomacieira commented 7 years ago

When trying to run the "ros" image with CC, I get a cryptic error message:

$ docker run -it ros     
cat: -: No such file or directory

The same image works fine with runc. Clear Containers appears to be working fine:

$ docker run -it busybox cat /proc/version
Linux version 4.9.47-77.5.container (abuild@) (gcc version 7.2.1 20170901 [gcc-7-branch revision 251580] (SUSE Linux) ) #1 SMP Fri Sep 15 18:23:40 UTC 2017
sameo commented 7 years ago

@thiagomacieira I can't reproduce it on my setup:

$ docker run -ti --runtime=cc ros
root@6eee79f5ed32:/# uname -r
4.9.47-77.1.container
$ cc-runtime cc-env
root@6eee79f5ed32:/# exit
[Meta]
  Version = "1.0.3"

[Runtime]
  [Runtime.Version]
    Semver = "3.0.0-beta.2"
    Commit = "c9ccbb7ff61b3546506c975dcef9421f9b270366"
    OCI = "1.0.0-rc5"
  [Runtime.Config]
    GlobalLogPath = ""
    [Runtime.Config.Location]
      Path = "/etc/clear-containers/configuration.toml"
      Resolved = "/etc/clear-containers/configuration.toml"

[Hypervisor]
  MachineType = "pc"
  Version = "QEMU emulator version 2.7.0, Copyright (c) 2003-2016 Fabrice Bellard and the QEMU Project developers"
  [Hypervisor.Location]
    Path = "/usr/bin/qemu-lite-system-x86_64"
    Resolved = "/usr/bin/qemu-lite-system-x86_64"

[Image]
  Path = "/usr/share/clear-containers/clear-containers.img"
  Resolved = "/usr/share/clear-containers/clear-17270-containers.img"

[Kernel]
  Parameters = ""
  [Kernel.Location]
    Path = "/usr/share/clear-containers/vmlinuz.container"
    Resolved = "/usr/share/clear-containers/vmlinuz-4.9.47-77.1.container"

[Proxy]
  Type = "ccProxy"
  Version = "Version: 3.0.0-beta.2-18-g0d66493-dirty"
  URL = "unix:///var/run/clear-containers/proxy.sock"

[Shim]
  Type = "ccShim"
  Version = "shim version: 3.0.0-alpha.4 (commit: 0)"
  [Shim.Location]
    Path = "/usr/libexec/clear-containers/cc-shim"
    Resolved = "/usr/libexec/clear-containers/cc-shim"

[Agent]
  Type = "hyperstart"
  Version = "<<unknown>>"
  [Agent.PauseBin]
    Path = "/var/lib/clear-containers/runtime/bundles/pause_bundle/bin/pause"
    Resolved = "/var/lib/clear-containers/runtime/bundles/pause_bundle/bin/pause"

[Host]
  Kernel = "4.12.5-300.fc26.x86_64"
  CCCapable = true
  [Host.Distro]
    Name = "Fedora"
    Version = "26"
  [Host.CPU]
    Vendor = "GenuineIntel"
    Model = "Intel(R) Core(TM) i7-4558U CPU @ 2.80GHz"

Could you please dump your cc-runtime cc-env output ?

thiagomacieira commented 7 years ago 
[Meta]
  Version = "1.0.2"

[Runtime]
  [Runtime.Version]
    Semver = "3.0.0-beta.1"
    Commit = "3d44244608b5cbe9e1295520aa9f54a494954b23"
    OCI = "1.0.0-rc5"
  [Runtime.Config]
    GlobalLogPath = ""
    [Runtime.Config.Location]
      Path = "/etc/clear-containers/configuration.toml"
      Resolved = "/etc/clear-containers/configuration.toml"

[Hypervisor]
  MachineType = "pc"
  [Hypervisor.Location]
    Path = "/usr/bin/qemu-system-x86_64"
    Resolved = "/usr/bin/qemu-system-x86_64"

[Image]
  Path = "/usr/share/clear-containers/clear-containers.img"
  Resolved = "/usr/share/clear-containers/clear-17270-containers.img"

[Kernel]
  Parameters = ""
  [Kernel.Location]
    Path = "/usr/share/clear-containers/vmlinuz.container"
    Resolved = "/usr/share/clear-containers/vmlinuz-4.9.47-77.5.container"

[Proxy]
  Type = "ccProxy"
  Version = "<<unknown>>"
  URL = "unix:///var/run/clear-containers/proxy.sock"

[Shim]
  Type = "ccShim"
  Version = "shim version: 3.0.0-alpha.4 (commit: 0)"
  [Shim.Location]
    Path = "/usr/lib64/clear-containers/cc-shim"
    Resolved = "/usr/lib64/clear-containers/cc-shim"

[Agent]
  Type = "hyperstart"
  Version = "<<unknown>>"
  [Agent.PauseBin]
    Path = "/var/lib/clear-containers/runtime/bundles/pause_bundle/bin/pause"
    Resolved = "/var/lib/clear-containers/runtime/bundles/pause_bundle/bin/pause"

[Host]
  Kernel = "4.12.11-1-default"
  CCCapable = false
  [Host.Distro]
    Name = "openSUSE Tumbleweed"
    Version = "20170913"
  [Host.CPU]
    Vendor = "GenuineIntel"
    Model = "Intel(R) Core(TM) i7-6560U CPU @ 2.20GHz"
sameo commented 7 years ago

Would you mind upgrading to beta.2 and verifying that this still fails ?

thiagomacieira commented 7 years ago

Same error:

[Meta]
  Version = "1.0.2"

[Runtime]
  [Runtime.Version]
    Semver = "3.0.0-beta.2"
    Commit = "06023ea"
    OCI = "1.0.0-rc5"
  [Runtime.Config]
    GlobalLogPath = ""
    [Runtime.Config.Location]
      Path = "/etc/clear-containers/configuration.toml"
      Resolved = "/etc/clear-containers/configuration.toml"

[Hypervisor]
  MachineType = "pc"
  [Hypervisor.Location]
    Path = "/usr/bin/qemu-system-x86_64"
    Resolved = "/usr/bin/qemu-system-x86_64"

[Image]
  Path = "/usr/share/clear-containers/clear-containers.img"
  Resolved = "/usr/share/clear-containers/clear-17270-containers.img"

[Kernel]
  Parameters = ""
  [Kernel.Location]
    Path = "/usr/share/clear-containers/vmlinuz.container"
    Resolved = "/usr/share/clear-containers/vmlinuz-4.9.47-77.5.container"

[Proxy]
  Type = "ccProxy"
  Version = "<<unknown>>"
  URL = "unix:///var/run/clear-containers/proxy.sock"

[Shim]
  Type = "ccShim"
  Version = "shim version: 3.0.0-alpha.4 (commit: 1f545df)"
  [Shim.Location]
    Path = "/usr/lib64/clear-containers/cc-shim"
    Resolved = "/usr/lib64/clear-containers/cc-shim"

[Agent]
  Type = "hyperstart"
  Version = "<<unknown>>"
  [Agent.PauseBin]
    Path = "/var/lib/clear-containers/runtime/bundles/pause_bundle/bin/pause"
    Resolved = "/var/lib/clear-containers/runtime/bundles/pause_bundle/bin/pause"

[Host]
  Kernel = "4.12.11-1-default"
  CCCapable = false
  [Host.Distro]
    Name = "openSUSE Tumbleweed"
    Version = "20170913"
  [Host.CPU]
    Vendor = "GenuineIntel"
    Model = "Intel(R) Core(TM) i7-6560U CPU @ 2.20GHz"
thiagomacieira commented 7 years ago

If I edit /etc/clear-containers/configuration.toml and change the hypervisor to qemu-lite-system-x86_64, it runs.

grahamwhaley commented 7 years ago

To confirm, using the system qemu also fails for me, on ubuntu server 16.04. I enabled the runtime logs, and I see in there:

2017-09-19 10:55:20.023922685 +0100 BST:25281:cc-runtime:error:Unable to launch qemu: exit status 1
2017-09-19 10:55:20.025538082 +0100 BST:25281:cc-runtime:error:qemu-system-x86_64: Property '.nvdimm' not found

The system qemu version installed is: QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.8), Copyright (c) 2003-2008 Fabrice Bellard

Having a quick stare at the QEMU git, it looks like initial nvdimm support went in almost exactly at the time v2.5.0 was released (early Dec. 2015). I guess there was a later addition we rely upon. Conclusion: that nearly 2 year old QEMU is probably too old to have all the features we require. I don't think we actually have a minimum QEMU version listed.

grahamwhaley commented 7 years ago

An addition... I was going to ask how you got the system default QEMU path in your config file @thiagomacieira , but having a peek here: https://github.com/clearcontainers/runtime/blob/master/Makefile#L87 I wondered if you were on Centos or RHEL - but I see above you are on openSUSE. I'm not up on the QEMU packaging names. /cc @gorozco1 @jcvenegas for any input and thoughts.

thiagomacieira commented 7 years ago

@grahamwhaley, that came from the binary packages provided via OBS: https://build.opensuse.org/project/show/home:clearcontainers:clear-containers-3-staging

The .spec file doesn't appear to be doing any shenanigans to the config file, so that came from source. From your link it looks like it should be the lite version, but wasn't. I don't know why.

gorozco1 commented 7 years ago

@thiagomacieira we are currently building packages in staging and release... and it will take sometime to update.. it might be that. the service in OBS is building the releases.

gorozco1 commented 7 years ago

@thiagomacieira what is the output of this code in your Tumbleweed:

distro=$(for file in /etc/os-release /usr/lib/os-release; do if [ -e $file ]; then grep ^ID= $file|cut -d= -f2-|tr -d '"';  break; fi done)

echo $distro
thiagomacieira commented 7 years ago 
sh-4.4$ for file in /etc/os-release /usr/lib/os-release; do if [ -e $file ]; then grep ^ID= $file|cut -d= -f2-|tr -d '"';  break; fi; done
opensuse
gorozco1 commented 7 years ago

@thiagomacieira this is due a missing requirement in OpenSuSE.

There is PR in the work to fix the detection of the distro and apply the proper qemu-lite binary.

https://github.com/clearcontainers/packaging/pull/116