libcompose may not handle shm_size correctly

[iankronquist]

A docker-compose.yml file errors with libcompose but succeeds with docker-compose. This may have to do with the handling of shm_size.

$ docker-compose --version
docker-compose version 1.7.1, build 6c29830

libcompose commit 8ee7bcc364f7b8194581a3c6bd9fa019467c7873 (Thu May 12).

I have a simple docker-compose.yml file:

---
mysql:
  shm_size: 64M
  image: mysql
  environment:
    MYSQL_PASSWORD: wp
    MYSQL_USER: wp
    MYSQL_DATABASE: wp
    MYSQL_ROOT_PASSWORD: root

I can easily bring this up with the docker-compose command:

$ docker-compose up -d mysql
Creating minimal_mysql_1

Great, let's try to reproduce this with libcompose:

package main

import (
        "log"
        "github.com/docker/libcompose/docker"
        "github.com/docker/libcompose/project"
        "github.com/docker/libcompose/project/options"
)

func main() {
        containers := []string{"mysql"}
        proj, err := docker.NewProject(&docker.Context{
                Context: project.Context{
                        ComposeFiles: []string{"docker-compose.yml"},
                        ProjectName: "beeswax",
                },
        });
        if err != nil {
                panic(err)
        }

        for _, containerName := range containers {
                err = proj.Up(options.Up{options.Create{true, false, false}}, containerName)
                if err != nil {
                        panic(err)
                }
                log.Printf("Starting the %s container\n", containerName);
    }
}

Compile and run:

$ go build && ./minimal

I get this ugly error:

./minimal 
INFO[0000] Project [beeswax]: Starting project          
INFO[0000] [0/1] [mysql]: Starting                      
ERRO[0000] Failed Starting mysql : Error response from daemon: SHM size must be greater then 0 
ERRO[0000] Failed to start: mysql : Error response from daemon: SHM size must be greater then 0 
panic: Error response from daemon: SHM size must be greater then 0

goroutine 1 [running]:
main.main()
    /home/vagrant/gopath/src/github.com/iankronquist/minimal/main.go:25 +0x347

goroutine 5 [syscall]:
os/signal.loop()
    /usr/lib/golang/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
    /usr/lib/golang/src/os/signal/signal_unix.go:27 +0x35

goroutine 6 [runnable]:
github.com/docker/libcompose/project.(*defaultListener).start(0xc2080d67a0)
    /home/vagrant/gopath/src/github.com/docker/libcompose/project/listener.go:48 +0x8e
created by github.com/docker/libcompose/project.NewDefaultListener
    /home/vagrant/gopath/src/github.com/docker/libcompose/project/listener.go:43 +0xa6

goroutine 10 [IO wait]:
net.(*pollDesc).Wait(0xc2080da140, 0x72, 0x0, 0x0)
    /usr/lib/golang/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080da140, 0x0, 0x0)
    /usr/lib/golang/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080da0e0, 0xc2080fe000, 0x1000, 0x1000, 0x0, 0x7f3836bd5190, 0xc20811dbc8)
    /usr/lib/golang/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20802c670, 0xc2080fe000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/lib/golang/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f3836bd7868, 0xc20802c670, 0xc2081414f8, 0xc2080fe000, 0x1000, 0x1000, 0x8100e0, 0x0, 0x0)
    /usr/lib/golang/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081600e0, 0xc2080fe000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
    <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208031a40)
    /usr/lib/golang/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc208031a40, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/lib/golang/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2081414a0)
    /usr/lib/golang/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
    /usr/lib/golang/src/net/http/transport.go:660 +0xc9f

goroutine 11 [select]:
net/http.(*persistConn).writeLoop(0xc2081414a0)
    /usr/lib/golang/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
    /usr/lib/golang/src/net/http/transport.go:661 +0xcbc

[vdemeester]

hey @iankronquist, Thank you for the report :wink:. You're right it fails miserably on 2 points : it did not deserialize it right (the shm_size) and it panic where it shouldn't have..

I'll take a look at it soon, and I've added it to the next release :)

[iankronquist]

@vdemeester, actually I think the panic comes from my proof of concept code which doesn't handle errors properly so as to be as simple as possible. Thanks for the quick reply.

[kunalkushwaha]

I tried to reproduce this issue, but it works fine at my machine. Used same libcompose code with docker 1.11.1 version.

What is your docker-engine version?

[iankronquist]

@kunalkushwaha docker --version Docker version 1.9.1, build 78ee77d/1.9.1 I'm just using the CentOS 7 Docker package, and nothing fancy like EPEL or Docker specific repositories.

[kunalkushwaha]

I think that is the problem. There were few significant changes in 1.10 release.

shm_size was also added in 1.10 release.

Upgrading docker engine will fix your problem.

[iankronquist]

@kunalkushwaha I'm confused. If it doesn't exist in Docker 1.9 why would I be getting that error?

[kunalkushwaha]

@iankronquist I guess, I misinterpreted the release notes, from 1.10, the modification of shm_size was allowed.

But yes, with 1.11 release I am not able to reproduce it. Need to dig into code, serialization of shm_size must have been fixed. I guess something like https://github.com/docker/docker/blob/14b5a50f0a95a2b8f3ed7ab7469bbab02189a09c/daemon/daemon_unix.go#L245

[joshwget]

@iankronquist @kunalkushwaha This should be fixed on master. Do one of you want to try it out and verify your issue is no longer present?

[joshwget]

Just tested this and it seems to be working. If anyone encounters any issues around shm_size, please open a new issue!