search

docker / compose

Define and run multi-container applications with Docker
https://docs.docker.com/compose/
Apache License 2.0
33.6k stars 5.19k forks source link

TypeError: coercing to Unicode: need string or buffer, NoneType found #1593

Closed sstarcher closed 9 years ago

sstarcher commented 9 years ago

The first time I do docker-compose up -d everything works The second time I get the following error

Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 31, in main
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 21, in sys_dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 34, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 24, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 66, in perform_command
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 460, in up
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.project", line 221, in up
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.project", line 261, in _get_convergence_plans
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 278, in convergence_plan
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 101, in containers
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.container", line 28, in from_ps
TypeError: coercing to Unicode: need string or buffer, NoneType found
docker-compose --version
docker-compose version: 1.3.1
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
docker info
Containers: 8
Images: 98
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 114
 Dirperm1 Supported: false
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.13.0-53-generic
Operating System: Ubuntu 14.04.2 LTS
CPUs: 1
Total Memory: 992.5 MiB
Name: ip-10-0-11-99
ID: 3RM6:3ZR5:FK7P:N6TM:CE35:2VFA:BUJ7:UDBR:IUDR:OSB7:7BP2:TEHS
WARNING: No swap limit support
lmakarov commented 9 years ago

@sstarcher try downgrading to docker-compose v1.2.0

I've been having a similar issue with a php-fpm container. The root cause is the "affinity:container" ENV variable that is being added when docker-compose recreates containers.

See https://github.com/docker/compose/issues/1598 for details.

devinrsmith commented 9 years ago

+1

aanand commented 9 years ago

@sstarcher @devinrsmith I can't reproduce. Could you paste your docker-compose.yml? Was it previously working with Compose 1.2 or is this your first time running it?

sstarcher commented 9 years ago

@aanand I'll get you an example tomorrow. It happened on several of our servers with different docker compose files.

devinrsmith commented 9 years ago

It was previously working on 1.2. Broke on 1.3. Downgraded to 1.2, back and working.

tomfotherby commented 9 years ago

We see the same error from docker-compose 1.3.1 and docker 1.7.0 using AUFS. It's working for me, but exactly the same setup isn't working for my Greek colleague who gets the same error as in the issue. Downgrading to docker-compose 1.2.0 is one solution, but is there a better solution?

mrijken commented 9 years ago

I have the same issue. However, when I use docker-compose migrate-to-labels, the containers are created and started correctly.

mrijken commented 9 years ago

When I removed some old (dead and exited) containers, docker-compose up -d is working again.

prologic commented 9 years ago

Identical Issue here too

prologic commented 9 years ago

I might try to fix this tomorrow and submit a pull request (unless someone is already on it!)

wayt commented 9 years ago

Same issue here, I was able to fix it by removing some dead containers, everything is going well again.

prologic commented 9 years ago

b368272 at least tries to fix this; I'm not sure how get_container_name() should change for a "Dead" container though.

sstarcher commented 9 years ago

docker-compose 1.3.1 docker 1.7.0

docker-compose.yml

{
  "image1": {
    "image": "ubuntu",
    "volumes": [
      "/:/logstashforwarder:ro"
    ],
    "command": "sleep 10000"
  },
  "image2": {
    "image": "ubuntu",
    "command": "sleep 10000",
    "volumes": [
      "/:/hostfs:ro"
    ]
  }
}

Run the following twice docker-compose up -d

Recreating opt_image2_1...
Cannot destroy container 982a0d62eab102811a5a697a3df2de46fad5c7acd34a9bb5bea44e98d47fb1ee: Driver aufs failed to remove root filesystem 982a0d62eab102811a5a697a3df2de46fad5c7acd34a9bb5bea44e98d47fb1ee: rename /var/lib/docker/aufs/mnt/982a0d62eab102811a5a697a3df2de46fad5c7acd34a9bb5bea44e98d47fb1ee /var/lib/docker/aufs/mnt/982a0d62eab102811a5a697a3df2de46fad5c7acd34a9bb5bea44e98d47fb1ee-removing: device or resource busy

Afterwards if you try docker-compose rm you will get

docker-compose rm
Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 31, in main
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 21, in sys_dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 34, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 24, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 66, in perform_command
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 247, in rm
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.project", line 281, in containers
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.container", line 28, in from_ps
TypeError: coercing to Unicode: need string or buffer, NoneType found
batmat commented 9 years ago

FWIW, confirmed with 1.3.3, rolling back to 1.2.0 indeed fixes the issue.

aanand commented 9 years ago

@prologic Can you paste the JSON of a "Dead" container? I can't reproduce one locally. I'm also not sure what we should do if we can't determine a container's name.

ferdy-lw commented 9 years ago

I get this error (1.3.3) when doing compose run --rm run bash. I have a service called run. Changing the service name works (e.g. runa).

This had been working (I think 1.3.0 was the last version I had - it was 1.3.something) up until now that I have 1.3.3.

ErrorProne commented 9 years ago

I can confirm this:

docker-compose up -d
Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 32, in main
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 21, in sys_dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 34, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 24, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 66, in perform_command
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 471, in up
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.project", line 230, in up
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 398, in remove_duplicate_containers
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 405, in duplicate_containers
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 112, in containers
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.legacy", line 56, in check_for_legacy_containers
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.legacy", line 138, in get_legacy_containers
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.legacy", line 152, in _get_legacy_containers_iter
TypeError: argument of type 'NoneType' is not iterable

Docker Compose

docker-compose --version
docker-compose version: 1.3.2
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013

Docker

docker version
Client version: 1.7.1
Client API version: 1.19
Go version (client): go1.4.2
Git commit (client): 786b29d
OS/Arch (client): linux/amd64
Server version: 1.7.1
Server API version: 1.19
Go version (server): go1.4.2
Git commit (server): 786b29d
OS/Arch (server): linux/amd64

docker-compose.yml

wildfly:
  build: ./sp
  ports:
    - "8080:8080"
    - "9990:9990"
    - "39331:8787"
  links:
    - mysql:db
  volumes:
    - ./volumes/remote_deploy:/opt/jboss/wildfly/standalone/deployments
    - ./volumes/sp_properties:/opt/jboss/wildfly/standalone/sp_properties

mysql:
  image: mysql
  environment:
    MYSQL_USER: mysql
    MYSQL_PASSWORD: mysql
    MYSQL_DATABASE: test

A rollback to 1.2.0 fixed the issue.

pejvan commented 9 years ago

I think same issue here:

# docker-compose logs
Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 32, in main
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 21, in sys_dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 34, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 24, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 66, in perform_command
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 158, in logs
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.project", line 301, in containers
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.container", line 28, in from_ps
TypeError: coercing to Unicode: need string or buffer, NoneType found

# docker-compose --version
docker-compose version: 1.3.3
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
estin commented 9 years ago

when docker-compose up -d <service_name> in progress (linked service starting) - same issue

$ docker-compose ps
Traceback (most recent call last):
  File "/usr/bin/docker-compose", line 9, in <module>
    load_entry_point('docker-compose==1.3.3', 'console_scripts', 'docker-compose')()
  File "/usr/lib/python2.7/site-packages/compose/cli/main.py", line 32, in main
    command.sys_dispatch()
  File "/usr/lib/python2.7/site-packages/compose/cli/docopt_command.py", line 21, in sys_dispatch
    self.dispatch(sys.argv[1:], None)
  File "/usr/lib/python2.7/site-packages/compose/cli/command.py", line 34, in dispatch
    super(Command, self).dispatch(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/compose/cli/docopt_command.py", line 24, in dispatch
    self.perform_command(*self.parse(argv, global_options))
  File "/usr/lib/python2.7/site-packages/compose/cli/command.py", line 66, in perform_command
    handler(project, command_options)
  File "/usr/lib/python2.7/site-packages/compose/cli/main.py", line 195, in ps
    project.containers(service_names=options['SERVICE'], stopped=True) +
  File "/usr/lib/python2.7/site-packages/compose/project.py", line 301, in containers
    filters={'label': self.labels(one_off=one_off)})]
  File "/usr/lib/python2.7/site-packages/compose/container.py", line 28, in from_ps
    'Name': '/' + get_container_name(dictionary),
TypeError: coercing to Unicode: need string or buffer, NoneType found

Versions

$ sudo docker version                                                                              
Client version: 1.7.1
Client API version: 1.19
Go version (client): go1.4.2
Git commit (client): 786b29d
OS/Arch (client): linux/amd64
Server version: 1.7.1
Server API version: 1.19
Go version (server): go1.4.2
Git commit (server): 786b29d
OS/Arch (server): linux/amd64

sudo docker-compose --version                                                                    
docker-compose version: 1.3.3
CPython version: 2.7.10
OpenSSL version: OpenSSL 1.0.2d 9 Jul 2015
nigifabio commented 9 years ago

we are receiving the same error with latest pip release

fnigi]# docker-compose -v dockdocker-compose version: 1.4.0rc2 fnigi]# docker -v Docker version 1.6.0, build 4749651

the same with version 1.7.1 and 1.3.3 od docker-compose

the same configuration work on an upgraded python env

on Python 2.6.6 (r266:84292, Jan 22 2014, 09:42:36) seem broken

aanand commented 9 years ago

Anyone experiencing this issue: please paste the output of docker ps -a immediately after you get the error.

estin commented 9 years ago

in compose/container.py add line print('get_container_name - container:', container) # debug in get_container_name function:

def get_container_name(container):
    print('get_container_name - container:', container)   # debug
    if not container.get('Name') and not container.get('Names'):
        return None
    # inspect
    if 'Name' in container:
        return container['Name']
    # ps
    shortest_name = min(container['Names'], key=lambda n: len(n.split('/')))
    return shortest_name.split('/')[-1]

When

$ sudo docker-compose up -d <service-name>

Then Immediately call

$ sudo docker-compose ps

And result is

....
(u'get_container_name - container:', {u'Status': u'Removal In Progress', u'Created': 1438267340, u'Image': u'ib_frontend', u'Labels': {u'com.docker.compose.service': u'frontend', u'com.docker.compose.config-hash': u'e527b462027b756c37e4c631b21f143dac9f1d39c986ee23b4d0e9acef2f62b1', u'com.docker.compose.project': u'ib', u'com.docker.compose.version': u'1.3.3', u'com.docker.compose.oneoff': u'False', u'com.docker.compose.container-number': u'1'}, u'Ports': [], u'Command': u'gulp --debug', u'Names': None, u'Id': u'2b8755775154015d668ef4212d6470f07ad3b83dd01c24e57bd6c2997de69d28'})
Traceback (most recent call last):
  File "/usr/bin/docker-compose", line 9, in <module>
    load_entry_point('docker-compose==1.3.3', 'console_scripts', 'docker-compose')()
  File "/usr/lib/python2.7/site-packages/compose/cli/main.py", line 32, in main
    command.sys_dispatch()
  File "/usr/lib/python2.7/site-packages/compose/cli/docopt_command.py", line 21, in sys_dispatch
    self.dispatch(sys.argv[1:], None)
  File "/usr/lib/python2.7/site-packages/compose/cli/command.py", line 34, in dispatch
    super(Command, self).dispatch(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/compose/cli/docopt_command.py", line 24, in dispatch
    self.perform_command(*self.parse(argv, global_options))
  File "/usr/lib/python2.7/site-packages/compose/cli/command.py", line 66, in perform_command
    handler(project, command_options)
  File "/usr/lib/python2.7/site-packages/compose/cli/main.py", line 195, in ps
    project.containers(service_names=options['SERVICE'], stopped=True) +
  File "/usr/lib/python2.7/site-packages/compose/project.py", line 301, in containers
    filters={'label': self.labels(one_off=one_off)})]
  File "/usr/lib/python2.7/site-packages/compose/container.py", line 28, in from_ps
    'Name': '/' + get_container_name(dictionary),
TypeError: coercing to Unicode: need string or buffer, NoneType found

On this dictionary get_container_name return None and we have TypeError

{u'Command': u'gulp --debug',
 u'Created': 1438267340,
 u'Id': u'2b8755775154015d668ef4212d6470f07ad3b83dd01c24e57bd6c2997de69d28',
 u'Image': u'ib_frontend',
 u'Labels': {u'com.docker.compose.config-hash': u'e527b462027b756c37e4c631b21f143dac9f1d39c986ee23b4d0e9acef2f62b1',
             u'com.docker.compose.container-number': u'1',
             u'com.docker.compose.oneoff': u'False',
             u'com.docker.compose.project': u'ib',
             u'com.docker.compose.service': u'frontend',
             u'com.docker.compose.version': u'1.3.3'},
 u'Names': None,
 u'Ports': [],
 u'Status': u'Removal In Progress'}

Names are None

After started docker-complse ps works without any errors.

When immediately call of sudo docker ps

$ sudo docker ps                                                                             
.... waiting about 1-2 seconds and then produced output ...
CONTAINER ID        IMAGE               COMMAND                CREATED              STATUS              PORTS                                        NAMES
4628aaee3ed7        ib_backend          "/var/webapps/IB/con   About a minute ago   Up About a minute   8000/tcp                                     ib_backend_run_1     
dc80f79e28da        memcached           "memcached -m 64"      26 minutes ago       Up 26 minutes       11211/tcp                                    ib_memcached_1       
546f431ea39d        elasticsearch       "/var/webapps/IB/con   26 minutes ago       Up 26 minutes       9000/tcp, 9300/tcp, 0.0.0.0:9200->9200/tcp   ib_elasticsearch_1   
b4a5d058d1c0        ib_redis            "/usr/bin/redis-serv   26 minutes ago       Up 26 minutes       6379/tcp                                     ib_redis_1           
27e3541b7778        ib_postgres         "/var/webapps/IB/con   26 minutes ago       Up 26 minutes       5432/tcp

Sorry for my poor English.

estin commented 9 years ago

and another shot of docker-compose ps and docker ps when docker-compose up -d in progress

$ sudo docker-compose ps
...
(u'get_container_name - container:', {u'Status': u'Removal In Progress', u'Created': 1438280708, u'Image': u'elasticsearch', u'Labels': {u'com.docker.compose.service': u'elasticsearch', u'com.docker.compose.config-hash': u'e83fac7baf70ed5d259c9c73b36224d7df63fc0c979750388ad1205350a5503f', u'com.docker.compose.project': u'ib', u'com.docker.compose.version': u'1.3.3', u'com.docker.compose.oneoff': u'False', u'com.docker.compose.container-number': u'1'}, u'Ports': [], u'Command': u'/var/webapps/IB/containers/elasticsearch_entrypoint.sh elasticsearch -Des.config=/var/webapps/IB/containers/elasticsearch.yml', u'Names': None, u'Id': u'546f431ea39ddff8c3c3c8b798c17a93348013d4c5a17cf46696087533349ff6'})
Traceback (most recent call last):
  File "/usr/bin/docker-compose", line 9, in <module>
    load_entry_point('docker-compose==1.3.3', 'console_scripts', 'docker-compose')()
  File "/usr/lib/python2.7/site-packages/compose/cli/main.py", line 32, in main
    command.sys_dispatch()
  File "/usr/lib/python2.7/site-packages/compose/cli/docopt_command.py", line 21, in sys_dispatch
    self.dispatch(sys.argv[1:], None)
  File "/usr/lib/python2.7/site-packages/compose/cli/command.py", line 34, in dispatch
    super(Command, self).dispatch(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/compose/cli/docopt_command.py", line 24, in dispatch
    self.perform_command(*self.parse(argv, global_options))
  File "/usr/lib/python2.7/site-packages/compose/cli/command.py", line 66, in perform_command
    handler(project, command_options)
  File "/usr/lib/python2.7/site-packages/compose/cli/main.py", line 195, in ps
    project.containers(service_names=options['SERVICE'], stopped=True) +
  File "/usr/lib/python2.7/site-packages/compose/project.py", line 301, in containers
    filters={'label': self.labels(one_off=one_off)})]
  File "/usr/lib/python2.7/site-packages/compose/container.py", line 28, in from_ps
    'Name': '/' + get_container_name(dictionary),
TypeError: coercing to Unicode: need string or buffer, NoneType found

$ sudo docker ps                                                                              
CONTAINER ID        IMAGE               COMMAND                CREATED             STATUS              PORTS                                        NAMES
e8a2ea48e3b0        elasticsearch       "/var/webapps/IB/con   4 seconds ago       Up 1 seconds        9000/tcp, 9300/tcp, 0.0.0.0:9200->9200/tcp   ib_elasticsearch_1   
d07381a7f001        ib_redis            "/usr/bin/redis-serv   19 seconds ago      Up 14 seconds       6379/tcp                                     ib_redis_1           
e07808b3640e        ib_postgres         "/var/webapps/IB/con   30 seconds ago      Up 27 seconds       5432/tcp                                     ib_postgres_1        
dc80f79e28da        memcached           "memcached -m 64"      33 minutes ago      Up 33 minutes       11211/tcp                                    ib_memcached_1
aanand commented 9 years ago

Sorry, I need the output of docker ps -a - updated the comment.

aanand commented 9 years ago

In your first paste, it looks like you've got a container without a name - the one running the ib_postgres image.

It looks like the /containers/json API endpoint can sometimes return a container JSON blob without a name. That definitely sounds like a Docker bug to me, so let's figure out how to reproduce it. It could be related to renaming containers.

Anyone who's experiencing this, please:

  1. Clear everything out with docker-compose kill && docker-compose rm -f
  2. Use docker-compose --verbose up -d to start/recreate your containers
  3. When you encounter the error, paste the output of the last docker-compose --verbose up -d command you ran
  4. Also paste the output of docker ps -a
estin commented 9 years ago
  1. Clear everything out with docker-compose kill && docker-compose rm -f

@aanand, this issue correspond to container state Removal in Progress and when docker-compose (up|rm|kill) will be recreate/remove some containers for linking it to another.

this patch worked for me:

diff --git a/compose/cli/docker_client.py b/compose/cli/docker_client.py
index 244bcbe..56cf4f8 100644
--- a/compose/cli/docker_client.py
+++ b/compose/cli/docker_client.py
@@ -4,6 +4,15 @@ import ssl
 import os

+# issue-1593 compose not interesting in 'Removal In Progress' containers
+class ComposeDockerClient(Client):
+    def containers(self, *args, **kwargs):
+        return list(filter(
+            lambda i: i['Status'] != 'Removal In Progress',
+            super(ComposeDockerClient, self).containers(*args, **kwargs),
+        ))
+
+
 def docker_client():
     """
     Returns a docker-py client configured using environment variables
@@ -34,4 +43,6 @@ def docker_client():
         )

     timeout = int(os.environ.get('DOCKER_CLIENT_TIMEOUT', 60))
-    return Client(base_url=base_url, tls=tls_config, version=api_version, timeout=timeout)
+    return ComposeDockerClient(
+        base_url=base_url, tls=tls_config, version=api_version, timeout=timeout
+    )

Sorry for my poor English.

PS. may be docker-py must have new filter params to exclude Removal in Progress containers

AndyBerman commented 9 years ago 
Compose version 1.3.2
Docker base_url: http+docker://localunixsocket
Docker version: KernelVersion=2.6.32-504.16.2.el6.x86_64, Arch=amd64, ApiVersion=1.18, Version=1.6.2, GitCommit=7c8fca2, Os=linux, GoVersion=go1.4.2
docker containers <- (all=True, filters={u'label': [u'com.docker.compose.project=dockerregistry', u'com.docker.compose.service=DockerRegistry', u'com.docker.compose.oneoff=False']})
docker containers -> (list with 0 items)
docker containers <- (all=True)
docker containers -> (list with 0 items)
docker containers <- (all=True, filters={u'label': [u'com.docker.compose.project=dockerregistry', u'com.docker.compose.service=DockerRegistryUI', u'com.docker.compose.oneoff=False']})
docker containers -> (list with 0 items)
docker containers <- (all=True)
docker containers -> (list with 0 items)
docker containers <- (all=True, filters={u'label': [u'com.docker.compose.project=dockerregistry', u'com.docker.compose.service=DockerRegistry', u'com.docker.compose.oneoff=False']})
docker containers -> (list with 0 items)
docker containers <- (all=True)
docker containers -> (list with 0 items)
docker containers <- (all=True, filters={u'label': [u'com.docker.compose.project=dockerregistry', u'com.docker.compose.service=DockerRegistryUI', u'com.docker.compose.oneoff=False']})
docker containers -> (list with 0 items)
docker containers <- (all=True)
docker containers -> (list with 0 items)
docker inspect_image <- ('docker-registry.mitre.org:8443/autobuild/docker_registry:9')
docker inspect_image -> {u'Architecture': u'amd64',
u'Author': u'Venkat Natarajan  <venkat@mitre.org>',
u'Comment': u'',
u'Config': {u'AttachStderr': False,
             u'AttachStdin': False,
             u'AttachStdout': False,
             u'Cmd': None,
             u'CpuShares': 0,
             u'Cpuset': u'',
             u'Domainname': u'',
...
docker containers <- (all=True, filters={u'label': [u'com.docker.compose.project=dockerregistry', u'com.docker.compose.service=DockerRegistry', u'com.docker.compose.oneoff=False']})
docker containers -> (list with 0 items)
docker inspect_image <- ('docker-registry.mitre.org:8443/autobuild/docker_registry:9')
docker inspect_image -> {u'Architecture': u'amd64',
u'Author': u'Venkat Natarajan  <venkat@mitre.org>',
u'Comment': u'',
u'Config': {u'AttachStderr': False,
             u'AttachStdin': False,
             u'AttachStdout': False,
             u'Cmd': None,
             u'CpuShares': 0,
             u'Cpuset': u'',
             u'Domainname': u'',
...
Creating dockerregistry_DockerRegistry_1...
docker create_container <- (name=u'dockerregistry_DockerRegistry_1', image='docker-registry.mitre.org:8443/autobuild/docker_registry:9', labels={u'com.docker.compose.service': u'DockerRegistry', 'com.docker.compose.config-hash': 'f5d00447e880c7fa0ccc47dde09aad90fb7136e944c2769897980ea46a1af4ed', u'com.docker.compose.project': u'dockerregistry', 'com.docker.compose.version': u'1.3.2', 'mitre.release': '@@RELEASE@@', u'com.docker.compose.oneoff': u'False', 'com.docker.compose.container-number': '1', 'mitre.environment': '@@ENVIRONMENT@@'}, host_config={'Links': [], 'PortBindings': {'443/tcp': [{'HostPort': '8443', 'HostIp': ''}], '5000/tcp': [{'HostPort': '5000', 'HostIp': u'127.0.0.1'}]}, 'Binds': [u'/data/docker_registry/config:/data/docker_registry/config:ro', u'/etc/pki/tls/private/r10a-venkat-docker.key:/etc/ssl/private/ssl-cert-snakeoil.key:rw', u'/data/docker_registry/includes/authnz_ldap.conf:/etc/apache2/conf-available/authnz_ldap.conf:ro', u'/data/docker_registry:/docker_registry:rw', u'/logs/docker_registry:/var/log/apache2:rw', u'/etc/pki/tls/certs/r10a-venkat-docker.crt:/etc/ssl/certs/ssl-cert-snakeoil.pem:rw'], 'RestartPolicy': {u'MaximumRetryCount': 0, u'Name': u'always'}, 'ExtraHosts': [], 'LogConfig': {'Type': u'json-file', 'Config': {}}, 'VolumesFrom': []}, environment={'SETTINGS_FLAVOR': 'dev', 'SEARCH_BACKEND': 'sqlalchemy', 'SQLALCHEMY_INDEX_DATABASE': 'sqlite:////tmp/docker-registry.db', 'DOCKER_REGISTRY_CONFIG': '/data/docker_registry/config/config.yml'}, volumes={u'/data/docker_registry/config': {}, u'/etc/ssl/private/ssl-cert-snakeoil.key': {}, u'/etc/apache2/conf-available/authnz_ldap.conf': {}, u'/docker_registry': {}, u'/var/log/apache2': {}, u'/etc/ssl/certs/ssl-cert-snakeoil.pem': {}}, detach=True, ports=[u'443', u'5000'])
docker create_container -> {u'Id': u'd95374a1ead2b98f089c4d5e0e3a6a42fb0702ef267f2edde0b380a81c592c42',
u'Warnings': None}
docker inspect_container <- (u'd95374a1ead2b98f089c4d5e0e3a6a42fb0702ef267f2edde0b380a81c592c42')
docker inspect_container -> {u'AppArmorProfile': u'',
u'Args': [u'-c', u'/etc/supervisord.conf'],
u'Config': {u'AttachStderr': False,
             u'AttachStdin': False,
             u'AttachStdout': False,
             u'Cmd': None,
             u'CpuShares': 0,
             u'Cpuset': u'',
             u'Domainname': u'',
             u'Entrypoint': [u'/usr/bin/supervisord',
...
docker start <- (u'd95374a1ead2b98f089c4d5e0e3a6a42fb0702ef267f2edde0b380a81c592c42')
docker start -> None
docker inspect_image <- ('docker-registry.mitre.org:8443/autobuild/docker_registry_ui:9')
docker inspect_image -> {u'Architecture': u'amd64',
u'Author': u'Venkatesan Natarajan  <venkat@mitre.org>',
u'Comment': u'',
u'Config': {u'AttachStderr': False,
             u'AttachStdin': False,
             u'AttachStdout': False,
             u'Cmd': [u'/bin/sh', u'-c', u'$START_SCRIPT'],
             u'CpuShares': 0,
             u'Cpuset': u'',
             u'Domainname': u'',
...
docker containers <- (all=True, filters={u'label': [u'com.docker.compose.project=dockerregistry', u'com.docker.compose.service=DockerRegistryUI', u'com.docker.compose.oneoff=False']})
docker containers -> (list with 0 items)
docker inspect_image <- ('docker-registry.mitre.org:8443/autobuild/docker_registry_ui:9')
docker inspect_image -> {u'Architecture': u'amd64',
u'Author': u'Venkatesan Natarajan  <venkat@mitre.org>',
u'Comment': u'',
u'Config': {u'AttachStderr': False,
             u'AttachStdin': False,
             u'AttachStdout': False,
             u'Cmd': [u'/bin/sh', u'-c', u'$START_SCRIPT'],
             u'CpuShares': 0,
             u'Cpuset': u'',
             u'Domainname': u'',
...
docker containers <- (all=False, filters={u'label': [u'com.docker.compose.project=dockerregistry', u'com.docker.compose.service=DockerRegistry', u'com.docker.compose.oneoff=False']})
docker containers -> (list with 1 items)
docker inspect_container <- (u'd95374a1ead2b98f089c4d5e0e3a6a42fb0702ef267f2edde0b380a81c592c42')
docker inspect_container -> {u'AppArmorProfile': u'',
u'Args': [u'-c', u'/etc/supervisord.conf'],
u'Config': {u'AttachStderr': False,
             u'AttachStdin': False,
             u'AttachStdout': False,
             u'Cmd': None,
             u'CpuShares': 0,
             u'Cpuset': u'',
             u'Domainname': u'',
             u'Entrypoint': [u'/usr/bin/supervisord',
...
Creating dockerregistry_DockerRegistryUI_1...
docker create_container <- (name=u'dockerregistry_DockerRegistryUI_1', image='docker-registry.mitre.org:8443/autobuild/docker_registry_ui:9', labels={u'com.docker.compose.service': u'DockerRegistryUI', 'com.docker.compose.config-hash': '8949aef8e893f2bd04b031dd96801466d7dc22ed2923920769e7201c2c939ca8', u'com.docker.compose.project': u'dockerregistry', 'com.docker.compose.version': u'1.3.2', 'mitre.release': '@@RELEASE@@', u'com.docker.compose.oneoff': u'False', 'com.docker.compose.container-number': '1', 'mitre.environment': '@@ENVIRONMENT@@'}, host_config={'Links': ['dockerregistry_DockerRegistry_1:DockerRegistry_1', 'dockerregistry_DockerRegistry_1:docker-registry.mitre.org', 'dockerregistry_DockerRegistry_1:dockerregistry_DockerRegistry_1'], 'PortBindings': {'443/tcp': [{'HostPort': '444', 'HostIp': ''}]}, 'Binds': [u'/etc/pki/tls/private/r10a-venkat-docker.key:/etc/apache2/docker.key:ro', u'/data/docker_registry_ui/includes/authnz_ldap.conf:/etc/apache2/conf-enabled/authnz_ldap.conf:rw', u'/logs/docker_registry_ui:/var/log/apache2:rw', u'/etc/pki/tls/certs/r10a-venkat-docker.crt:/etc/apache2/docker.crt:ro'], 'RestartPolicy': {u'MaximumRetryCount': 0, u'Name': u'always'}, 'ExtraHosts': [], 'LogConfig': {'Type': u'json-file', 'Config': {}}, 'VolumesFrom': []}, environment={'ENV_DOCKER_REGISTRY_PORT': '8443', 'ENV_DOCKER_REGISTRY_HOST': 'docker-registry.mitre.org', 'ENV_USE_SSL': 'yes'}, volumes={u'/etc/apache2/docker.key': {}, u'/etc/apache2/conf-enabled/authnz_ldap.conf': {}, u'/var/log/apache2': {}, u'/etc/apache2/docker.crt': {}}, detach=True, ports=[u'443'])
docker create_container -> {u'Id': u'03373d7eaba13df0c686aa3097ca1602c492d2a0800a5ea97330024464b47c1e',
u'Warnings': None}
docker inspect_container <- (u'03373d7eaba13df0c686aa3097ca1602c492d2a0800a5ea97330024464b47c1e')
docker inspect_container -> {u'AppArmorProfile': u'',
u'Args': [u'-c', u'$START_SCRIPT'],
u'Config': {u'AttachStderr': False,
             u'AttachStdin': False,
             u'AttachStdout': False,
             u'Cmd': [u'/bin/sh', u'-c', u'$START_SCRIPT'],
             u'CpuShares': 0,
             u'Cpuset': u'',
             u'Domainname': u'',
             u'Entrypoint': None,
...
docker start <- (u'03373d7eaba13df0c686aa3097ca1602c492d2a0800a5ea97330024464b47c1e')
docker start -> None
docker containers <- (all=False, filters={u'label': [u'com.docker.compose.project=dockerregistry', u'com.docker.compose.service=DockerRegistry', u'com.docker.compose.oneoff=False']})
docker containers -> (list with 1 items)
docker containers <- (all=False, filters={u'label': [u'com.docker.compose.project=dockerregistry', u'com.docker.compose.service=DockerRegistryUI', u'com.docker.compose.oneoff=False']})
docker containers -> (list with 1 items)

-----------------------------------------------------------

Compose version 1.3.2
Docker base_url: http+docker://localunixsocket
Docker version: KernelVersion=2.6.32-504.16.2.el6.x86_64, Arch=amd64, ApiVersion=1.18, Version=1.6.2, GitCommit=7c8fca2, Os=linux, GoVersion                                              =go1.4.2
docker containers <- (all=True, filters={u'label': [u'com.docker.compose.project=dockerregistry', u'com.docker.compose.service=DockerRegistr                                              y', u'com.docker.compose.oneoff=False']})
docker containers -> (list with 1 items)
Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 32, in main
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 21, in sys_dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 34, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 24, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 66, in perform_command
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 471, in up
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.project", line 230, in up
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 398, in remove_duplicate_containers
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 405, in duplicate_containers
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 106, in containers
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.container", line 28, in from_ps
TypeError: coercing to Unicode: need string or buffer, NoneType found
[venkat@r10a-venkat-docker bin]$ docker ps -a
CONTAINER ID        IMAGE                                                                       COMMAND                CREATED             STATUS                       PORTS               NAMES
ef9075284e6c        docker-registry.mitre.org:8443/autobuild/mitre_open_docker_registry_ui:18   "/bin/sh -c $START_S   3 minutes ago       Dead                                           
03373d7eaba1        docker-registry.mitre.org:8443/autobuild/docker_registry_ui:9               "/bin/sh -c $START_S   3 minutes ago       Exited (137) 2 minutes ago                       dockerregistry_DockerRegistryUI_1
08ab0eaf1b69        docker-registry.mitre.org:8443/autobuild/mitre_open_docker_registry_ui:18   "/bin/sh -c $START_S   22 minutes ago      Dead                                             loving_meitner
9dcfb52a9a1a        docker-registry.mitre.org:8443/autobuild/mitre_open_docker_registry_ui:18   "/bin/sh -c $START_S   34 minutes ago      Dead                                             angry_fermi
471f00dccbe2        docker-registry.mitre.org:8443/autobuild/docker_registry_ui:9               "/bin/sh -c $START_S   35 minutes ago      Dead                                             loving_hodgkin
fe121f0792a4        docker-registry.mitre.org:8443/autobuild/docker_registry:9                  "/usr/bin/supervisor   35 minutes ago      Dead                                           
8d713fdd23ea        docker-registry.mitre.org:8443/autobuild/mitre_open_docker_registry_ui:18   "/bin/sh -c $START_S   2 weeks ago         Dead                                             angry_darwin
f2f298d71e6f        docker-registry.mitre.org:8443/autobuild/docker_registry_ui:9               "/bin/sh -c $START_S   2 weeks ago         Dead                                             adoring_davinci
aanand commented 9 years ago

I've implemented a fix here: https://github.com/aanand/fig/compare/bump-1.4.0...aanand:fix-name-none

Please try it out if you've been experiencing this issue:

http://cl.ly/0G3c3y0y1k19/docker-compose-Darwin-x86_64 http://cl.ly/0r3J020O2s2C/docker-compose-Linux-x86_64

estin commented 9 years ago

Works for me

call ps without fix

$ docker-compose ps                                                                                      
Traceback (most recent call last):
  File "/usr/bin/docker-compose", line 9, in <module>
    load_entry_point('docker-compose==1.3.3', 'console_scripts', 'docker-compose')()
  File "/usr/lib/python2.7/site-packages/compose/cli/main.py", line 32, in main
    command.sys_dispatch()
  File "/usr/lib/python2.7/site-packages/compose/cli/docopt_command.py", line 21, in sys_dispatch
    self.dispatch(sys.argv[1:], None)
  File "/usr/lib/python2.7/site-packages/compose/cli/command.py", line 34, in dispatch
    super(Command, self).dispatch(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/compose/cli/docopt_command.py", line 24, in dispatch
    self.perform_command(*self.parse(argv, global_options))
  File "/usr/lib/python2.7/site-packages/compose/cli/command.py", line 66, in perform_command
    handler(project, command_options)
  File "/usr/lib/python2.7/site-packages/compose/cli/main.py", line 195, in ps
    project.containers(service_names=options['SERVICE'], stopped=True) +
  File "/usr/lib/python2.7/site-packages/compose/project.py", line 301, in containers
    filters={'label': self.labels(one_off=one_off)})]
  File "/usr/lib/python2.7/site-packages/compose/container.py", line 28, in from_ps
    'Name': '/' + get_container_name(dictionary),
TypeError: coercing to Unicode: need string or buffer, NoneType found

and call ps with fix

$ /tmp/docker-compose-Linux-x86_64 ps                                                                   
        Name                       Command               State                      Ports                    
------------------------------------------------------------------------------------------------------------
64ffb1d005_ib_redis_1   /usr/bin/redis-server            Exit 0                                              
ib_backend_1            /var/webapps/IB/containers ...   Up       0.0.0.0:8000->8000/tcp                     
ib_celery_1             /var/webapps/IB/containers ...   Up                                                  
ib_elasticsearch_1      /var/webapps/IB/containers ...   Up       9000/tcp, 0.0.0.0:9200->9200/tcp, 9300/tcp 
ib_frontend_1           gulp --debug                     Up       0.0.0.0:5000->5000/tcp                     
ib_memcached_1          memcached -m 64                  Up       11211/tcp                                  
ib_postgres_1           /var/webapps/IB/containers ...   Up       5432/tcp                                   
ib_redis_1              /usr/bin/redis-server            Up       6379/tcp                                   
ib_ws_1                 /var/webapps/IB/containers ...   Up       0.0.0.0:8080->8080/tcp

When do double call up -d without fix 

$ docker-compose up -d
Traceback (most recent call last):
  File "/usr/bin/docker-compose", line 9, in <module>
    load_entry_point('docker-compose==1.3.3', 'console_scripts', 'docker-compose')()
  File "/usr/lib/python2.7/site-packages/compose/cli/main.py", line 32, in main
    command.sys_dispatch()
  File "/usr/lib/python2.7/site-packages/compose/cli/docopt_command.py", line 21, in sys_dispatch
    self.dispatch(sys.argv[1:], None)
  File "/usr/lib/python2.7/site-packages/compose/cli/command.py", line 34, in dispatch
    super(Command, self).dispatch(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/compose/cli/docopt_command.py", line 24, in dispatch
    self.perform_command(*self.parse(argv, global_options))
  File "/usr/lib/python2.7/site-packages/compose/cli/command.py", line 66, in perform_command
    handler(project, command_options)
  File "/usr/lib/python2.7/site-packages/compose/cli/main.py", line 471, in up
    timeout=timeout
  File "/usr/lib/python2.7/site-packages/compose/project.py", line 230, in up
    service.remove_duplicate_containers()
  File "/usr/lib/python2.7/site-packages/compose/service.py", line 398, in remove_duplicate_containers
    for c in self.duplicate_containers():
  File "/usr/lib/python2.7/site-packages/compose/service.py", line 405, in duplicate_containers
    self.containers(stopped=True),
  File "/usr/lib/python2.7/site-packages/compose/service.py", line 106, in containers
    filters={'label': self.labels(one_off=one_off)})]
  File "/usr/lib/python2.7/site-packages/compose/container.py", line 28, in from_ps
    'Name': '/' + get_container_name(dictionary),
TypeError: coercing to Unicode: need string or buffer, NoneType found

When do double call up -d with fix

$ /tmp/docker-compose-Linux-x86_64 up -d                                                               
Removing ib_elasticsearch_1...
Removing ib_memcached_1...
Cannot destroy container 7cce897e11ad63816286bcf70cdf742f28d006e28d5b6831fd49922a5c690db7: Driver devicemapper failed to remove init filesystem 7cce897e11ad63816286bcf70cdf742f28d006e28d5b6831fd49922a5c690db7-init: Device is Busy

Where double call is call docker-compose up -d at second time util first up -d is not done

robertoandrade commented 9 years ago

even with the fix, when running a docker-compose run having the service being up already, returns a weird message:

Could not get container for
robertoandrade commented 9 years ago

I also noticed that with the patch, my containers no longer carry the name that docker-compose lists on the logs when uping it. So when I do docker-compose ps and docker ps I see the container but the name doesn't show on the second command.

May be something with docker itself? I'm running the latest 1.4.1 there

aanand commented 9 years ago

@robertoandrade Do you mean 1.7.1? We don't support 1.4.x any more.