docker-compose up fails with 'expected string or buffer' when specifying ports

[agladysh]

$ cat docker-compose.yml
version: "3"
services:
  bug:
    image: hello-world
    ports:
      - "8000:8000"

$ docker-compose up
ERROR: for bug  expected string or buffer
Traceback (most recent call last):
  File "/usr/local/bin/docker-compose", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 68, in main
    command()
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 118, in perform_command
    handler(command, command_options)
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 926, in up
    scale_override=parse_scale_args(options['--scale']),
  File "/usr/local/lib/python2.7/dist-packages/compose/project.py", line 424, in up
    get_deps
  File "/usr/local/lib/python2.7/dist-packages/compose/parallel.py", line 69, in parallel_execute
    raise error_to_reraise
TypeError: expected string or buffer

$ docker-compose --version
docker-compose version 1.14.0, build c7bdf9e

$ docker --version
Docker version 17.05.0-ce, build 89658be

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 17.04
Release:    17.04
Codename:   zesty

[agladysh]

NB: long ports syntax yields the same error message with a different stack:

version: "3.2"
services:
  bug:
    image: hello-world
    ports:
      - target: 8000
        published: 8000
        mode: host
        protocol: tcp

$ docker-compose up
Traceback (most recent call last):
  File "/usr/local/bin/docker-compose", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 68, in main
    command()
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 115, in perform_command
    project = project_from_options('.', options)
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/command.py", line 37, in project_from_options
    override_dir=options.get('--project-directory'),
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/command.py", line 91, in get_project
    config_data = config.load(config_details)
  File "/usr/local/lib/python2.7/dist-packages/compose/config/config.py", line 362, in load
    for config_file in config_details.config_files
  File "/usr/local/lib/python2.7/dist-packages/compose/config/config.py", line 528, in process_config_file
    validate_against_config_schema(config_file)
  File "/usr/local/lib/python2.7/dist-packages/compose/config/validation.py", line 378, in validate_against_config_schema
    config_file.filename)
  File "/usr/local/lib/python2.7/dist-packages/compose/config/validation.py", line 435, in handle_errors
    errors = list(sorted(errors, key=str))
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 105, in iter_errors
    for error in errors:
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/_validators.py", line 304, in properties_draft4
    schema_path=property,
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 121, in descend
    for error in self.iter_errors(instance, schema):
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 105, in iter_errors
    for error in errors:
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/_validators.py", line 16, in patternProperties
    v, subschema, path=k, schema_path=pattern,
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 121, in descend
    for error in self.iter_errors(instance, schema):
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 105, in iter_errors
    for error in errors:
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/_validators.py", line 216, in ref
    for error in validator.descend(instance, resolved):
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 121, in descend
    for error in self.iter_errors(instance, schema):
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 105, in iter_errors
    for error in errors:
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/_validators.py", line 304, in properties_draft4
    schema_path=property,
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 121, in descend
    for error in self.iter_errors(instance, schema):
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 105, in iter_errors
    for error in errors:
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/_validators.py", line 55, in items
    for error in validator.descend(item, items, path=index):
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 121, in descend
    for error in self.iter_errors(instance, schema):
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 105, in iter_errors
    for error in errors:
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/_validators.py", line 341, in oneOf_draft4
    errs = list(validator.descend(instance, subschema, schema_path=index))
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 121, in descend
    for error in self.iter_errors(instance, schema):
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 105, in iter_errors
    for error in errors:
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/_validators.py", line 163, in format
    validator.format_checker.check(instance, format)
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/_format.py", line 96, in check
    result = func(instance)
  File "/usr/local/lib/python2.7/dist-packages/compose/config/validation.py", line 51, in format_ports
    split_port(instance)
  File "/home/agladysh/.local/lib/python2.7/site-packages/docker/utils/ports.py", line 57, in split_port
    match = PORT_SPEC.match(port)
TypeError: expected string or buffer

[agladysh]

NB: Verbose output (for "8000:8000"):

$ docker-compose --verbose up
compose.config.config.find: Using configuration files: ./docker-compose.yml
docker.auth.find_config_file: Trying paths: ['/home/agladysh/.docker/config.json', '/home/agladysh/.dockercfg']
docker.auth.find_config_file: No config file found
compose.cli.command.get_client: docker-compose version 1.14.0, build c7bdf9e
docker-py version: 2.4.0
CPython version: 2.7.13
OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
compose.cli.command.get_client: Docker base_url: http+docker://localunixsocket
compose.cli.command.get_client: Docker version: KernelVersion=4.10.0-24-generic, Arch=amd64, BuildTime=2017-05-04T22:27:42.481436991+00:00, ApiVersion=1.29, Version=17.05.0-ce, MinAPIVersion=1.12, GitCommit=89658be, Os=linux, GoVersion=go1.7.5
compose.cli.verbose_proxy.proxy_callable: docker info <- ()
compose.cli.verbose_proxy.proxy_callable: docker info -> {u'Architecture': u'x86_64',
 u'BridgeNfIp6tables': True,
 u'BridgeNfIptables': True,
 u'CPUSet': True,
 u'CPUShares': True,
 u'CgroupDriver': u'cgroupfs',
 u'ClusterAdvertise': u'',
 u'ClusterStore': u'',
 u'ContainerdCommit': {u'Expected': u'9048e5e50717ea4497b757314bad98ea3763c145',
                       u'ID': u'9048e5e50717ea4497b757314bad98ea3763c145'},
...
compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- (u'scale_default')
compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {u'Attachable': True,
 u'Containers': {},
 u'Created': u'2017-06-28T17:36:51.315937208+03:00',
 u'Driver': u'bridge',
 u'EnableIPv6': False,
 u'IPAM': {u'Config': [{u'Gateway': u'172.19.0.1',
                        u'Subnet': u'172.19.0.0/16'}],
           u'Driver': u'default',
           u'Options': None},
 u'Id': u'f0da86e837353be0987f32109b8dd81696adfef59add61767ea30e90769c939d',
...
compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={u'label': [u'com.docker.compose.project=scale', u'com.docker.compose.oneoff=False']})
compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items)
compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={u'label': [u'com.docker.compose.project=scale', u'com.docker.compose.service=bug', u'com.docker.compose.oneoff=False']})
compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items)
compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('hello-world')
compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {u'Architecture': u'amd64',
 u'Author': u'',
 u'Comment': u'',
 u'Config': {u'ArgsEscaped': True,
             u'AttachStderr': False,
             u'AttachStdin': False,
             u'AttachStdout': False,
             u'Cmd': [u'/hello'],
             u'Domainname': u'',
             u'Entrypoint': None,
...
compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={u'label': [u'com.docker.compose.project=scale', u'com.docker.compose.service=bug', u'com.docker.compose.oneoff=False']})
compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items)
compose.parallel.feed_queue: Pending: set([<Service: bug>])
compose.parallel.feed_queue: Starting producer thread for <Service: bug>
compose.parallel.parallel_execute_iter: Failed: <Service: bug>
compose.parallel.feed_queue: Pending: set([])

ERROR: for bug  expected string or buffer
Traceback (most recent call last):
  File "/usr/local/bin/docker-compose", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 68, in main
    command()
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 118, in perform_command
    handler(command, command_options)
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 926, in up
    scale_override=parse_scale_args(options['--scale']),
  File "/usr/local/lib/python2.7/dist-packages/compose/project.py", line 424, in up
    get_deps
  File "/usr/local/lib/python2.7/dist-packages/compose/parallel.py", line 69, in parallel_execute
    raise error_to_reraise
TypeError: expected string or buffer

[agladysh]

NB: docker-compose run --service-ports works:

$ docker-compose run --service-ports bug /hello

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://cloud.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/engine/userguide/

[agladysh]

NB: Same error for docker-compose 1.13.0. 1.12.0 seems to work.

sudo pip install docker-compose==1.12.0

[sono-bfio]

Same issue here via python3

ERROR: for consul  expected string or bytes-like object
Traceback (most recent call last):
  File "/usr/local/bin/docker-compose", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.5/dist-packages/compose/cli/main.py", line 68, in main
    command()
  File "/usr/local/lib/python3.5/dist-packages/compose/cli/main.py", line 118, in perform_command
    handler(command, command_options)
  File "/usr/local/lib/python3.5/dist-packages/compose/cli/main.py", line 926, in up
    scale_override=parse_scale_args(options['--scale']),
  File "/usr/local/lib/python3.5/dist-packages/compose/project.py", line 424, in up
    get_deps
  File "/usr/local/lib/python3.5/dist-packages/compose/parallel.py", line 69, in parallel_execute
    raise error_to_reraise
  File "/usr/local/lib/python3.5/dist-packages/compose/parallel.py", line 167, in producer
    result = func(obj)
  File "/usr/local/lib/python3.5/dist-packages/compose/project.py", line 410, in do
    rescale=rescale
  File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 460, in execute_convergence_plan
    self.show_scale_warnings(scale)
  File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 205, in show_scale_warnings
    if self.specifies_host_port() and desired_num > 1:
  File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 983, in specifies_host_port
    return any(has_host_port(binding) for binding in self.options.get('ports', []))
  File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 983, in <genexpr>
    return any(has_host_port(binding) for binding in self.options.get('ports', []))
  File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 963, in has_host_port
    _, external_bindings = split_port(binding)
  File "/usr/local/lib/python3.5/dist-packages/docker/utils/ports.py", line 57, in split_port
    match = PORT_SPEC.match(port)
TypeError: expected string or bytes-like object

Worked yesterday morning, started failing this morning, here is a consul snippet with the port defined (This produces the error above).

  consul:
    image: consul:0.8.3
    network_mode: "host"
    restart: always
    command: "agent -server -ui -bootstrap-expect=1 -client=0.0.0.0 -recursor=8.8.8.8 -recursor=8.8.4.4 -dns-port=53 
    environment:
      CONSUL_BIND_INTERFACE: eth0
      CONSUL_LOCAL_CONFIG: '{"dns_config":{"allow_stale":false} }'
      CONSUL_ALLOW_PRIVILEGED_PORTS: "true"
    volumes:
      - "/path/to/vol:/consul/data"
    ports:
      - "8300:8300/tcp"

If we remove the ports section consul starts.

Python 3 Modules

docker==2.4.0
docker-compose==1.14.0
docker-pycreds==0.2.1
dockerpty==0.4.1

Work around: docker was updated to 2.4.0 15 hours ago. I reverted back to docker 2.3.0 and it worked.

pip uninstall docker
pip uninstall docker-compose

pip install docker==2.3.0
pip install docker-compose==1.14.0

Works^^

[thesquelched]

Changing all of the ports in my docker-compose.yml to strings did not appear to fix anything, FYI.

[digitaldavenyc]

Can confirm the downgrade of docker and docker-compose resolves the problem. This issue caused all our docker deployments on circle to fail.

[rikvanderkemp]

My Two cents, I am having similar issues where it fails with TypeError: expected string or bytes-like object

It seems like it passes the function split_port twice, once with the "correct" values like 8080:80 and once with a ServicePort object.

[rikvanderkemp]

It seems the culprit on my system was python-docker 2.4, I downgraded to 2.3.0 and Docker / Docker-compose both work with the latest versions!

So if you have this issue try downgrading to python-docker 2.3 first

[inhumantsar]

Downgrading the docker python library worked for me as well:

pip install --force docker==2.3.0

[shin-]

Thanks for the reports everyone, I'm currently working on a 2.4.2 release of the Python SDK that should fix this particular issue. It should be available in the next few hours.

[shin-]

For anyone affected, please pip install -U docker==2.4.2. Let me know if there are any issues still.

[lforg37]

Hello After upgrading I still have the issue with the following docker-compose excerpt:

services:
    nginx:
        build:
            context: ./mdb_httpproxy
        networks:
            dmz:
        ports:
            - "80"
        volumes:
            - corel-10k:/data/corel_images/:ro

[shin-]

@lforg37 What's the output of pip freeze ?

[WhyNotHugo]

Created a virtualenv, and installed docker-compose with latest deps (my distro hasn't updated to 2.4.2 yet). Working now.

[lforg37]

The output of pip freeze is

cached-property==1.3.0
certifi==2017.4.17
chardet==3.0.4
colorama==0.3.9
docker==2.4.2
docker-compose==1.14.0
docker-pycreds==0.2.1
dockerpty==0.4.1
docopt==0.6.2
idna==2.5
jsonschema==2.6.0
PyYAML==3.12
requests==2.11.1
six==1.10.0
texttable==0.8.8
urllib3==1.21.1
websocket-client==0.44.0

[lforg37]

I've tried to install a new virtualenv and now it is working. Sorry for the false alert.

[hernandev]

@lforg37 I get this error using official arch linux package

[wtwde]

I get the same issue

[lukasojd]

+1 on clear debian with use pip

[nogo]

Issue still exists for version 1.14 and 1.13 Arch linux

[cnu]

Issue exists in Arch Linux for v1.14, but sudo pip install -U docker==2.4.2 fixes the problem.

[shin-]

Distros will need to update their system packages, there's not much I can do about that. As always, our advice is to install using pip inside a virtualenv to avoid these issues.

[nogo]

@shin- you are right, was distro related. everything work now after "python-docker" package upgrade

[hernandev]

Not seeing the problem after a new upgrade on arch, have been working for a couple of days now.

Just a follow up.

[kenji-getpowered]

Using hypriot

Version 1.5.0 | hypriotos-rpi-v1.5.0.img.zip Checksum | Docker 17.05.0-ce, kernel 4.4.50 | 29.06.2017

Downgrading

pip install docker==2.3.0

Worked for me

[FDiskas]

in my case I changed ports to integers :doughnut: and this fixed my issue

ports:
      - 8000:8000