jina-ai / serve

☁️ Build multimodal AI applications with cloud-native stack
https://jina.ai/serve
Apache License 2.0
21.13k stars 2.22k forks source link

Docker-compose do not Found no NVIDIA driver #4696

Closed jerrychen1990 closed 2 years ago

jerrychen1990 commented 2 years ago

Describe the bug I build a flow with two executors from jina hub. When I start the server with

with f:
    f.block()

it works well, and docker container can use gpu. If I dump the flow with to_docker_compose_yaml and start the server with docker-compose command the executor container can't use gpu and throw a exception like this: image

the flow is built with:

    f = (
        Flow(port=port, protocol='http')
            .add(name='encoder', uses='jinahub+docker://CLIPImageEncoder/latest-gpu',
                 uses_with={'device': "cuda"}, gpus="all", replicas=1,
                 volumes="/home/chenhao/.cache:/root/.cache")
            .add(
            name='indexer',
            uses='jinahub+docker://SimpleIndexer',
            workspace = "/root/indexer",
            uses_with={'table_name': 'simple_indexer_table'}
        )
    )

the generated docker-compose yaml is:


version: '3.3'
networks:
  jina-network:
    driver: bridge
services:
  encoder:
    image: jinahub/0hnlmu3q:62679f5aed6b00a3c0539a71
    entrypoint:
    - jina
    command:
    - executor
    - --name
    - encoder
    - --uses
    - config.yml
    - --port
    - '8081'
    - --gpus
    - all
    - --uses-metas
    - '{}'
    - --uses-with
    - '{"device": "cuda"}'
    - --native
    healthcheck:
      test: python -m jina.resources.health_check.pod localhost:8081
      interval: 2s
    environment:
    - JINA_LOG_LEVEL=INFO
    volumes:
    - /home/chenhao/.cache:/root/.cache
    networks:
    - jina-network
  indexer:
    image: jinahub/zb38xlt4:62679f821f235059f408243c
    entrypoint:
    - jina
    command:
    - executor
    - --name
    - indexer
    - --workspace
    - /root/indexer
    - --uses
    - config.yml
    - --port
    - '8081'
    - --uses-metas
    - '{}'
    - --uses-with
    - '{"table_name": "simple_indexer_table"}'
    - --native
    healthcheck:
      test: python -m jina.resources.health_check.pod localhost:8081
      interval: 2s
    environment:
    - JINA_LOG_LEVEL=INFO
    volumes:
    - /Users/chenhao/.jina/executor-workspace/672c66f7ca4745508b80ea329a679074:/app
    networks:
    - jina-network
  gateway:
    image: jinaai/jina:3.3.10-py38-standard
    entrypoint:
    - jina
    command:
    - gateway
    - --polling
    - ANY
    - --port
    - '12345'
    - --expose-endpoints
    - '{}'
    - --protocol
    - HTTP
    - --graph-description
    - '{"encoder": ["indexer"], "start-gateway": ["encoder"], "indexer": ["end-gateway"]}'
    - --deployments-addresses
    - '{"encoder": ["encoder:8081"], "indexer": ["indexer:8081"]}'
    - --pod-role
    - GATEWAY
    expose:
    - '12345'
    ports:
    - 12345:12345
    healthcheck:
      test: python -m jina.resources.health_check.gateway localhost:12345 http
      interval: 2s
    environment:
    - JINA_LOG_LEVEL=INFO
    networks:
    - jina-network

Environment


- jina 3.3.10
- docarray 0.12.2
- jina-proto 0.1.8
- jina-vcs-tag (unset)
- protobuf 3.20.0
- proto-backend cpp
- grpcio 1.43.0
- pyyaml 6.0
- python 3.10.2
- platform Linux
- platform-release 4.4.0-186-generic
- platform-version #216-Ubuntu SMP Wed Jul 1 05:34:05 UTC 2020
- architecture x86_64
- processor x86_64
- uid 48710637999860
- session-id 0418e664-c549-11ec-858d-2c4d544656f4
- uptime 2022-04-26T18:09:53.184838
- ci-vendor (unset)
* JINA_ARRAY_QUANT (unset)
* JINA_CONTROL_PORT (unset)
* JINA_DEFAULT_HOST (unset)
* JINA_DEFAULT_TIMEOUT_CTRL (unset)
* JINA_DISABLE_UVLOOP (unset)
* JINA_FULL_CLI (unset)
* JINA_HUBBLE_REGISTRY (unset)
* JINA_HUB_CACHE_DIR (unset)
* JINA_HUB_ROOT (unset)
* JINA_GATEWAY_IMAGE (unset)
* JINA_LOG_CONFIG (unset)
* JINA_LOG_LEVEL (unset)
* JINA_LOG_NO_COLOR (unset)
* JINA_LOG_WORKSPACE (unset)
* JINA_MP_START_METHOD (unset)
* JINA_OPTIMIZER_TRIAL_WORKSPACE(unset)
* JINA_DEPLOYMENT_NAME (unset)
* JINA_RANDOM_PORT_MAX (unset)
* JINA_RANDOM_PORT_MIN (unset)
* JINA_VCS_VERSION (unset)

Screenshots

JoanFM commented 2 years ago

While we take alook at the issue, u may want to edit the file urself to add this

https://docs.docker.com/compose/gpu-support/#enabling-gpu-access-to-service-containers