Build or Run CVAT on Mac M1 with Rosetta

shubham99bisht commented 1 year ago

My actions before raising this issue

[Y] Read/searched the docs
[Y] Searched past issues

I'm using a Mac with M1 processor and unable to run the docker images built for AMD platform. After some reading, I came across a blog which mentioned that having Rosetta and latest version of docker will enable me to run images built for AMD on ARM.

Steps to Reproduce (for bugs)

Followed the installation steps to run docker compose up -d but the cvat/server image kept restarting with logs on the lines of "Tensorflow is not built for this platform"
Then tried building the image on laptop using docker compose -f docker-compose.yml -f docker-compose.dev.yml build and even docker build . but both result in errors while building stuck at the step where the wheel packages are being installed

Expected Behaviour

I expected the AMD image to run having Rosetta installed.

Your Environment

Docker version (Docker Client 24.0.5, Docker Desktop 4.22.0)
Are you using Docker Swarm or Kubernetes? No
Operating System and version: MacOS

shubham99bisht commented 1 year ago

Is there anyone who has successfully built and run CVAT on Apple Silicon? I'm also open to running it without Tensorflow or any prebuilt ML models. I just need to use this as an annotation tool and make some updates on the source code.

kaixi-wang commented 1 year ago

I've gotten it to build with docker compose by setting DOCKER_DEFAULT_PLATFORM=linux/amd64 and checking out a version CVAT version <= 2.4.5.

I tried using buildx to build after modifying the config files. Sometimes I got it to appear to build completely, but ran into issues when I tried to run it.

pebabion commented 1 year ago

@kaixi-wang is there such version 1.4.5? I'm trying to docker-compose up with that version but it's complaining cvat_server doesn't have that version.

kaixi-wang commented 1 year ago

Did you try prepending a v (v2.4.5)?

kaixi-wang commented 1 year ago

Sorry just noticed I typed 1 instead of 2

matthewchung74 commented 1 year ago

I'm having issues as well. i can build on my m1, but running does not work.

robmarkcole commented 1 year ago

Likewise it runs but the UI displays

In the logs:

redis.exceptions.ConnectionError: Error 111 connecting to cvat_redis:6379. Connection refused.

euan-cowie commented 11 months ago

I'm having the exact same issue as @robmarkcole

BeardedUnicorn commented 11 months ago

Likewise it runs but the UI displays

In the logs:
redis.exceptions.ConnectionError: Error 111 connecting to cvat_redis:6379. Connection refused.

@robmarkcole @euan-cowie Pull the latest changes in master. There was a commit a few days ago that reverted the keydb version from v6.3.4 to v6.3.2 which fixed the issue.

robmarkcole commented 11 months ago

@BeardedUnicorn pulled latest and docker compose up now:

and logs

traefik                        | {"ClientAddr":"192.168.65.1:35976","ClientHost":"192.168.65.1","ClientPort":"35976","ClientUsername":"-","DownstreamContentSize":145,"DownstreamStatus":500,"Duration":38100500,"OriginContentSize":145,"OriginDuration":37996916,"OriginStatus":500,"Overhead":103584,"RequestAddr":"localhost:8080","RequestContentSize":0,"RequestCount":17,"RequestHost":"localhost","RequestMethod":"GET","RequestPath":"/api/server/health/?format=json\u0026org=","RequestPort":"8080","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"RouterName":"cvat@docker","ServiceAddr":"172.18.0.14:8080","ServiceName":"cvat@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"172.18.0.14:8080","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2023-12-15T15:57:55.248264087Z","StartUTC":"2023-12-15T15:57:55.248264087Z","entryPointName":"web","level":"info","msg":"","time":"2023-12-15T15:57:55Z"}
cvat_server                    | 2023-12-15 15:57:58,432 DEBG 'uvicorn-1' stdout output:
cvat_server                    | INFO:     172.18.0.16:0 - "GET /api/auth/rules HTTP/1.0" 304 Not Modified
cvat_server                    | 
cvat_server                    | 2023-12-15 15:57:59,265 DEBG 'uvicorn-1' stderr output:
cvat_server                    | [2023-12-15 15:57:59,263] ERROR health-check: Unexpected Error!
cvat_server                    | Traceback (most recent call last):
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 698, in connect
cvat_server                    |     sock = self.retry.call_with_retry(
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/redis/retry.py", line 46, in call_with_retry
cvat_server                    |     return do()
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 699, in <lambda>
cvat_server                    |     lambda: self._connect(), lambda error: self.disconnect(error)
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 955, in _connect
cvat_server                    |     for res in socket.getaddrinfo(
cvat_server                    |   File "/usr/lib/python3.10/socket.py", line 955, in getaddrinfo
cvat_server                    |     for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
cvat_server                    | socket.gaierror: [Errno -2] Name or service not known
cvat_server                    | 
cvat_server                    | During handling of the above exception, another exception occurred:
cvat_server                    | 
cvat_server                    | Traceback (most recent call last):
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/health_check/backends.py", line 30, in run_check
cvat_server                    |     self.check_status()
cvat_server                    |   File "/home/django/cvat/apps/health/backends.py", line 44, in check_status
cvat_server                    |     cache.set("djangohealtcheck_test", "itworks")
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/django/core/cache/backends/redis.py", line 191, in set
cvat_server                    |     self._cache.set(key, value, self.get_backend_timeout(timeout))
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/django/core/cache/backends/redis.py", line 108, in set
cvat_server                    |     client.set(key, value, ex=timeout)
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/redis/commands/core.py", line 2302, in set
cvat_server                    |     return self.execute_command("SET", *pieces, **options)
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/redis/client.py", line 1255, in execute_command
cvat_server                    |     conn = self.connection or pool.get_connection(command_name, **options)
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 1442, in get_connection
cvat_server                    |     connection.connect()
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 704, in connect
cvat_server                    |     raise ConnectionError(self._error_message(e))
cvat_server                    | redis.exceptions.ConnectionError: Error -2 connecting to cvat_redis:6379. Name or service not known.
cvat_server                    | 
cvat_server                    | 2023-12-15 15:57:59,290 DEBG 'uvicorn-1' stderr output:
cvat_server                    | [2023-12-15 15:57:59,289] ERROR django.request: Internal Server Error: /api/server/health/
cvat_server                    | Traceback (most recent call last):
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 698, in connect
cvat_server                    |     sock = self.retry.call_with_retry(
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/redis/retry.py", line 46, in call_with_retry
cvat_server                    |     return do()
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 699, in <lambda>
cvat_server                    |     lambda: self._connect(), lambda error: self.disconnect(error)
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 955, in _connect
cvat_server                    |     for res in socket.getaddrinfo(
cvat_server                    |   File "/usr/lib/python3.10/socket.py", line 955, in getaddrinfo
cvat_server                    |     for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
cvat_server                    | socket.gaierror: [Errno -2] Name or service not known
cvat_server                    | 
cvat_server                    | During handling of the above exception, another exception occurred:
cvat_server                    | 
cvat_server                    | Traceback (most recent call last):
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/asgiref/sync.py", line 534, in thread_handler
cvat_server                    |     raise exc_info[1]
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 42, in inner
cvat_server                    |     response = await get_response(request)
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 253, in _get_response_async
cvat_server                    |     response = await wrapped_callback(
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/asgiref/sync.py", line 479, in __call__
cvat_server                    |     ret: _R = await loop.run_in_executor(
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/asgiref/current_thread_executor.py", line 40, in run
cvat_server                    |     result = self.fn(*self.args, **self.kwargs)
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/asgiref/sync.py", line 538, in thread_handler
cvat_server                    |     return func(*args, **kwargs)
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/django/views/generic/base.py", line 104, in view
cvat_server                    |     return self.dispatch(request, *args, **kwargs)
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/django/views/generic/base.py", line 143, in dispatch
cvat_server                    |     return handler(request, *args, **kwargs)
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/django/utils/decorators.py", line 46, in _wrapper
cvat_server                    |     return bound_method(*args, **kwargs)
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/django/views/decorators/cache.py", line 62, in _wrapper_view_func
cvat_server                    |     response = view_func(request, *args, **kwargs)
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/health_check/views.py", line 91, in get
cvat_server                    |     status_code = 500 if self.errors else 200
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/health_check/mixins.py", line 16, in errors
cvat_server                    |     self._errors = self.run_check()
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/health_check/mixins.py", line 44, in run_check
cvat_server                    |     for plugin in executor.map(_run, self.plugins):
cvat_server                    |   File "/usr/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
cvat_server                    |     yield _result_or_cancel(fs.pop())
cvat_server                    |   File "/usr/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
cvat_server                    |     return fut.result(timeout)
cvat_server                    |   File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result
cvat_server                    |     return self.__get_result()
cvat_server                    |   File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
cvat_server                    |     raise self._exception
cvat_server                    |   File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
cvat_server                    |     result = self.fn(*self.args, **self.kwargs)
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/health_check/mixins.py", line 35, in _run
cvat_server                    |     plugin.run_check()
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/health_check/backends.py", line 30, in run_check
cvat_server                    |     self.check_status()
cvat_server                    |   File "/home/django/cvat/apps/health/backends.py", line 44, in check_status
cvat_server                    |     cache.set("djangohealtcheck_test", "itworks")
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/django/core/cache/backends/redis.py", line 191, in set
cvat_server                    |     self._cache.set(key, value, self.get_backend_timeout(timeout))
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/django/core/cache/backends/redis.py", line 108, in set
cvat_server                    |     client.set(key, value, ex=timeout)
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/redis/commands/core.py", line 2302, in set
cvat_server                    |     return self.execute_command("SET", *pieces, **options)
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/redis/client.py", line 1255, in execute_command
cvat_server                    |     conn = self.connection or pool.get_connection(command_name, **options)
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 1442, in get_connection
cvat_server                    |     connection.connect()
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 704, in connect
cvat_server                    |     raise ConnectionError(self._error_message(e))
cvat_server                    | redis.exceptions.ConnectionError: Error -2 connecting to cvat_redis:6379. Name or service not known.
cvat_server                    | 
cvat_server                    | 2023-12-15 15:57:59,290 DEBG 'uvicorn-1' stdout output:
cvat_server                    | INFO:     192.168.65.1:0 - "GET /api/server/health/?format=json&org= HTTP/1.0" 500 Internal Server Error

BeardedUnicorn commented 11 months ago

@robmarkcole Did you run docker-compose down before you ran docker-compose up? Can you docker-compose ps and post the output?

robmarkcole commented 11 months ago

@BeardedUnicorn yes same with down first.

 > docker-compose ps
NAME                            IMAGE                                       COMMAND                  SERVICE                         CREATED         STATUS                  PORTS
cvat_clickhouse                 clickhouse/clickhouse-server:23.11-alpine   "/entrypoint.sh"         cvat_clickhouse                 7 seconds ago   Up 6 seconds            8123/tcp, 9000/tcp, 9009/tcp
cvat_db                         postgres:15-alpine                          "docker-entrypoint.s…"   cvat_db                         7 seconds ago   Up 6 seconds            5432/tcp
cvat_grafana                    grafana/grafana-oss:10.1.2                  "sh -euc 'mkdir -p /…"   cvat_grafana                    7 seconds ago   Up 6 seconds            3000/tcp
cvat_opa                        openpolicyagent/opa:0.45.0-rootless         "/opa run --server -…"   cvat_opa                        7 seconds ago   Up 6 seconds            
cvat_redis                      eqalpha/keydb:x86_64_v6.3.2                 "docker-entrypoint.s…"   cvat_redis                      7 seconds ago   Up Less than a second   6379/tcp
cvat_server                     cvat/server:dev                             "./backend_entrypoin…"   cvat_server                     7 seconds ago   Up 5 seconds            8080/tcp
cvat_ui                         cvat/ui:dev                                 "/docker-entrypoint.…"   cvat_ui                         7 seconds ago   Up 5 seconds            80/tcp
cvat_utils                      cvat/server:dev                             "./backend_entrypoin…"   cvat_utils                      7 seconds ago   Up 5 seconds            8080/tcp
cvat_vector                     timberio/vector:0.26.0-alpine               "/usr/local/bin/vect…"   cvat_vector                     7 seconds ago   Up 5 seconds            
cvat_worker_analytics_reports   cvat/server:dev                             "./backend_entrypoin…"   cvat_worker_analytics_reports   7 seconds ago   Up 6 seconds            8080/tcp
cvat_worker_annotation          cvat/server:dev                             "./backend_entrypoin…"   cvat_worker_annotation          7 seconds ago   Up 6 seconds            8080/tcp
cvat_worker_export              cvat/server:dev                             "./backend_entrypoin…"   cvat_worker_export              7 seconds ago   Up 6 seconds            8080/tcp
cvat_worker_import              cvat/server:dev                             "./backend_entrypoin…"   cvat_worker_import              7 seconds ago   Up 5 seconds            8080/tcp
cvat_worker_quality_reports     cvat/server:dev                             "./backend_entrypoin…"   cvat_worker_quality_reports     7 seconds ago   Up 6 seconds            8080/tcp
cvat_worker_webhooks            cvat/server:dev                             "./backend_entrypoin…"   cvat_worker_webhooks            7 seconds ago   Up 5 seconds            8080/tcp
traefik                         traefik:v2.10                               "/entrypoint.sh trae…"   traefik                         7 seconds ago   Up 6 seconds            0.0.0.0:8080->8080/tcp, 80/tcp, 0.0.0.0:8090->8090/tcp

solo2307 commented 11 months ago

I have the similar issue to compose the docker: CVAT_VERSION=dev DOCKER_DEFAULT_PLATFORM=linux/amd64 docker compose up -d

the response to the above command line: Error response from daemon: image with reference traefik:v2.10 was found but does not match the specified platform: wanted linux/amd64, actual: linux/arm64/v8

henryle97 commented 7 months ago

Error 1: Platform-related Error

If you encounter the error: Error response from daemon: image with reference IMAGE was found but does not match the specified platform: wanted linux/amd64, actual: linux/arm64/v8 Follow these steps:

Remove all existing CVAT-related Docker images.
```
docker-compose down --rmi all
```

Set the Docker default platform and start the containers:

  export DOCKER_DEFAULT_PLATFORM=linux/amd64
  docker compose up

If you still see this error: Error response from daemon: image with reference <IMAGE> was found but does not match the specified platform: wanted linux/amd64, actual: linux/arm64. Remove the problematic image and restart the compose process:
```
docker image rmi traefik:v2.10
docker compose up
```
Continue this process until there are no more platform-related errors.

Error 2: CVAT_OPA Error

If CVAT still doesn't run successfully and you encounter errors related to cvat_opa:

cvat_opa  | {"level":"error","msg":"Bundle load failed: request failed: Get \"http://cvat-server:8080/api/auth/rules\": dial tcp 192.168.0.17:8080: connect: connection refused","name":"cvat","plugin":"bundle","time":"2024-04-27T08:37:17Z"}

or

  File "/home/django/cvat/apps/iam/views.py", line 168, in _etag_func
    with open(file_path, 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/django/static/opa/bundle.tar.gz'

Try the following:

Bring down all services:
```
docker compose down
```
Attempt to start cvat_opa separately:
```
docker compose up cvat_opa -d
```

Wait for 15s then comment out the cvat_opa service in the docker-compose.yml file.

services:
...
cvat_server:
container_name: cvat_server
image: cvat/server:${CVAT_VERSION:-dev}
restart: always
depends_on:
  <<: *backend-deps
  # cvat_opa:
  #   condition:
  #     service_healthy
...
# cvat_opa:
#   container_name: cvat_opa
#   image: openpolicyagent/opa:0.63.0
#   restart: always
#   networks:
#     cvat:
#       aliases:
#         - opa
#   command:
#     - run
#     - --server
#     - --log-level=error
#     - --set=services.cvat.url=http://cvat-server:8080
#     - --set=bundles.cvat.service=cvat
#     - --set=bundles.cvat.resource=/api/auth/rules
#     - --set=bundles.cvat.polling.min_delay_seconds=5
#     - --set=bundles.cvat.polling.max_delay_seconds=15

Restart all services in detached mode:

docker compose up -d

These steps should help you resolve common issues when setting up CVAT on Mac M1.

cvat-ai / cvat