Could not get models from the server - Error: Request failed with status code 500.

cognitiveRobot commented 4 years ago

My actions before raising this issue

[ ] Read/searched the docs
[ ] Searched past issues

Expected Behaviour

I was expecting to see a model that is currently running.

Current Behaviour

After deploying the TensorFlow model, when I go to http://localhost:8080/models I see the error message as below.

Could not get models from the server
Error: Request failed with status code 500.
"<h1>Server Error (500)</h1>".

Possible Solution

Still searching.

Your Environment

Git hash commit (git log -1):


Author: Dmitry Kruchinin <33020454+dvkruchinin@users.noreply.github.com>
Date:   Thu Sep 3 13:59:59 2020 +0300

Cypress test for issue 1568. (#2106)

Co-authored-by: Dmitry Kruchinin <dmitryx.kruchinin@intel.com>

- Docker version :

Docker version 19.03.12, build 48a66213fe

- Are you using Docker Swarm or Kubernetes?
`No`
- Operating System and version:

Distributor ID: Ubuntu Description: Ubuntu 16.04.7 LTS Release: 16.04 Codename: xenial

- Kernel

4.15.0-112-generic

- Docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 844d53c73bb3 nginx:stable-alpine "/docker-entrypoint.…" 34 minutes ago Up 34 minutes 0.0.0.0:8080->80/tcp cvat_proxy cba3a870d17b cvat/ui "/docker-entrypoint.…" 34 minutes ago Up 34 minutes 80/tcp cvat_ui d75ba5df87b0 cvat/server "/usr/bin/supervisord" 34 minutes ago Up 34 minutes 8080/tcp, 8443/tcp cvat 3110cba77a41 redis:4.0-alpine "docker-entrypoint.s…" 34 minutes ago Up 34 minutes 6379/tcp cvat_redis aed39d784a81 quay.io/nuclio/dashboard:1.4.8-amd64 "sh -c ./runner.sh" 34 minutes ago Up 34 minutes 80/tcp, 0.0.0.0:8070->8070/tcp nuclio 6d2d0b0fcdb0 postgres:10-alpine "docker-entrypoint.s…" 34 minutes ago Up 34 minutes 5432/tcp cvat_db eb242020f1a8 cvat/tf.faster_rcnn_inception_v2_coco:latest "processor" 35 minutes ago Up 35 minutes (healthy) 0.0.0.0:42561->8080/tcp nuclio-nuclio-tf.faster_rcnn_inception_v2_coco ae3f4109232a alpine:3.11 "/bin/sh -c '/bin/sl…" 4 hours ago Up 4 hours nuclio-local-storage-reader e33a707a4c6e nuclio/processor-test:latest "processor" 10 hours ago Up 10 hours (healthy) 0.0.0.0:36521->8080/tcp nuclio-nuclio-test

- Server deployment code:
https://github.com/opencv/cvat/tree/develop/serverless/tensorflow/faster_rcnn_inception_v2_coco/nuclio 
- Server deployment command:

./nuctl deploy --project-name cvat_test --path cvat/serverless/tensorflow/faster_rcnn_inception_v2_coco/nuclio --volume pwd/cvat/serverless/openvino/common:/opt/nuclio/common --platform local

- Server deployment log:

20.09.04 18:47:52.471 nuctl (I) Deploying function {"name": ""} 20.09.04 18:47:52.471 nuctl (I) Building {"versionInfo": "Label: 1.4.17, Git commit: 278c7a4fb23a93973d16d87dbaaad87823e9644f, OS: linux, Arch: amd64, Go version: go1.14.3", "name": ""} 20.09.04 18:47:52.702 nuctl (I) Cleaning up before deployment 20.09.04 18:47:52.749 nuctl (I) Staging files and preparing base images 20.09.04 18:47:52.750 nuctl (I) Building processor image {"imageName": "cvat/tf.faster_rcnn_inception_v2_coco:latest"} 20.09.04 18:47:52.750 nuctl.platform.docker (I) Pulling image {"imageName": "quay.io/nuclio/handler-builder-python-onbuild:1.4.17-amd64"} 20.09.04 18:47:56.570 nuctl.platform.docker (I) Pulling image {"imageName": "quay.io/nuclio/uhttpc:0.0.1-amd64"} 20.09.04 18:48:00.607 nuctl.platform (I) Building docker image {"image": "cvat/tf.faster_rcnn_inception_v2_coco:latest"} 20.09.04 18:48:01.002 nuctl.platform (I) Pushing docker image into registry {"image": "cvat/tf.faster_rcnn_inception_v2_coco:latest", "registry": ""} 20.09.04 18:48:01.002 nuctl.platform (I) Docker image was successfully built and pushed into docker registry {"image": "cvat/tf.faster_rcnn_inception_v2_coco:latest"} 20.09.04 18:48:01.002 nuctl (I) Build complete {"result": {"Image":"cvat/tf.faster_rcnn_inception_v2_coco:latest","UpdatedFunctionConfig":{"metadata":{"name":"tf.faster_rcnn_inception_v2_coco","namespace":"nuclio","labels":{"nuclio.io/project-name":"cvat_test"},"annotations":{"framework":"tensorflow","name":"Faster RCNN via Tensorflow","spec":"[\n { \"id\": 1, \"name\": \"person\" },\n { \"id\": 2, \"name\": \"bicycle\" },\n { \"id\": 3, \"name\": \"car\" },\n { \"id\": 4, \"name\": \"motorcycle\" },\n { \"id\": 5, \"name\": \"airplane\" },\n { \"id\": 6, \"name\": \"bus\" },\n { \"id\": 7, \"name\": \"train\" },\n { \"id\": 8, \"name\": \"truck\" },\n { \"id\": 9, \"name\": \"boat\" },\n { \"id\":10, \"name\": \"traffic_light\" },\n { \"id\":11, \"name\": \"fire_hydrant\" },\n { \"id\":13, \"name\": \"stop_sign\" },\n { \"id\":14, \"name\": \"parking_meter\" },\n { \"id\":15, \"name\": \"bench\" },\n { \"id\":16, \"name\": \"bird\" },\n { \"id\":17, \"name\": \"cat\" },\n { \"id\":18, \"name\": \"dog\" },\n { \"id\":19, \"name\": \"horse\" },\n { \"id\":20, \"name\": \"sheep\" },\n { \"id\":21, \"name\": \"cow\" },\n { \"id\":22, \"name\": \"elephant\" },\n { \"id\":23, \"name\": \"bear\" },\n { \"id\":24, \"name\": \"zebra\" },\n { \"id\":25, \"name\": \"giraffe\" },\n { \"id\":27, \"name\": \"backpack\" },\n { \"id\":28, \"name\": \"umbrella\" },\n { \"id\":31, \"name\": \"handbag\" },\n { \"id\":32, \"name\": \"tie\" },\n { \"id\":33, \"name\": \"suitcase\" },\n { \"id\":34, \"name\": \"frisbee\" },\n { \"id\":35, \"name\": \"skis\" },\n { \"id\":36, \"name\": \"snowboard\" },\n { \"id\":37, \"name\": \"sports_ball\" },\n { \"id\":38, \"name\": \"kite\" },\n { \"id\":39, \"name\": \"baseball_bat\" },\n { \"id\":40, \"name\": \"baseball_glove\" },\n { \"id\":41, \"name\": \"skateboard\" },\n { \"id\":42, \"name\": \"surfboard\" },\n { \"id\":43, \"name\": \"tennis_racket\" },\n { \"id\":44, \"name\": \"bottle\" },\n { \"id\":46, \"name\": \"wine_glass\" },\n { \"id\":47, \"name\": \"cup\" },\n { \"id\":48, \"name\": \"fork\" },\n { \"id\":49, \"name\": \"knife\" },\n { \"id\":50, \"name\": \"spoon\" },\n { \"id\":51, \"name\": \"bowl\" },\n { \"id\":52, \"name\": \"banana\" },\n { \"id\":53, \"name\": \"apple\" },\n { \"id\":54, \"name\": \"sandwich\" },\n { \"id\":55, \"name\": \"orange\" },\n { \"id\":56, \"name\": \"broccoli\" },\n { \"id\":57, \"name\": \"carrot\" },\n { \"id\":58, \"name\": \"hot_dog\" },\n { \"id\":59, \"name\": \"pizza\" },\n { \"id\":60, \"name\": \"donut\" },\n { \"id\":61, \"name\": \"cake\" },\n { \"id\":62, \"name\": \"chair\" },\n { \"id\":63, \"name\": \"couch\" },\n { \"id\":64, \"name\": \"potted_plant\" },\n { \"id\":65, \"name\": \"bed\" },\n { \"id\":67, \"name\": \"dining_table\" },\n { \"id\":70, \"name\": \"toilet\" },\n { \"id\":72, \"name\": \"tv\" },\n { \"id\":73, \"name\": \"laptop\" },\n { \"id\":74, \"name\": \"mouse\" },\n { \"id\":75, \"name\": \"remote\" },\n { \"id\":76, \"name\": \"keyboard\" },\n { \"id\":77, \"name\": \"cell_phone\" },\n { \"id\":78, \"name\": \"microwave\" },\n { \"id\":79, \"name\": \"oven\" },\n { \"id\":80, \"name\": \"toaster\" },\n { \"id\":81, \"name\": \"sink\" },\n { \"id\":83, \"name\": \"refrigerator\" },\n { \"id\":84, \"name\": \"book\" },\n { \"id\":85, \"name\": \"clock\" },\n { \"id\":86, \"name\": \"vase\" },\n { \"id\":87, \"name\": \"scissors\" },\n { \"id\":88, \"name\": \"teddy_bear\" },\n { \"id\":89, \"name\": \"hair_drier\" },\n { \"id\":90, \"name\": \"toothbrush\" }\n]\n","type":"detector"}},"spec":{"description":"Faster RCNN from Tensorflow Object Detection API","handler":"main:handler","runtime":"python:3.6","resources":{},"image":"cvat/tf.faster_rcnn_inception_v2_coco:latest","targetCPU":75,"triggers":{"myHttpTrigger":{"class":"","kind":"http","name":"","maxWorkers":2,"workerAvailabilityTimeoutMilliseconds":10000,"attributes":{"maxRequestBodySize":33554432}}},"volumes":[{"volume":{"name":"volume-1","hostPath":{"path":"/mnt/hdd/projects/cvat/serverless/openvino/common"}},"volumeMount":{"name":"volume-1","mountPath":"/opt/nuclio/common"}}],"build":{"image":"cvat/tf.faster_rcnn_inception_v2_coco","baseImage":"tensorflow/tensorflow:2.1.1","directives":{"postCopy":[{"kind":"RUN","value":"curl -O http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_v2_coco_2018_01_28.tar.gz"},{"kind":"RUN","value":"tar -xzf faster_rcnn_inception_v2_coco_2018_01_28.tar.gz \u0026\u0026 rm faster_rcnn_inception_v2_coco_2018_01_28.tar.gz"},{"kind":"RUN","value":"ln -s faster_rcnn_inception_v2_coco_2018_01_28 faster_rcnn"},{"kind":"RUN","value":"pip install pillow pyyaml"}],"preCopy":[{"kind":"RUN","value":"apt install curl"},{"kind":"WORKDIR","value":"/opt/nuclio"}]},"codeEntryType":"image"},"platform":{"attributes":{"restartPolicy":{"maximumRetryCount":3,"name":"always"}}},"readinessTimeoutSeconds":60,"eventTimeout":"30s"}}}} 20.09.04 18:48:02.165 nuctl.platform (I) Waiting for function to be ready {"timeout": 60} 20.09.04 18:48:03.501 nuctl (I) Function deploy complete {"functionName": "tf.faster_rcnn_inception_v2_coco", "httpPort": 42561}



Please help me to find a solution. :)

nmanovic commented 4 years ago

@cognitiveRobot , could you please attach server logs? (docker logs cvat). Also please attach logs from the serverless function (docker logs nuclio-nuclio-tf.faster_rcnn_inception_v2_coco)

cognitiveRobot commented 4 years ago

Thank @nmanovic . Here they are. docker logs cvat:

2020-09-04 23:19:19,073 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.

2020-09-04 23:19:19,073 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.

2020-09-04 23:19:19,074 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.

2020-09-04 23:23:49,098 DEBG 'runserver' stderr output:
[Fri Sep 04 23:23:49.098474 2020] [wsgi:error] [pid 295:tid 139981089711872] [remote 172.28.0.7:36636] [2020-09-04 23:23:49,098] WARNING django.request: Not Found: /analytics/app/kibana

2020-09-04 23:23:49,100 DEBG 'runserver' stderr output:
[Fri Sep 04 23:23:49.100487 2020] [wsgi:error] [pid 295:tid 139981089711872] [remote 172.28.0.7:36636] WARNING:django.request:Not Found: /analytics/app/kibana

2020-09-04 23:23:49,271 DEBG 'runserver' stderr output:
[Fri Sep 04 23:23:49.270997 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628] [2020-09-04 23:23:49,270] ERROR django.request: Internal Server Error: /api/v1/lambda/functions
[Fri Sep 04 23:23:49.271014 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628] Traceback (most recent call last):
[Fri Sep 04 23:23:49.271018 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/exception.py", line 34, in inner
[Fri Sep 04 23:23:49.271022 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     response = get_response(request)
[Fri Sep 04 23:23:49.271025 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/base.py", line 115, in _get_response
[Fri Sep 04 23:23:49.271029 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     response = self.process_exception_by_middleware(e, request)
[Fri Sep 04 23:23:49.271033 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/base.py", line 113, in _get_response
[Fri Sep 04 23:23:49.271036 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     response = wrapped_callback(request, *callback_args, **callback_kwargs)
[Fri Sep 04 23:23:49.271039 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
[Fri Sep 04 23:23:49.271043 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     return view_func(*args, **kwargs)
[Fri Sep 04 23:23:49.271046 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/rest_framework/viewsets.py", line 114, in view
[Fri Sep 04 23:23:49.271050 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     return self.dispatch(request, *args, **kwargs)
[Fri Sep 04 23:23:49.271053 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/rest_framework/views.py", line 505, in dispatch
[Fri Sep 04 23:23:49.271056 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     response = self.handle_exception(exc)

2020-09-04 23:23:49,271 DEBG 'runserver' stderr output:
[Fri Sep 04 23:23:49.271060 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/rest_framework/views.py", line 465, in handle_exception
[Fri Sep 04 23:23:49.271064 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     self.raise_uncaught_exception(exc)
[Fri Sep 04 23:23:49.271067 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/rest_framework/views.py", line 476, in raise_uncaught_exception
[Fri Sep 04 23:23:49.271071 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     raise exc
[Fri Sep 04 23:23:49.271074 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/rest_framework/views.py", line 502, in dispatch
[Fri Sep 04 23:23:49.271077 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     response = handler(request, *args, **kwargs)
[Fri Sep 04 23:23:49.271081 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/home/django/cvat/apps/lambda_manager/views.py", line 485, in func_wrapper
[Fri Sep 04 23:23:49.271084 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     data = func(*args, **kwargs)
[Fri Sep 04 23:23:49.271087 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/home/django/cvat/apps/lambda_manager/views.py", line 523, in list
[Fri Sep 04 23:23:49.271091 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     return [f.to_dict() for f in gateway.list()]
[Fri Sep 04 23:23:49.271094 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/home/django/cvat/apps/lambda_manager/views.py", line 63, in list
[Fri Sep 04 23:23:49.271097 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     response = [LambdaFunction(self, item) for item in data.values()]
[Fri Sep 04 23:23:49.271101 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/home/django/cvat/apps/lambda_manager/views.py", line 63, in <listcomp>
[Fri Sep 04 23:23:49.271104 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     response = [LambdaFunction(self, item) for item in data.values()]
[Fri Sep 04 23:23:49.271107 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/home/django/cvat/apps/lambda_manager/views.py", line 89, in __init__
[Fri Sep 04 23:23:49.271111 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     kind = data['metadata']['annotations'].get('type')
[Fri Sep 04 23:23:49.271114 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628] KeyError: 'annotations'
[Fri Sep 04 23:23:49.271246 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628] ERROR:django.request:Internal Server Error: /api/v1/lambda/functions
[Fri Sep 04 23:23:49.271252 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628] Traceback (most recent call last):
[Fri Sep 04 23:23:49.271255 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/exception.py", line 34, in inner
[Fri Sep 04 23:23:49.271259 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     response = get_response(request)
[Fri Sep 04 23:23:49.271262 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/base.py", line 115, in _get_response
[Fri Sep 04 23:23:49.271265 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     response = self.process_exception_by_middleware(e, request)
[Fri Sep 04 23:23:49.271269 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/base.py", line 113, in _get_response
[Fri Sep 04 23:23:49.271272 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     response = wrapped_callback(request, *callback_args, **callback_kwargs)
[Fri Sep 04 23:23:49.271278 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
[Fri Sep 04 23:23:49.271282 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     return view_func(*args, **kwargs)
[Fri Sep 04 23:23:49.271285 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/rest_framework/viewsets.py", line 114, in view
[Fri Sep 04 23:23:49.271289 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     return self.dispatch(request, *args, **kwargs)
[Fri Sep 04 23:23:49.271292 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/rest_framework/views.py", line 505, in dispatch
[Fri Sep 04 23:23:49.271295 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     response = self.handle_exception(exc)
[Fri Sep 04 23:23:49.271298 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/rest_framework/views.py", line 465, in handle_exception
[Fri Sep 04 23:23:49.271302 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     self.raise_uncaught_exception(exc)
[Fri Sep 04 23:23:49.271305 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/rest_framework/views.py", line 476, in raise_uncaught_exception
[Fri Sep 04 23:23:49.271309 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     raise exc
[Fri Sep 04 23:23:49.271312 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/usr/local/lib/python3.8/dist-packages/rest_framework/views.py", line 502, in dispatch
[Fri Sep 04 23:23:49.271315 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     response = handler(request, *args, **kwargs)
[Fri Sep 04 23:23:49.271319 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/home/django/cvat/apps/lambda_manager/views.py", line 485, in func_wrapper
[Fri Sep 04 23:23:49.271322 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     data = func(*args, **kwargs)
[Fri Sep 04 23:23:49.271325 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/home/django/cvat/apps/lambda_manager/views.py", line 523, in list
[Fri Sep 04 23:23:49.271328 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     return [f.to_dict() for f in gateway.list()]
[Fri Sep 04 23:23:49.271332 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/home/django/cvat/apps/lambda_manager/views.py", line 63, in list
[Fri Sep 04 23:23:49.271335 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     response = [LambdaFunction(self, item) for item in data.values()]
[Fri Sep 04 23:23:49.271338 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/home/django/cvat/apps/lambda_manager/views.py", line 63, in <listcomp>
[Fri Sep 04 23:23:49.271342 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     response = [LambdaFunction(self, item) for item in data.values()]
[Fri Sep 04 23:23:49.271345 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]   File "/home/django/cvat/apps/lambda_manager/views.py", line 89, in __init__
[Fri Sep 04 23:23:49.271348 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628]     kind = data['metadata']['annotations'].get('type')
[Fri Sep 04 23:23:49.271352 2020] [wsgi:error] [pid 295:tid 139981098104576] [remote 172.28.0.7:36628] KeyError: 'annotations'

docker logs nuclio-nuclio-tf.faster_rcnn_inception_v2_coco

20.09.04 23:21:48.846                 processor (I) Starting processor {"version": "Label: 1.4.17, Git commit: 278c7a4fb23a93973d16d87dbaaad87823e9644f, OS: linux, Arch: amd64, Go version: go1.14.3"}
20.09.04 23:21:48.846                 processor (D) Read configuration {"config": "{\n    \"metadata\": {\n        \"name\": \"tf.faster_rcnn_inception_v2_coco\",\n        \"namespace\": \"nuclio\",\n        \"labels\": {\n            \"nuclio.io/project-name\": \"cvat_test\"\n        },\n        \"annotations\": {\n            \"framework\": \"tensorflow\",\n            \"name\": \"Faster RCNN via Tensorflow\",\n            \"spec\": \"[\\n  { \\\"id\\\": 1, \\\"name\\\": \\\"person\\\" },\\n  { \\\"id\\\": 2, \\\"name\\\": \\\"bicycle\\\" },\\n  { \\\"id\\\": 3, \\\"name\\\": \\\"car\\\" },\\n  { \\\"id\\\": 4, \\\"name\\\": \\\"motorcycle\\\" },\\n  { \\\"id\\\": 5, \\\"name\\\": \\\"airplane\\\" },\\n  { \\\"id\\\": 6, \\\"name\\\": \\\"bus\\\" },\\n  { \\\"id\\\": 7, \\\"name\\\": \\\"train\\\" },\\n  { \\\"id\\\": 8, \\\"name\\\": \\\"truck\\\" },\\n  { \\\"id\\\": 9, \\\"name\\\": \\\"boat\\\" },\\n  { \\\"id\\\":10, \\\"name\\\": \\\"traffic_light\\\" },\\n  { \\\"id\\\":11, \\\"name\\\": \\\"fire_hydrant\\\" },\\n  { \\\"id\\\":13, \\\"name\\\": \\\"stop_sign\\\" },\\n  { \\\"id\\\":14, \\\"name\\\": \\\"parking_meter\\\" },\\n  { \\\"id\\\":15, \\\"name\\\": \\\"bench\\\" },\\n  { \\\"id\\\":16, \\\"name\\\": \\\"bird\\\" },\\n  { \\\"id\\\":17, \\\"name\\\": \\\"cat\\\" },\\n  { \\\"id\\\":18, \\\"name\\\": \\\"dog\\\" },\\n  { \\\"id\\\":19, \\\"name\\\": \\\"horse\\\" },\\n  { \\\"id\\\":20, \\\"name\\\": \\\"sheep\\\" },\\n  { \\\"id\\\":21, \\\"name\\\": \\\"cow\\\" },\\n  { \\\"id\\\":22, \\\"name\\\": \\\"elephant\\\" },\\n  { \\\"id\\\":23, \\\"name\\\": \\\"bear\\\" },\\n  { \\\"id\\\":24, \\\"name\\\": \\\"zebra\\\" },\\n  { \\\"id\\\":25, \\\"name\\\": \\\"giraffe\\\" },\\n  { \\\"id\\\":27, \\\"name\\\": \\\"backpack\\\" },\\n  { \\\"id\\\":28, \\\"name\\\": \\\"umbrella\\\" },\\n  { \\\"id\\\":31, \\\"name\\\": \\\"handbag\\\" },\\n  { \\\"id\\\":32, \\\"name\\\": \\\"tie\\\" },\\n  { \\\"id\\\":33, \\\"name\\\": \\\"suitcase\\\" },\\n  { \\\"id\\\":34, \\\"name\\\": \\\"frisbee\\\" },\\n  { \\\"id\\\":35, \\\"name\\\": \\\"skis\\\" },\\n  { \\\"id\\\":36, \\\"name\\\": \\\"snowboard\\\" },\\n  { \\\"id\\\":37, \\\"name\\\": \\\"sports_ball\\\" },\\n  { \\\"id\\\":38, \\\"name\\\": \\\"kite\\\" },\\n  { \\\"id\\\":39, \\\"name\\\": \\\"baseball_bat\\\" },\\n  { \\\"id\\\":40, \\\"name\\\": \\\"baseball_glove\\\" },\\n  { \\\"id\\\":41, \\\"name\\\": \\\"skateboard\\\" },\\n  { \\\"id\\\":42, \\\"name\\\": \\\"surfboard\\\" },\\n  { \\\"id\\\":43, \\\"name\\\": \\\"tennis_racket\\\" },\\n  { \\\"id\\\":44, \\\"name\\\": \\\"bottle\\\" },\\n  { \\\"id\\\":46, \\\"name\\\": \\\"wine_glass\\\" },\\n  { \\\"id\\\":47, \\\"name\\\": \\\"cup\\\" },\\n  { \\\"id\\\":48, \\\"name\\\": \\\"fork\\\" },\\n  { \\\"id\\\":49, \\\"name\\\": \\\"knife\\\" },\\n  { \\\"id\\\":50, \\\"name\\\": \\\"spoon\\\" },\\n  { \\\"id\\\":51, \\\"name\\\": \\\"bowl\\\" },\\n  { \\\"id\\\":52, \\\"name\\\": \\\"banana\\\" },\\n  { \\\"id\\\":53, \\\"name\\\": \\\"apple\\\" },\\n  { \\\"id\\\":54, \\\"name\\\": \\\"sandwich\\\" },\\n  { \\\"id\\\":55, \\\"name\\\": \\\"orange\\\" },\\n  { \\\"id\\\":56, \\\"name\\\": \\\"broccoli\\\" },\\n  { \\\"id\\\":57, \\\"name\\\": \\\"carrot\\\" },\\n  { \\\"id\\\":58, \\\"name\\\": \\\"hot_dog\\\" },\\n  { \\\"id\\\":59, \\\"name\\\": \\\"pizza\\\" },\\n  { \\\"id\\\":60, \\\"name\\\": \\\"donut\\\" },\\n  { \\\"id\\\":61, \\\"name\\\": \\\"cake\\\" },\\n  { \\\"id\\\":62, \\\"name\\\": \\\"chair\\\" },\\n  { \\\"id\\\":63, \\\"name\\\": \\\"couch\\\" },\\n  { \\\"id\\\":64, \\\"name\\\": \\\"potted_plant\\\" },\\n  { \\\"id\\\":65, \\\"name\\\": \\\"bed\\\" },\\n  { \\\"id\\\":67, \\\"name\\\": \\\"dining_table\\\" },\\n  { \\\"id\\\":70, \\\"name\\\": \\\"toilet\\\" },\\n  { \\\"id\\\":72, \\\"name\\\": \\\"tv\\\" },\\n  { \\\"id\\\":73, \\\"name\\\": \\\"laptop\\\" },\\n  { \\\"id\\\":74, \\\"name\\\": \\\"mouse\\\" },\\n  { \\\"id\\\":75, \\\"name\\\": \\\"remote\\\" },\\n  { \\\"id\\\":76, \\\"name\\\": \\\"keyboard\\\" },\\n  { \\\"id\\\":77, \\\"name\\\": \\\"cell_phone\\\" },\\n  { \\\"id\\\":78, \\\"name\\\": \\\"microwave\\\" },\\n  { \\\"id\\\":79, \\\"name\\\": \\\"oven\\\" },\\n  { \\\"id\\\":80, \\\"name\\\": \\\"toaster\\\" },\\n  { \\\"id\\\":81, \\\"name\\\": \\\"sink\\\" },\\n  { \\\"id\\\":83, \\\"name\\\": \\\"refrigerator\\\" },\\n  { \\\"id\\\":84, \\\"name\\\": \\\"book\\\" },\\n  { \\\"id\\\":85, \\\"name\\\": \\\"clock\\\" },\\n  { \\\"id\\\":86, \\\"name\\\": \\\"vase\\\" },\\n  { \\\"id\\\":87, \\\"name\\\": \\\"scissors\\\" },\\n  { \\\"id\\\":88, \\\"name\\\": \\\"teddy_bear\\\" },\\n  { \\\"id\\\":89, \\\"name\\\": \\\"hair_drier\\\" },\\n  { \\\"id\\\":90, \\\"name\\\": \\\"toothbrush\\\" }\\n]\\n\",\n            \"type\": \"detector\"\n        }\n    },\n    \"spec\": {\n        \"description\": \"Faster RCNN from Tensorflow Object Detection API\",\n        \"handler\": \"main:handler\",\n        \"runtime\": \"python:3.6\",\n        \"resources\": {},\n        \"image\": \"cvat/tf.faster_rcnn_inception_v2_coco:latest\",\n        \"targetCPU\": 75,\n        \"triggers\": {\n            \"myHttpTrigger\": {\n                \"class\": \"\",\n                \"kind\": \"http\",\n                \"name\": \"\",\n                \"maxWorkers\": 2,\n                \"workerAvailabilityTimeoutMilliseconds\": 10000,\n                \"attributes\": {\n                    \"maxRequestBodySize\": 33554432\n                }\n            }\n        },\n        \"volumes\": [\n            {\n                \"volume\": {\n                    \"name\": \"volume-1\",\n                    \"hostPath\": {\n                        \"path\": \"/mnt/hdd/projects/cvat/serverless/openvino/common\"\n                    }\n                },\n                \"volumeMount\": {\n                    \"name\": \"volume-1\",\n                    \"mountPath\": \"/opt/nuclio/common\"\n                }\n            }\n        ],\n        \"build\": {\n            \"image\": \"cvat/tf.faster_rcnn_inception_v2_coco\",\n            \"baseImage\": \"tensorflow/tensorflow:2.1.1\",\n            \"directives\": {\n                \"postCopy\": [\n                    {\n                        \"kind\": \"RUN\",\n                        \"value\": \"curl -O http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_v2_coco_2018_01_28.tar.gz\"\n                    },\n                    {\n                        \"kind\": \"RUN\",\n                        \"value\": \"tar -xzf faster_rcnn_inception_v2_coco_2018_01_28.tar.gz \\u0026\\u0026 rm faster_rcnn_inception_v2_coco_2018_01_28.tar.gz\"\n                    },\n                    {\n                        \"kind\": \"RUN\",\n                        \"value\": \"ln -s faster_rcnn_inception_v2_coco_2018_01_28 faster_rcnn\"\n                    },\n                    {\n                        \"kind\": \"RUN\",\n                        \"value\": \"pip install pillow pyyaml\"\n                    }\n                ],\n                \"preCopy\": [\n                    {\n                        \"kind\": \"RUN\",\n                        \"value\": \"apt install curl\"\n                    },\n                    {\n                        \"kind\": \"WORKDIR\",\n                        \"value\": \"/opt/nuclio\"\n                    }\n                ]\n            },\n            \"codeEntryType\": \"image\",\n            \"timestamp\": 1599261708\n        },\n        \"platform\": {\n            \"attributes\": {\n                \"restartPolicy\": {\n                    \"maximumRetryCount\": 3,\n                    \"name\": \"always\"\n                }\n            }\n        },\n        \"readinessTimeoutSeconds\": 60,\n        \"eventTimeout\": \"30s\"\n    },\n    \"PlatformConfig\": null\n}", "platformConfig": "{\n    \"kind\": \"local\",\n    \"webAdmin\": {\n        \"enabled\": true,\n        \"listenAddress\": \":8081\"\n    },\n    \"healthCheck\": {\n        \"enabled\": true,\n        \"listenAddress\": \":8082\"\n    },\n    \"logger\": {\n        \"sinks\": {\n            \"stdout\": {\n                \"kind\": \"stdout\"\n            }\n        },\n        \"system\": [\n            {\n                \"level\": \"debug\",\n                \"sink\": \"stdout\"\n            }\n        ],\n        \"functions\": [\n            {\n                \"level\": \"debug\",\n                \"sink\": \"stdout\"\n            }\n        ]\n    },\n    \"metrics\": {},\n    \"scaleToZero\": {},\n    \"autoScale\": {},\n    \"cronTriggerCreationMode\": \"processor\"\n}"}
20.09.04 23:21:48.847 cessor.healthcheck.server (I) Listening {"listenAddress": ":8082"}
20.09.04 23:21:48.847            processor.http (D) Creating worker pool {"num": 2}
20.09.04 23:21:48.847 sor.http.w1.python.logger (D) Creating listener socket {"path": "/tmp/nuclio-rpc-bt9co339v39n39jg38l0.sock"}
20.09.04 23:21:48.847 sor.http.w0.python.logger (D) Creating listener socket {"path": "/tmp/nuclio-rpc-bt9co339v39n39jg38lg.sock"}
20.09.04 23:21:48.847 sor.http.w1.python.logger (D) Using Python wrapper script path {"path": "/opt/nuclio/_nuclio_wrapper.py"}
20.09.04 23:21:48.847 sor.http.w0.python.logger (D) Using Python wrapper script path {"path": "/opt/nuclio/_nuclio_wrapper.py"}
20.09.04 23:21:48.847 sor.http.w1.python.logger (D) Using Python handler {"handler": "main:handler"}
20.09.04 23:21:48.847 sor.http.w0.python.logger (D) Using Python handler {"handler": "main:handler"}
20.09.04 23:21:48.847 sor.http.w1.python.logger (D) Using Python executable {"path": "/usr/bin/python3"}
20.09.04 23:21:48.847 sor.http.w1.python.logger (D) Setting PYTHONPATH {"value": "PYTHONPATH=/opt/nuclio"}
20.09.04 23:21:48.847 sor.http.w0.python.logger (D) Using Python executable {"path": "/usr/bin/python3"}
20.09.04 23:21:48.847 sor.http.w1.python.logger (D) Running wrapper {"command": "/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --socket-path /tmp/nuclio-rpc-bt9co339v39n39jg38l0.sock --platform-kind local --namespace nuclio --worker-id 1 --trigger-kind http --trigger-name myHttpTrigger"}
20.09.04 23:21:48.847 sor.http.w0.python.logger (D) Setting PYTHONPATH {"value": "PYTHONPATH=/opt/nuclio"}
20.09.04 23:21:48.847 sor.http.w0.python.logger (D) Running wrapper {"command": "/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --socket-path /tmp/nuclio-rpc-bt9co339v39n39jg38lg.sock --platform-kind local --namespace nuclio --worker-id 0 --trigger-kind http --trigger-name myHttpTrigger"}
2020-09-04 23:21:50.408831: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory
2020-09-04 23:21:50.408957: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory
2020-09-04 23:21:50.408971: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2020-09-04 23:21:50.412591: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory
2020-09-04 23:21:50.412680: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory
2020-09-04 23:21:50.412696: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/compat/v2_compat.py:96: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
20.09.04 23:21:51.426 sor.http.w0.python.logger (I) Wrapper connected {"wid": 0}
20.09.04 23:21:51.426 sor.http.w0.python.logger (D) Waiting for start
l{"datetime": "2020-09-04 23:21:51,426", "level": "info", "message": "Init context...  0%", "with": {"worker_id": "0"}}
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/compat/v2_compat.py:96: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
20.09.04 23:21:51.433 sor.http.w1.python.logger (I) Wrapper connected {"wid": 1}
20.09.04 23:21:51.433 sor.http.w1.python.logger (D) Waiting for start
l{"datetime": "2020-09-04 23:21:51,433", "level": "info", "message": "Init context...  0%", "with": {"worker_id": "1"}}
2020-09-04 23:21:53.788851: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-09-04 23:21:53.798042: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2199995000 Hz
2020-09-04 23:21:53.799234: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x869ef80 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-09-04 23:21:53.799265: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-09-04 23:21:53.801616: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2020-09-04 23:21:53.801637: E tensorflow/stream_executor/cuda/cuda_driver.cc:351] failed call to cuInit: UNKNOWN ERROR (303)
2020-09-04 23:21:53.801665: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:163] no NVIDIA GPU device is present: /dev/nvidia0 does not exist
2020-09-04 23:21:53.825143: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-09-04 23:21:53.833935: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2199995000 Hz
2020-09-04 23:21:53.834752: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x8654090 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-09-04 23:21:53.834781: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-09-04 23:21:53.836733: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2020-09-04 23:21:53.836752: E tensorflow/stream_executor/cuda/cuda_driver.cc:351] failed call to cuInit: UNKNOWN ERROR (303)
2020-09-04 23:21:53.836778: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:163] no NVIDIA GPU device is present: /dev/nvidia0 does not exist
l{"datetime": "2020-09-04 23:21:53,857", "level": "info", "message": "Init context...100%", "with": {"worker_id": "0"}}
l{"datetime": "2020-09-04 23:21:53,857", "level": "info", "message": "Replacing logger output", "with": {"handler_name": "default", "worker_id": "0"}}
20.09.04 23:21:53.857 sor.http.w0.python.logger (D) Started
l{"datetime": "2020-09-04 23:21:53,890", "level": "info", "message": "Init context...100%", "with": {"worker_id": "1"}}
l{"datetime": "2020-09-04 23:21:53,890", "level": "info", "message": "Replacing logger output", "with": {"handler_name": "default", "worker_id": "1"}}
20.09.04 23:21:53.891 sor.http.w1.python.logger (D) Started
20.09.04 23:21:53.891                 processor (I) Starting event timeout watcher {"timeout": "30s"}
20.09.04 23:21:53.891 .webadmin.server.triggers (D) Registered custom route {"pattern": "/{id}/stats", "method": "GET"}
20.09.04 23:21:53.891 processor.webadmin.server (D) Registered resource {"name": "triggers"}
20.09.04 23:21:53.891                 processor (W) No metric sinks configured, metrics will not be published
20.09.04 23:21:53.891                 processor (D) Starting triggers {"triggers": [{"ID":"myHttpTrigger","Logger":{},"WorkerAllocator":{},"Class":"sync","Kind":"http","Name":"myHttpTrigger","Statistics":{"EventsHandledSuccessTotal":0,"EventsHandledFailureTotal":0,"WorkerAllocatorStatistics":{"WorkerAllocationCount":0,"WorkerAllocationSuccessImmediateTotal":0,"WorkerAllocationSuccessAfterWaitTotal":0,"WorkerAllocationTimeoutTotal":0,"WorkerAllocationWaitDurationMilliSecondsSum":0,"WorkerAllocationWorkersAvailablePercentage":0}},"Namespace":"nuclio","FunctionName":"tf.faster_rcnn_inception_v2_coco"}]}
20.09.04 23:21:53.891            processor.http (I) Starting {"listenAddress": ":8080", "readBufferSize": 16384, "maxRequestBodySize": 33554432, "cors": null}
20.09.04 23:21:53.891 processor.webadmin.server (I) Listening {"listenAddress": ":8081"}
20.09.04 23:21:53.891                 processor (D) Processor started

cognitiveRobot commented 4 years ago

@nmanovic, any update on this? Thanks.

cognitiveRobot commented 4 years ago

To resolve this, I installed a fresh ubuntu 18.04. Now, I am running into a new problem. I can't deploy the model. Env:

  Operating System: Ubuntu 18.04.5 LTS
            Kernel: Linux 5.4.0-45-generic
      Architecture: x86-64
Docker version 19.03.12, build 48a66213fe

When I run the deploying commad:

./nuctl deploy --project-name cvat --path cvat/serverless/openvino/omz/public/faster_rcnn_inception_v2_coco/nuclio/ --volume /mnt/hdd/projects//cvat/serverless/openvino/common:/opt/nuclio/common --platform local

20.09.08 15:23:39.423                     nuctl (I) Deploying function {"name": ""}
20.09.08 15:23:39.454                     nuctl (I) Building {"name": ""}
20.09.08 15:23:39.598                     nuctl (I) Cleaning up before deployment
20.09.08 15:23:39.652                     nuctl (I) Function already exists, deleting
20.09.08 15:23:39.683                     nuctl (I) Staging files and preparing base images
20.09.08 15:23:39.684                     nuctl (I) Pulling image {"imageName": "quay.io/nuclio/uhttpc:0.0.1-amd64"}
20.09.08 15:23:43.346                     nuctl (I) Pulling image {"imageName": "quay.io/nuclio/handler-builder-python-onbuild:1.1.37-amd64"}
20.09.08 15:23:47.352                     nuctl (I) Building processor image {"imageName": "cvat/openvino.omz.public.faster_rcnn_inception_v2_coco:latest"}
20.09.08 15:23:47.518                     nuctl (I) Build complete {"result": {"Image":"cvat/openvino.omz.public.faster_rcnn_inception_v2_coco:latest","UpdatedFunctionConfig":{"metadata":{"name":"openvino.omz.public.faster_rcnn_inception_v2_coco","namespace":"nuclio","labels":{"nuclio.io/project-name":"cvat"},"annotations":{"framework":"openvino","name":"Faster RCNN","spec":"[\n  { \"id\": 1, \"name\": \"person\" },\n  { \"id\": 2, \"name\": \"bicycle\" },\n  { \"id\": 3, \"name\": \"car\" },\n  { \"id\": 4, \"name\": \"motorcycle\" },\n  { \"id\": 5, \"name\": \"airplane\" },\n  { \"id\": 6, \"name\": \"bus\" },\n  { \"id\": 7, \"name\": \"train\" },\n  { \"id\": 8, \"name\": \"truck\" },\n  { \"id\": 9, \"name\": \"boat\" },\n  { \"id\":10, \"name\": \"traffic_light\" },\n  { \"id\":11, \"name\": \"fire_hydrant\" },\n  { \"id\":13, \"name\": \"stop_sign\" },\n  { \"id\":14, \"name\": \"parking_meter\" },\n  { \"id\":15, \"name\": \"bench\" },\n  { \"id\":16, \"name\": \"bird\" },\n  { \"id\":17, \"name\": \"cat\" },\n  { \"id\":18, \"name\": \"dog\" },\n  { \"id\":19, \"name\": \"horse\" },\n  { \"id\":20, \"name\": \"sheep\" },\n  { \"id\":21, \"name\": \"cow\" },\n  { \"id\":22, \"name\": \"elephant\" },\n  { \"id\":23, \"name\": \"bear\" },\n  { \"id\":24, \"name\": \"zebra\" },\n  { \"id\":25, \"name\": \"giraffe\" },\n  { \"id\":27, \"name\": \"backpack\" },\n  { \"id\":28, \"name\": \"umbrella\" },\n  { \"id\":31, \"name\": \"handbag\" },\n  { \"id\":32, \"name\": \"tie\" },\n  { \"id\":33, \"name\": \"suitcase\" },\n  { \"id\":34, \"name\": \"frisbee\" },\n  { \"id\":35, \"name\": \"skis\" },\n  { \"id\":36, \"name\": \"snowboard\" },\n  { \"id\":37, \"name\": \"sports_ball\" },\n  { \"id\":38, \"name\": \"kite\" },\n  { \"id\":39, \"name\": \"baseball_bat\" },\n  { \"id\":40, \"name\": \"baseball_glove\" },\n  { \"id\":41, \"name\": \"skateboard\" },\n  { \"id\":42, \"name\": \"surfboard\" },\n  { \"id\":43, \"name\": \"tennis_racket\" },\n  { \"id\":44, \"name\": \"bottle\" },\n  { \"id\":46, \"name\": \"wine_glass\" },\n  { \"id\":47, \"name\": \"cup\" },\n  { \"id\":48, \"name\": \"fork\" },\n  { \"id\":49, \"name\": \"knife\" },\n  { \"id\":50, \"name\": \"spoon\" },\n  { \"id\":51, \"name\": \"bowl\" },\n  { \"id\":52, \"name\": \"banana\" },\n  { \"id\":53, \"name\": \"apple\" },\n  { \"id\":54, \"name\": \"sandwich\" },\n  { \"id\":55, \"name\": \"orange\" },\n  { \"id\":56, \"name\": \"broccoli\" },\n  { \"id\":57, \"name\": \"carrot\" },\n  { \"id\":58, \"name\": \"hot_dog\" },\n  { \"id\":59, \"name\": \"pizza\" },\n  { \"id\":60, \"name\": \"donut\" },\n  { \"id\":61, \"name\": \"cake\" },\n  { \"id\":62, \"name\": \"chair\" },\n  { \"id\":63, \"name\": \"couch\" },\n  { \"id\":64, \"name\": \"potted_plant\" },\n  { \"id\":65, \"name\": \"bed\" },\n  { \"id\":67, \"name\": \"dining_table\" },\n  { \"id\":70, \"name\": \"toilet\" },\n  { \"id\":72, \"name\": \"tv\" },\n  { \"id\":73, \"name\": \"laptop\" },\n  { \"id\":74, \"name\": \"mouse\" },\n  { \"id\":75, \"name\": \"remote\" },\n  { \"id\":76, \"name\": \"keyboard\" },\n  { \"id\":77, \"name\": \"cell_phone\" },\n  { \"id\":78, \"name\": \"microwave\" },\n  { \"id\":79, \"name\": \"oven\" },\n  { \"id\":80, \"name\": \"toaster\" },\n  { \"id\":81, \"name\": \"sink\" },\n  { \"id\":83, \"name\": \"refrigerator\" },\n  { \"id\":84, \"name\": \"book\" },\n  { \"id\":85, \"name\": \"clock\" },\n  { \"id\":86, \"name\": \"vase\" },\n  { \"id\":87, \"name\": \"scissors\" },\n  { \"id\":88, \"name\": \"teddy_bear\" },\n  { \"id\":89, \"name\": \"hair_drier\" },\n  { \"id\":90, \"name\": \"toothbrush\" }\n]\n","type":"detector"}},"spec":{"description":"Faster RCNN inception v2 COCO via Intel OpenVINO toolkit","handler":"main:handler","runtime":"python:3.6","env":[{"name":"NUCLIO_PYTHON_EXE_PATH","value":"/opt/nuclio/common/python3"}],"resources":{},"targetCPU":75,"triggers":{"myHttpTrigger":{"class":"","kind":"http","name":"","maxWorkers":2,"workerAvailabilityTimeoutMilliseconds":10000,"attributes":{"maxRequestBodySize":33554432}}},"volumes":[{"volume":{"name":"volume-1","hostPath":{"path":"/mnt/hdd/projects//cvat/serverless/openvino/common"}},"volumeMount":{"name":"volume-1","mountPath":"/opt/nuclio/common"}}],"build":{"path":"cvat/serverless/openvino/omz/public/faster_rcnn_inception_v2_coco/nuclio/","image":"cvat/openvino.omz.public.faster_rcnn_inception_v2_coco","baseImage":"openvino/ubuntu18_dev:2020.2","directives":{"preCopy":[{"kind":"USER","value":"root"},{"kind":"WORKDIR","value":"/opt/nuclio"},{"kind":"RUN","value":"ln -s /usr/bin/pip3 /usr/bin/pip"},{"kind":"RUN","value":"/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/downloader.py --name faster_rcnn_inception_v2_coco -o /opt/nuclio/open_model_zoo"},{"kind":"RUN","value":"/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name faster_rcnn_inception_v2_coco --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo"}]}},"platform":{"attributes":{"restartPolicy":{"maximumRetryCount":3,"name":"always"}}},"readinessTimeoutSeconds":60,"eventTimeout":"30s"}}}}
20.09.08 15:23:47.923            nuctl.platform (I) Waiting for function to be ready {"timeout": 60}
20.09.08 15:24:47.951     nuctl.platform.docker (W) Container wasn't healthy within timeout {"timeout": "1m0s", "logs": "20.09.08 03:23:47.966 \u001b[37m                processor\u001b[0m \u001b[32m(D)\u001b[0m Read configuration {\"config\": \"{\\n    \\\"metadata\\\": {\\n        \\\"name\\\": \\\"openvino.omz.public.faster_rcnn_inception_v2_coco\\\",\\n        \\\"namespace\\\": \\\"nuclio\\\",\\n        \\\"labels\\\": {\\n            \\\"nuclio.io/project-name\\\": \\\"cvat\\\"\\n        },\\n        \\\"annotations\\\": {\\n            \\\"framework\\\": \\\"openvino\\\",\\n            \\\"name\\\": \\\"Faster RCNN\\\",\\n            \\\"spec\\\": \\\"[\\\\n  { \\\\\\\"id\\\\\\\": 1, \\\\\\\"name\\\\\\\": \\\\\\\"person\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 2, \\\\\\\"name\\\\\\\": \\\\\\\"bicycle\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 3, \\\\\\\"name\\\\\\\": \\\\\\\"car\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 4, \\\\\\\"name\\\\\\\": \\\\\\\"motorcycle\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 5, \\\\\\\"name\\\\\\\": \\\\\\\"airplane\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 6, \\\\\\\"name\\\\\\\": \\\\\\\"bus\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 7, \\\\\\\"name\\\\\\\": \\\\\\\"train\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 8, \\\\\\\"name\\\\\\\": \\\\\\\"truck\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 9, \\\\\\\"name\\\\\\\": \\\\\\\"boat\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":10, \\\\\\\"name\\\\\\\": \\\\\\\"traffic_light\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":11, \\\\\\\"name\\\\\\\": \\\\\\\"fire_hydrant\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":13, \\\\\\\"name\\\\\\\": \\\\\\\"stop_sign\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":14, \\\\\\\"name\\\\\\\": \\\\\\\"parking_meter\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":15, \\\\\\\"name\\\\\\\": \\\\\\\"bench\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":16, \\\\\\\"name\\\\\\\": \\\\\\\"bird\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":17, \\\\\\\"name\\\\\\\": \\\\\\\"cat\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":18, \\\\\\\"name\\\\\\\": \\\\\\\"dog\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":19, \\\\\\\"name\\\\\\\": \\\\\\\"horse\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":20, \\\\\\\"name\\\\\\\": \\\\\\\"sheep\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":21, \\\\\\\"name\\\\\\\": \\\\\\\"cow\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":22, \\\\\\\"name\\\\\\\": \\\\\\\"elephant\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":23, \\\\\\\"name\\\\\\\": \\\\\\\"bear\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":24, \\\\\\\"name\\\\\\\": \\\\\\\"zebra\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":25, \\\\\\\"name\\\\\\\": \\\\\\\"giraffe\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":27, \\\\\\\"name\\\\\\\": \\\\\\\"backpack\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":28, \\\\\\\"name\\\\\\\": \\\\\\\"umbrella\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":31, \\\\\\\"name\\\\\\\": \\\\\\\"handbag\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":32, \\\\\\\"name\\\\\\\": \\\\\\\"tie\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":33, \\\\\\\"name\\\\\\\": \\\\\\\"suitcase\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":34, \\\\\\\"name\\\\\\\": \\\\\\\"frisbee\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":35, \\\\\\\"name\\\\\\\": \\\\\\\"skis\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":36, \\\\\\\"name\\\\\\\": \\\\\\\"snowboard\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":37, \\\\\\\"name\\\\\\\": \\\\\\\"sports_ball\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":38, \\\\\\\"name\\\\\\\": \\\\\\\"kite\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":39, \\\\\\\"name\\\\\\\": \\\\\\\"baseball_bat\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":40, \\\\\\\"name\\\\\\\": \\\\\\\"baseball_glove\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":41, \\\\\\\"name\\\\\\\": \\\\\\\"skateboard\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":42, \\\\\\\"name\\\\\\\": \\\\\\\"surfboard\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":43, \\\\\\\"name\\\\\\\": \\\\\\\"tennis_racket\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":44, \\\\\\\"name\\\\\\\": \\\\\\\"bottle\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":46, \\\\\\\"name\\\\\\\": \\\\\\\"wine_glass\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":47, \\\\\\\"name\\\\\\\": \\\\\\\"cup\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":48, \\\\\\\"name\\\\\\\": \\\\\\\"fork\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":49, \\\\\\\"name\\\\\\\": \\\\\\\"knife\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":50, \\\\\\\"name\\\\\\\": \\\\\\\"spoon\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":51, \\\\\\\"name\\\\\\\": \\\\\\\"bowl\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":52, \\\\\\\"name\\\\\\\": \\\\\\\"banana\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":53, \\\\\\\"name\\\\\\\": \\\\\\\"apple\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":54, \\\\\\\"name\\\\\\\": \\\\\\\"sandwich\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":55, \\\\\\\"name\\\\\\\": \\\\\\\"orange\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":56, \\\\\\\"name\\\\\\\": \\\\\\\"broccoli\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":57, \\\\\\\"name\\\\\\\": \\\\\\\"carrot\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":58, \\\\\\\"name\\\\\\\": \\\\\\\"hot_dog\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":59, \\\\\\\"name\\\\\\\": \\\\\\\"pizza\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":60, \\\\\\\"name\\\\\\\": \\\\\\\"donut\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":61, \\\\\\\"name\\\\\\\": \\\\\\\"cake\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":62, \\\\\\\"name\\\\\\\": \\\\\\\"chair\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":63, \\\\\\\"name\\\\\\\": \\\\\\\"couch\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":64, \\\\\\\"name\\\\\\\": \\\\\\\"potted_plant\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":65, \\\\\\\"name\\\\\\\": \\\\\\\"bed\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":67, \\\\\\\"name\\\\\\\": \\\\\\\"dining_table\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":70, \\\\\\\"name\\\\\\\": \\\\\\\"toilet\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":72, \\\\\\\"name\\\\\\\": \\\\\\\"tv\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":73, \\\\\\\"name\\\\\\\": \\\\\\\"laptop\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":74, \\\\\\\"name\\\\\\\": \\\\\\\"mouse\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":75, \\\\\\\"name\\\\\\\": \\\\\\\"remote\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":76, \\\\\\\"name\\\\\\\": \\\\\\\"keyboard\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":77, \\\\\\\"name\\\\\\\": \\\\\\\"cell_phone\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":78, \\\\\\\"name\\\\\\\": \\\\\\\"microwave\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":79, \\\\\\\"name\\\\\\\": \\\\\\\"oven\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":80, \\\\\\\"name\\\\\\\": \\\\\\\"toaster\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":81, \\\\\\\"name\\\\\\\": \\\\\\\"sink\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":83, \\\\\\\"name\\\\\\\": \\\\\\\"refrigerator\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":84, \\\\\\\"name\\\\\\\": \\\\\\\"book\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":85, \\\\\\\"name\\\\\\\": \\\\\\\"clock\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":86, \\\\\\\"name\\\\\\\": \\\\\\\"vase\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":87, \\\\\\\"name\\\\\\\": \\\\\\\"scissors\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":88, \\\\\\\"name\\\\\\\": \\\\\\\"teddy_bear\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":89, \\\\\\\"name\\\\\\\": \\\\\\\"hair_drier\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":90, \\\\\\\"name\\\\\\\": \\\\\\\"toothbrush\\\\\\\" }\\\\n]\\\\n\\\",\\n            \\\"type\\\": \\\"detector\\\"\\n        }\\n    },\\n    \\\"spec\\\": {\\n        \\\"description\\\": \\\"Faster RCNN inception v2 COCO via Intel OpenVINO toolkit\\\",\\n        \\\"handler\\\": \\\"main:handler\\\",\\n        \\\"runtime\\\": \\\"python:3.6\\\",\\n        \\\"env\\\": [\\n            {\\n                \\\"name\\\": \\\"NUCLIO_PYTHON_EXE_PATH\\\",\\n                \\\"value\\\": \\\"/opt/nuclio/common/python3\\\"\\n            }\\n        ],\\n        \\\"resources\\\": {},\\n        \\\"image\\\": \\\"cvat/openvino.omz.public.faster_rcnn_inception_v2_coco:latest\\\",\\n        \\\"targetCPU\\\": 75,\\n        \\\"triggers\\\": {\\n            \\\"myHttpTrigger\\\": {\\n                \\\"class\\\": \\\"\\\",\\n                \\\"kind\\\": \\\"http\\\",\\n                \\\"name\\\": \\\"\\\",\\n                \\\"maxWorkers\\\": 2,\\n                \\\"workerAvailabilityTimeoutMilliseconds\\\": 10000,\\n                \\\"attributes\\\": {\\n                    \\\"maxRequestBodySize\\\": 33554432\\n                }\\n            }\\n        },\\n        \\\"volumes\\\": [\\n            {\\n                \\\"volume\\\": {\\n                    \\\"name\\\": \\\"volume-1\\\",\\n                    \\\"hostPath\\\": {\\n                        \\\"path\\\": \\\"/mnt/hdd/projects//cvat/serverless/openvino/common\\\"\\n                    }\\n                },\\n                \\\"volumeMount\\\": {\\n                    \\\"name\\\": \\\"volume-1\\\",\\n                    \\\"mountPath\\\": \\\"/opt/nuclio/common\\\"\\n                }\\n            }\\n        ],\\n        \\\"build\\\": {\\n            \\\"path\\\": \\\"cvat/serverless/openvino/omz/public/faster_rcnn_inception_v2_coco/nuclio/\\\",\\n            \\\"image\\\": \\\"cvat/openvino.omz.public.faster_rcnn_inception_v2_coco\\\",\\n            \\\"baseImage\\\": \\\"openvino/ubuntu18_dev:2020.2\\\",\\n            \\\"directives\\\": {\\n                \\\"preCopy\\\": [\\n                    {\\n                        \\\"kind\\\": \\\"USER\\\",\\n                        \\\"value\\\": \\\"root\\\"\\n                    },\\n                    {\\n                        \\\"kind\\\": \\\"WORKDIR\\\",\\n                        \\\"value\\\": \\\"/opt/nuclio\\\"\\n                    },\\n                    {\\n                        \\\"kind\\\": \\\"RUN\\\",\\n                        \\\"value\\\": \\\"ln -s /usr/bin/pip3 /usr/bin/pip\\\"\\n                    },\\n                    {\\n                        \\\"kind\\\": \\\"RUN\\\",\\n                        \\\"value\\\": \\\"/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/downloader.py --name faster_rcnn_inception_v2_coco -o /opt/nuclio/open_model_zoo\\\"\\n                    },\\n                    {\\n                        \\\"kind\\\": \\\"RUN\\\",\\n                        \\\"value\\\": \\\"/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name faster_rcnn_inception_v2_coco --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo\\\"\\n                    }\\n                ]\\n            },\\n            \\\"timestamp\\\": 1599535427\\n        },\\n        \\\"platform\\\": {\\n            \\\"attributes\\\": {\\n                \\\"restartPolicy\\\": {\\n                    \\\"maximumRetryCount\\\": 3,\\n                    \\\"name\\\": \\\"always\\\"\\n                }\\n            }\\n        },\\n        \\\"readinessTimeoutSeconds\\\": 60,\\n        \\\"eventTimeout\\\": \\\"30s\\\"\\n    },\\n    \\\"PlatformConfig\\\": null\\n}\", \"platformConfig\": \"{\\n    \\\"kind\\\": \\\"local\\\",\\n    \\\"webAdmin\\\": {\\n        \\\"enabled\\\": true,\\n        \\\"listenAddress\\\": \\\":8081\\\"\\n    },\\n    \\\"healthCheck\\\": {\\n        \\\"enabled\\\": true,\\n        \\\"listenAddress\\\": \\\":8082\\\"\\n    },\\n    \\\"logger\\\": {\\n        \\\"sinks\\\": {\\n            \\\"stdout\\\": {\\n                \\\"kind\\\": \\\"stdout\\\"\\n            }\\n        },\\n        \\\"system\\\": [\\n            {\\n                \\\"level\\\": \\\"debug\\\",\\n                \\\"sink\\\": \\\"stdout\\\"\\n            }\\n        ],\\n        \\\"functions\\\": [\\n            {\\n                \\\"level\\\": \\\"debug\\\",\\n                \\\"sink\\\": \\\"stdout\\\"\\n            }\\n        ]\\n    },\\n    \\\"metrics\\\": {},\\n    \\\"scaleToZero\\\": {},\\n    \\\"autoScale\\\": {}\\n}\"}\n20.09.08 03:23:47.966 \u001b[37mcessor.healthcheck.server\u001b[0m \u001b[34m(I)\u001b[0m Listening {\"listenAddress\": \":8082\"}\n20.09.08 03:23:47.966 \u001b[37m           processor.http\u001b[0m \u001b[32m(D)\u001b[0m Creating worker pool {\"num\": 2}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Creating listener socket {\"path\": \"/tmp/nuclio-rpc-btbfigu11teg00f8q550.sock\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Creating listener socket {\"path\": \"/tmp/nuclio-rpc-btbfigu11teg00f8q55g.sock\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python wrapper script path {\"path\": \"/opt/nuclio/_nuclio_wrapper.py\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python handler {\"handler\": \"main:handler\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python wrapper script path {\"path\": \"/opt/nuclio/_nuclio_wrapper.py\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python handler {\"handler\": \"main:handler\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python executable {\"path\": \"/usr/bin/python3\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python executable {\"path\": \"/usr/bin/python3\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Setting PYTHONPATH {\"value\": \"PYTHONPATH=/opt/nuclio\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Setting PYTHONPATH {\"value\": \"PYTHONPATH=/opt/nuclio\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Running wrapper {\"command\": \"/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --socket-path /tmp/nuclio-rpc-btbfigu11teg00f8q550.sock --platform-kind local --namespace nuclio --worker-id 1 --trigger-kind http --trigger-name myHttpTrigger\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Running wrapper {\"command\": \"/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --socket-path /tmp/nuclio-rpc-btbfigu11teg00f8q55g.sock --platform-kind local --namespace nuclio --worker-id 0 --trigger-kind http --trigger-name myHttpTrigger\"}\nl{\"datetime\": \"2020-09-08 03:23:48,039\", \"level\": \"error\", \"message\": \"Caught unhandled exception while initializing\", \"with\": {\"err\": \"No module named 'model_loader'\", \"traceback\": \"Traceback (most recent call last):\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 286, in run_wrapper\\n    args.trigger_name)\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 51, in __init__\\n    self._entrypoint = self._load_entrypoint_from_handler(handler)\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 190, in _load_entrypoint_from_handler\\n    module = __import__(module_name)\\n  File \\\"/opt/nuclio/main.py\\\", line 5, in <module>\\n    from model_handler import ModelHandler\\n  File \\\"/opt/nuclio/model_handler.py\\\", line 6, in <module>\\n    from model_loader import ModelLoader\\nModuleNotFoundError: No module named 'model_loader'\\n\"}}\nl{\"datetime\": \"2020-09-08 03:23:48,039\", \"level\": \"error\", \"message\": \"Caught unhandled exception while initializing\", \"with\": {\"err\": \"No module named 'model_loader'\", \"traceback\": \"Traceback (most recent call last):\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 286, in run_wrapper\\n    args.trigger_name)\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 51, in __init__\\n    self._entrypoint = self._load_entrypoint_from_handler(handler)\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 190, in _load_entrypoint_from_handler\\n    module = __import__(module_name)\\n  File \\\"/opt/nuclio/main.py\\\", line 5, in <module>\\n    from model_handler import ModelHandler\\n  File \\\"/opt/nuclio/model_handler.py\\\", line 6, in <module>\\n    from model_loader import ModelLoader\\nModuleNotFoundError: No module named 'model_loader'\\n\"}}\n20.09.08 03:23:48.046 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[31m(E)\u001b[0m Unexpected termination of child process {\"error\": null, \"status\": \"exit status 1\"}\n20.09.08 03:23:48.046 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[31m(E)\u001b[0m Unexpected termination of child process {\"error\": null, \"status\": \"exit status 1\"}\npanic: Wrapper process for worker 0 exited unexpectedly with: exit status 1\n\ngoroutine 53 [running]:\ngithub.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).watchWrapperProcess(0xc420363130)\n\t/go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:453 +0x5bb\ncreated by github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).startWrapper\n\t/go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:232 +0x1c8\npanic: Wrapper process for worker 1 exited unexpectedly with: exit status 1\n\ngoroutine 102 [running]:\ngithub.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).watchWrapperProcess(0xc42053e210)\n\t/go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:453 +0x5bb\ncreated by github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).startWrapper\n\t/go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:232 +0x1c8\n"}
20.09.08 15:24:47.979                     nuctl (W) Create function failed, setting function status {"err": "Function wasn't ready in time. Logs:\n20.09.08 03:23:47.966 \u001b[37m                processor\u001b[0m \u001b[32m(D)\u001b[0m Read configuration {\"config\": \"{\\n    \\\"metadata\\\": {\\n        \\\"name\\\": \\\"openvino.omz.public.faster_rcnn_inception_v2_coco\\\",\\n        \\\"namespace\\\": \\\"nuclio\\\",\\n        \\\"labels\\\": {\\n            \\\"nuclio.io/project-name\\\": \\\"cvat\\\"\\n        },\\n        \\\"annotations\\\": {\\n            \\\"framework\\\": \\\"openvino\\\",\\n            \\\"name\\\": \\\"Faster RCNN\\\",\\n            \\\"spec\\\": \\\"[\\\\n  { \\\\\\\"id\\\\\\\": 1, \\\\\\\"name\\\\\\\": \\\\\\\"person\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 2, \\\\\\\"name\\\\\\\": \\\\\\\"bicycle\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 3, \\\\\\\"name\\\\\\\": \\\\\\\"car\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 4, \\\\\\\"name\\\\\\\": \\\\\\\"motorcycle\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 5, \\\\\\\"name\\\\\\\": \\\\\\\"airplane\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 6, \\\\\\\"name\\\\\\\": \\\\\\\"bus\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 7, \\\\\\\"name\\\\\\\": \\\\\\\"train\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 8, \\\\\\\"name\\\\\\\": \\\\\\\"truck\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 9, \\\\\\\"name\\\\\\\": \\\\\\\"boat\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":10, \\\\\\\"name\\\\\\\": \\\\\\\"traffic_light\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":11, \\\\\\\"name\\\\\\\": \\\\\\\"fire_hydrant\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":13, \\\\\\\"name\\\\\\\": \\\\\\\"stop_sign\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":14, \\\\\\\"name\\\\\\\": \\\\\\\"parking_meter\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":15, \\\\\\\"name\\\\\\\": \\\\\\\"bench\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":16, \\\\\\\"name\\\\\\\": \\\\\\\"bird\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":17, \\\\\\\"name\\\\\\\": \\\\\\\"cat\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":18, \\\\\\\"name\\\\\\\": \\\\\\\"dog\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":19, \\\\\\\"name\\\\\\\": \\\\\\\"horse\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":20, \\\\\\\"name\\\\\\\": \\\\\\\"sheep\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":21, \\\\\\\"name\\\\\\\": \\\\\\\"cow\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":22, \\\\\\\"name\\\\\\\": \\\\\\\"elephant\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":23, \\\\\\\"name\\\\\\\": \\\\\\\"bear\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":24, \\\\\\\"name\\\\\\\": \\\\\\\"zebra\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":25, \\\\\\\"name\\\\\\\": \\\\\\\"giraffe\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":27, \\\\\\\"name\\\\\\\": \\\\\\\"backpack\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":28, \\\\\\\"name\\\\\\\": \\\\\\\"umbrella\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":31, \\\\\\\"name\\\\\\\": \\\\\\\"handbag\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":32, \\\\\\\"name\\\\\\\": \\\\\\\"tie\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":33, \\\\\\\"name\\\\\\\": \\\\\\\"suitcase\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":34, \\\\\\\"name\\\\\\\": \\\\\\\"frisbee\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":35, \\\\\\\"name\\\\\\\": \\\\\\\"skis\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":36, \\\\\\\"name\\\\\\\": \\\\\\\"snowboard\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":37, \\\\\\\"name\\\\\\\": \\\\\\\"sports_ball\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":38, \\\\\\\"name\\\\\\\": \\\\\\\"kite\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":39, \\\\\\\"name\\\\\\\": \\\\\\\"baseball_bat\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":40, \\\\\\\"name\\\\\\\": \\\\\\\"baseball_glove\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":41, \\\\\\\"name\\\\\\\": \\\\\\\"skateboard\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":42, \\\\\\\"name\\\\\\\": \\\\\\\"surfboard\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":43, \\\\\\\"name\\\\\\\": \\\\\\\"tennis_racket\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":44, \\\\\\\"name\\\\\\\": \\\\\\\"bottle\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":46, \\\\\\\"name\\\\\\\": \\\\\\\"wine_glass\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":47, \\\\\\\"name\\\\\\\": \\\\\\\"cup\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":48, \\\\\\\"name\\\\\\\": \\\\\\\"fork\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":49, \\\\\\\"name\\\\\\\": \\\\\\\"knife\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":50, \\\\\\\"name\\\\\\\": \\\\\\\"spoon\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":51, \\\\\\\"name\\\\\\\": \\\\\\\"bowl\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":52, \\\\\\\"name\\\\\\\": \\\\\\\"banana\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":53, \\\\\\\"name\\\\\\\": \\\\\\\"apple\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":54, \\\\\\\"name\\\\\\\": \\\\\\\"sandwich\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":55, \\\\\\\"name\\\\\\\": \\\\\\\"orange\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":56, \\\\\\\"name\\\\\\\": \\\\\\\"broccoli\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":57, \\\\\\\"name\\\\\\\": \\\\\\\"carrot\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":58, \\\\\\\"name\\\\\\\": \\\\\\\"hot_dog\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":59, \\\\\\\"name\\\\\\\": \\\\\\\"pizza\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":60, \\\\\\\"name\\\\\\\": \\\\\\\"donut\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":61, \\\\\\\"name\\\\\\\": \\\\\\\"cake\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":62, \\\\\\\"name\\\\\\\": \\\\\\\"chair\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":63, \\\\\\\"name\\\\\\\": \\\\\\\"couch\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":64, \\\\\\\"name\\\\\\\": \\\\\\\"potted_plant\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":65, \\\\\\\"name\\\\\\\": \\\\\\\"bed\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":67, \\\\\\\"name\\\\\\\": \\\\\\\"dining_table\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":70, \\\\\\\"name\\\\\\\": \\\\\\\"toilet\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":72, \\\\\\\"name\\\\\\\": \\\\\\\"tv\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":73, \\\\\\\"name\\\\\\\": \\\\\\\"laptop\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":74, \\\\\\\"name\\\\\\\": \\\\\\\"mouse\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":75, \\\\\\\"name\\\\\\\": \\\\\\\"remote\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":76, \\\\\\\"name\\\\\\\": \\\\\\\"keyboard\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":77, \\\\\\\"name\\\\\\\": \\\\\\\"cell_phone\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":78, \\\\\\\"name\\\\\\\": \\\\\\\"microwave\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":79, \\\\\\\"name\\\\\\\": \\\\\\\"oven\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":80, \\\\\\\"name\\\\\\\": \\\\\\\"toaster\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":81, \\\\\\\"name\\\\\\\": \\\\\\\"sink\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":83, \\\\\\\"name\\\\\\\": \\\\\\\"refrigerator\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":84, \\\\\\\"name\\\\\\\": \\\\\\\"book\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":85, \\\\\\\"name\\\\\\\": \\\\\\\"clock\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":86, \\\\\\\"name\\\\\\\": \\\\\\\"vase\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":87, \\\\\\\"name\\\\\\\": \\\\\\\"scissors\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":88, \\\\\\\"name\\\\\\\": \\\\\\\"teddy_bear\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":89, \\\\\\\"name\\\\\\\": \\\\\\\"hair_drier\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":90, \\\\\\\"name\\\\\\\": \\\\\\\"toothbrush\\\\\\\" }\\\\n]\\\\n\\\",\\n            \\\"type\\\": \\\"detector\\\"\\n        }\\n    },\\n    \\\"spec\\\": {\\n        \\\"description\\\": \\\"Faster RCNN inception v2 COCO via Intel OpenVINO toolkit\\\",\\n        \\\"handler\\\": \\\"main:handler\\\",\\n        \\\"runtime\\\": \\\"python:3.6\\\",\\n        \\\"env\\\": [\\n            {\\n                \\\"name\\\": \\\"NUCLIO_PYTHON_EXE_PATH\\\",\\n                \\\"value\\\": \\\"/opt/nuclio/common/python3\\\"\\n            }\\n        ],\\n        \\\"resources\\\": {},\\n        \\\"image\\\": \\\"cvat/openvino.omz.public.faster_rcnn_inception_v2_coco:latest\\\",\\n        \\\"targetCPU\\\": 75,\\n        \\\"triggers\\\": {\\n            \\\"myHttpTrigger\\\": {\\n                \\\"class\\\": \\\"\\\",\\n                \\\"kind\\\": \\\"http\\\",\\n                \\\"name\\\": \\\"\\\",\\n                \\\"maxWorkers\\\": 2,\\n                \\\"workerAvailabilityTimeoutMilliseconds\\\": 10000,\\n                \\\"attributes\\\": {\\n                    \\\"maxRequestBodySize\\\": 33554432\\n                }\\n            }\\n        },\\n        \\\"volumes\\\": [\\n            {\\n                \\\"volume\\\": {\\n                    \\\"name\\\": \\\"volume-1\\\",\\n                    \\\"hostPath\\\": {\\n                        \\\"path\\\": \\\"/mnt/hdd/projects//cvat/serverless/openvino/common\\\"\\n                    }\\n                },\\n                \\\"volumeMount\\\": {\\n                    \\\"name\\\": \\\"volume-1\\\",\\n                    \\\"mountPath\\\": \\\"/opt/nuclio/common\\\"\\n                }\\n            }\\n        ],\\n        \\\"build\\\": {\\n            \\\"path\\\": \\\"cvat/serverless/openvino/omz/public/faster_rcnn_inception_v2_coco/nuclio/\\\",\\n            \\\"image\\\": \\\"cvat/openvino.omz.public.faster_rcnn_inception_v2_coco\\\",\\n            \\\"baseImage\\\": \\\"openvino/ubuntu18_dev:2020.2\\\",\\n            \\\"directives\\\": {\\n                \\\"preCopy\\\": [\\n                    {\\n                        \\\"kind\\\": \\\"USER\\\",\\n                        \\\"value\\\": \\\"root\\\"\\n                    },\\n                    {\\n                        \\\"kind\\\": \\\"WORKDIR\\\",\\n                        \\\"value\\\": \\\"/opt/nuclio\\\"\\n                    },\\n                    {\\n                        \\\"kind\\\": \\\"RUN\\\",\\n                        \\\"value\\\": \\\"ln -s /usr/bin/pip3 /usr/bin/pip\\\"\\n                    },\\n                    {\\n                        \\\"kind\\\": \\\"RUN\\\",\\n                        \\\"value\\\": \\\"/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/downloader.py --name faster_rcnn_inception_v2_coco -o /opt/nuclio/open_model_zoo\\\"\\n                    },\\n                    {\\n                        \\\"kind\\\": \\\"RUN\\\",\\n                        \\\"value\\\": \\\"/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name faster_rcnn_inception_v2_coco --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo\\\"\\n                    }\\n                ]\\n            },\\n            \\\"timestamp\\\": 1599535427\\n        },\\n        \\\"platform\\\": {\\n            \\\"attributes\\\": {\\n                \\\"restartPolicy\\\": {\\n                    \\\"maximumRetryCount\\\": 3,\\n                    \\\"name\\\": \\\"always\\\"\\n                }\\n            }\\n        },\\n        \\\"readinessTimeoutSeconds\\\": 60,\\n        \\\"eventTimeout\\\": \\\"30s\\\"\\n    },\\n    \\\"PlatformConfig\\\": null\\n}\", \"platformConfig\": \"{\\n    \\\"kind\\\": \\\"local\\\",\\n    \\\"webAdmin\\\": {\\n        \\\"enabled\\\": true,\\n        \\\"listenAddress\\\": \\\":8081\\\"\\n    },\\n    \\\"healthCheck\\\": {\\n        \\\"enabled\\\": true,\\n        \\\"listenAddress\\\": \\\":8082\\\"\\n    },\\n    \\\"logger\\\": {\\n        \\\"sinks\\\": {\\n            \\\"stdout\\\": {\\n                \\\"kind\\\": \\\"stdout\\\"\\n            }\\n        },\\n        \\\"system\\\": [\\n            {\\n                \\\"level\\\": \\\"debug\\\",\\n                \\\"sink\\\": \\\"stdout\\\"\\n            }\\n        ],\\n        \\\"functions\\\": [\\n            {\\n                \\\"level\\\": \\\"debug\\\",\\n                \\\"sink\\\": \\\"stdout\\\"\\n            }\\n        ]\\n    },\\n    \\\"metrics\\\": {},\\n    \\\"scaleToZero\\\": {},\\n    \\\"autoScale\\\": {}\\n}\"}\n20.09.08 03:23:47.966 \u001b[37mcessor.healthcheck.server\u001b[0m \u001b[34m(I)\u001b[0m Listening {\"listenAddress\": \":8082\"}\n20.09.08 03:23:47.966 \u001b[37m           processor.http\u001b[0m \u001b[32m(D)\u001b[0m Creating worker pool {\"num\": 2}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Creating listener socket {\"path\": \"/tmp/nuclio-rpc-btbfigu11teg00f8q550.sock\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Creating listener socket {\"path\": \"/tmp/nuclio-rpc-btbfigu11teg00f8q55g.sock\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python wrapper script path {\"path\": \"/opt/nuclio/_nuclio_wrapper.py\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python handler {\"handler\": \"main:handler\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python wrapper script path {\"path\": \"/opt/nuclio/_nuclio_wrapper.py\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python handler {\"handler\": \"main:handler\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python executable {\"path\": \"/usr/bin/python3\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python executable {\"path\": \"/usr/bin/python3\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Setting PYTHONPATH {\"value\": \"PYTHONPATH=/opt/nuclio\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Setting PYTHONPATH {\"value\": \"PYTHONPATH=/opt/nuclio\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Running wrapper {\"command\": \"/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --socket-path /tmp/nuclio-rpc-btbfigu11teg00f8q550.sock --platform-kind local --namespace nuclio --worker-id 1 --trigger-kind http --trigger-name myHttpTrigger\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Running wrapper {\"command\": \"/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --socket-path /tmp/nuclio-rpc-btbfigu11teg00f8q55g.sock --platform-kind local --namespace nuclio --worker-id 0 --trigger-kind http --trigger-name myHttpTrigger\"}\nl{\"datetime\": \"2020-09-08 03:23:48,039\", \"level\": \"error\", \"message\": \"Caught unhandled exception while initializing\", \"with\": {\"err\": \"No module named 'model_loader'\", \"traceback\": \"Traceback (most recent call last):\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 286, in run_wrapper\\n    args.trigger_name)\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 51, in __init__\\n    self._entrypoint = self._load_entrypoint_from_handler(handler)\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 190, in _load_entrypoint_from_handler\\n    module = __import__(module_name)\\n  File \\\"/opt/nuclio/main.py\\\", line 5, in <module>\\n    from model_handler import ModelHandler\\n  File \\\"/opt/nuclio/model_handler.py\\\", line 6, in <module>\\n    from model_loader import ModelLoader\\nModuleNotFoundError: No module named 'model_loader'\\n\"}}\nl{\"datetime\": \"2020-09-08 03:23:48,039\", \"level\": \"error\", \"message\": \"Caught unhandled exception while initializing\", \"with\": {\"err\": \"No module named 'model_loader'\", \"traceback\": \"Traceback (most recent call last):\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 286, in run_wrapper\\n    args.trigger_name)\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 51, in __init__\\n    self._entrypoint = self._load_entrypoint_from_handler(handler)\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 190, in _load_entrypoint_from_handler\\n    module = __import__(module_name)\\n  File \\\"/opt/nuclio/main.py\\\", line 5, in <module>\\n    from model_handler import ModelHandler\\n  File \\\"/opt/nuclio/model_handler.py\\\", line 6, in <module>\\n    from model_loader import ModelLoader\\nModuleNotFoundError: No module named 'model_loader'\\n\"}}\n20.09.08 03:23:48.046 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[31m(E)\u001b[0m Unexpected termination of child process {\"error\": null, \"status\": \"exit status 1\"}\n20.09.08 03:23:48.046 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[31m(E)\u001b[0m Unexpected termination of child process {\"error\": null, \"status\": \"exit status 1\"}\npanic: Wrapper process for worker 0 exited unexpectedly with: exit status 1\n\ngoroutine 53 [running]:\ngithub.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).watchWrapperProcess(0xc420363130)\n\t/go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:453 +0x5bb\ncreated by github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).startWrapper\n\t/go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:232 +0x1c8\npanic: Wrapper process for worker 1 exited unexpectedly with: exit status 1\n\ngoroutine 102 [running]:\ngithub.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).watchWrapperProcess(0xc42053e210)\n\t/go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:453 +0x5bb\ncreated by github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).startWrapper\n\t/go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:232 +0x1c8\n", "errVerbose": "\nError - Container wasn't healthy in time\n    .../nuclio/nuclio/pkg/dockerclient/shell.go:395\n\nCall stack:\nContainer wasn't healthy in time\n    .../nuclio/nuclio/pkg/dockerclient/shell.go:395\nFunction wasn't ready in time. Logs:\n20.09.08 03:23:47.966 \u001b[37m                processor\u001b[0m \u001b[32m(D)\u001b[0m Read configuration {\"config\": \"{\\n    \\\"metadata\\\": {\\n        \\\"name\\\": \\\"openvino.omz.public.faster_rcnn_inception_v2_coco\\\",\\n        \\\"namespace\\\": \\\"nuclio\\\",\\n        \\\"labels\\\": {\\n            \\\"nuclio.io/project-name\\\": \\\"cvat\\\"\\n        },\\n        \\\"annotations\\\": {\\n            \\\"framework\\\": \\\"openvino\\\",\\n            \\\"name\\\": \\\"Faster RCNN\\\",\\n            \\\"spec\\\": \\\"[\\\\n  { \\\\\\\"id\\\\\\\": 1, \\\\\\\"name\\\\\\\": \\\\\\\"person\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 2, \\\\\\\"name\\\\\\\": \\\\\\\"bicycle\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 3, \\\\\\\"name\\\\\\\": \\\\\\\"car\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 4, \\\\\\\"name\\\\\\\": \\\\\\\"motorcycle\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 5, \\\\\\\"name\\\\\\\": \\\\\\\"airplane\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 6, \\\\\\\"name\\\\\\\": \\\\\\\"bus\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 7, \\\\\\\"name\\\\\\\": \\\\\\\"train\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 8, \\\\\\\"name\\\\\\\": \\\\\\\"truck\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 9, \\\\\\\"name\\\\\\\": \\\\\\\"boat\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":10, \\\\\\\"name\\\\\\\": \\\\\\\"traffic_light\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":11, \\\\\\\"name\\\\\\\": \\\\\\\"fire_hydrant\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":13, \\\\\\\"name\\\\\\\": \\\\\\\"stop_sign\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":14, \\\\\\\"name\\\\\\\": \\\\\\\"parking_meter\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":15, \\\\\\\"name\\\\\\\": \\\\\\\"bench\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":16, \\\\\\\"name\\\\\\\": \\\\\\\"bird\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":17, \\\\\\\"name\\\\\\\": \\\\\\\"cat\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":18, \\\\\\\"name\\\\\\\": \\\\\\\"dog\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":19, \\\\\\\"name\\\\\\\": \\\\\\\"horse\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":20, \\\\\\\"name\\\\\\\": \\\\\\\"sheep\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":21, \\\\\\\"name\\\\\\\": \\\\\\\"cow\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":22, \\\\\\\"name\\\\\\\": \\\\\\\"elephant\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":23, \\\\\\\"name\\\\\\\": \\\\\\\"bear\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":24, \\\\\\\"name\\\\\\\": \\\\\\\"zebra\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":25, \\\\\\\"name\\\\\\\": \\\\\\\"giraffe\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":27, \\\\\\\"name\\\\\\\": \\\\\\\"backpack\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":28, \\\\\\\"name\\\\\\\": \\\\\\\"umbrella\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":31, \\\\\\\"name\\\\\\\": \\\\\\\"handbag\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":32, \\\\\\\"name\\\\\\\": \\\\\\\"tie\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":33, \\\\\\\"name\\\\\\\": \\\\\\\"suitcase\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":34, \\\\\\\"name\\\\\\\": \\\\\\\"frisbee\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":35, \\\\\\\"name\\\\\\\": \\\\\\\"skis\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":36, \\\\\\\"name\\\\\\\": \\\\\\\"snowboard\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":37, \\\\\\\"name\\\\\\\": \\\\\\\"sports_ball\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":38, \\\\\\\"name\\\\\\\": \\\\\\\"kite\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":39, \\\\\\\"name\\\\\\\": \\\\\\\"baseball_bat\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":40, \\\\\\\"name\\\\\\\": \\\\\\\"baseball_glove\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":41, \\\\\\\"name\\\\\\\": \\\\\\\"skateboard\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":42, \\\\\\\"name\\\\\\\": \\\\\\\"surfboard\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":43, \\\\\\\"name\\\\\\\": \\\\\\\"tennis_racket\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":44, \\\\\\\"name\\\\\\\": \\\\\\\"bottle\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":46, \\\\\\\"name\\\\\\\": \\\\\\\"wine_glass\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":47, \\\\\\\"name\\\\\\\": \\\\\\\"cup\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":48, \\\\\\\"name\\\\\\\": \\\\\\\"fork\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":49, \\\\\\\"name\\\\\\\": \\\\\\\"knife\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":50, \\\\\\\"name\\\\\\\": \\\\\\\"spoon\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":51, \\\\\\\"name\\\\\\\": \\\\\\\"bowl\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":52, \\\\\\\"name\\\\\\\": \\\\\\\"banana\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":53, \\\\\\\"name\\\\\\\": \\\\\\\"apple\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":54, \\\\\\\"name\\\\\\\": \\\\\\\"sandwich\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":55, \\\\\\\"name\\\\\\\": \\\\\\\"orange\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":56, \\\\\\\"name\\\\\\\": \\\\\\\"broccoli\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":57, \\\\\\\"name\\\\\\\": \\\\\\\"carrot\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":58, \\\\\\\"name\\\\\\\": \\\\\\\"hot_dog\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":59, \\\\\\\"name\\\\\\\": \\\\\\\"pizza\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":60, \\\\\\\"name\\\\\\\": \\\\\\\"donut\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":61, \\\\\\\"name\\\\\\\": \\\\\\\"cake\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":62, \\\\\\\"name\\\\\\\": \\\\\\\"chair\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":63, \\\\\\\"name\\\\\\\": \\\\\\\"couch\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":64, \\\\\\\"name\\\\\\\": \\\\\\\"potted_plant\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":65, \\\\\\\"name\\\\\\\": \\\\\\\"bed\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":67, \\\\\\\"name\\\\\\\": \\\\\\\"dining_table\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":70, \\\\\\\"name\\\\\\\": \\\\\\\"toilet\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":72, \\\\\\\"name\\\\\\\": \\\\\\\"tv\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":73, \\\\\\\"name\\\\\\\": \\\\\\\"laptop\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":74, \\\\\\\"name\\\\\\\": \\\\\\\"mouse\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":75, \\\\\\\"name\\\\\\\": \\\\\\\"remote\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":76, \\\\\\\"name\\\\\\\": \\\\\\\"keyboard\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":77, \\\\\\\"name\\\\\\\": \\\\\\\"cell_phone\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":78, \\\\\\\"name\\\\\\\": \\\\\\\"microwave\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":79, \\\\\\\"name\\\\\\\": \\\\\\\"oven\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":80, \\\\\\\"name\\\\\\\": \\\\\\\"toaster\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":81, \\\\\\\"name\\\\\\\": \\\\\\\"sink\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":83, \\\\\\\"name\\\\\\\": \\\\\\\"refrigerator\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":84, \\\\\\\"name\\\\\\\": \\\\\\\"book\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":85, \\\\\\\"name\\\\\\\": \\\\\\\"clock\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":86, \\\\\\\"name\\\\\\\": \\\\\\\"vase\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":87, \\\\\\\"name\\\\\\\": \\\\\\\"scissors\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":88, \\\\\\\"name\\\\\\\": \\\\\\\"teddy_bear\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":89, \\\\\\\"name\\\\\\\": \\\\\\\"hair_drier\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":90, \\\\\\\"name\\\\\\\": \\\\\\\"toothbrush\\\\\\\" }\\\\n]\\\\n\\\",\\n            \\\"type\\\": \\\"detector\\\"\\n        }\\n    },\\n    \\\"spec\\\": {\\n        \\\"description\\\": \\\"Faster RCNN inception v2 COCO via Intel OpenVINO toolkit\\\",\\n        \\\"handler\\\": \\\"main:handler\\\",\\n        \\\"runtime\\\": \\\"python:3.6\\\",\\n        \\\"env\\\": [\\n            {\\n                \\\"name\\\": \\\"NUCLIO_PYTHON_EXE_PATH\\\",\\n                \\\"value\\\": \\\"/opt/nuclio/common/python3\\\"\\n            }\\n        ],\\n        \\\"resources\\\": {},\\n        \\\"image\\\": \\\"cvat/openvino.omz.public.faster_rcnn_inception_v2_coco:latest\\\",\\n        \\\"targetCPU\\\": 75,\\n        \\\"triggers\\\": {\\n            \\\"myHttpTrigger\\\": {\\n                \\\"class\\\": \\\"\\\",\\n                \\\"kind\\\": \\\"http\\\",\\n                \\\"name\\\": \\\"\\\",\\n                \\\"maxWorkers\\\": 2,\\n                \\\"workerAvailabilityTimeoutMilliseconds\\\": 10000,\\n                \\\"attributes\\\": {\\n                    \\\"maxRequestBodySize\\\": 33554432\\n                }\\n            }\\n        },\\n        \\\"volumes\\\": [\\n            {\\n                \\\"volume\\\": {\\n                    \\\"name\\\": \\\"volume-1\\\",\\n                    \\\"hostPath\\\": {\\n                        \\\"path\\\": \\\"/mnt/hdd/projects//cvat/serverless/openvino/common\\\"\\n                    }\\n                },\\n                \\\"volumeMount\\\": {\\n                    \\\"name\\\": \\\"volume-1\\\",\\n                    \\\"mountPath\\\": \\\"/opt/nuclio/common\\\"\\n                }\\n            }\\n        ],\\n        \\\"build\\\": {\\n            \\\"path\\\": \\\"cvat/serverless/openvino/omz/public/faster_rcnn_inception_v2_coco/nuclio/\\\",\\n            \\\"image\\\": \\\"cvat/openvino.omz.public.faster_rcnn_inception_v2_coco\\\",\\n            \\\"baseImage\\\": \\\"openvino/ubuntu18_dev:2020.2\\\",\\n            \\\"directives\\\": {\\n                \\\"preCopy\\\": [\\n                    {\\n                        \\\"kind\\\": \\\"USER\\\",\\n                        \\\"value\\\": \\\"root\\\"\\n                    },\\n                    {\\n                        \\\"kind\\\": \\\"WORKDIR\\\",\\n                        \\\"value\\\": \\\"/opt/nuclio\\\"\\n                    },\\n                    {\\n                        \\\"kind\\\": \\\"RUN\\\",\\n                        \\\"value\\\": \\\"ln -s /usr/bin/pip3 /usr/bin/pip\\\"\\n                    },\\n                    {\\n                        \\\"kind\\\": \\\"RUN\\\",\\n                        \\\"value\\\": \\\"/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/downloader.py --name faster_rcnn_inception_v2_coco -o /opt/nuclio/open_model_zoo\\\"\\n                    },\\n                    {\\n                        \\\"kind\\\": \\\"RUN\\\",\\n                        \\\"value\\\": \\\"/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name faster_rcnn_inception_v2_coco --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo\\\"\\n                    }\\n                ]\\n            },\\n            \\\"timestamp\\\": 1599535427\\n        },\\n        \\\"platform\\\": {\\n            \\\"attributes\\\": {\\n                \\\"restartPolicy\\\": {\\n                    \\\"maximumRetryCount\\\": 3,\\n                    \\\"name\\\": \\\"always\\\"\\n                }\\n            }\\n        },\\n        \\\"readinessTimeoutSeconds\\\": 60,\\n        \\\"eventTimeout\\\": \\\"30s\\\"\\n    },\\n    \\\"PlatformConfig\\\": null\\n}\", \"platformConfig\": \"{\\n    \\\"kind\\\": \\\"local\\\",\\n    \\\"webAdmin\\\": {\\n        \\\"enabled\\\": true,\\n        \\\"listenAddress\\\": \\\":8081\\\"\\n    },\\n    \\\"healthCheck\\\": {\\n        \\\"enabled\\\": true,\\n        \\\"listenAddress\\\": \\\":8082\\\"\\n    },\\n    \\\"logger\\\": {\\n        \\\"sinks\\\": {\\n            \\\"stdout\\\": {\\n                \\\"kind\\\": \\\"stdout\\\"\\n            }\\n        },\\n        \\\"system\\\": [\\n            {\\n                \\\"level\\\": \\\"debug\\\",\\n                \\\"sink\\\": \\\"stdout\\\"\\n            }\\n        ],\\n        \\\"functions\\\": [\\n            {\\n                \\\"level\\\": \\\"debug\\\",\\n                \\\"sink\\\": \\\"stdout\\\"\\n            }\\n        ]\\n    },\\n    \\\"metrics\\\": {},\\n    \\\"scaleToZero\\\": {},\\n    \\\"autoScale\\\": {}\\n}\"}\n20.09.08 03:23:47.966 \u001b[37mcessor.healthcheck.server\u001b[0m \u001b[34m(I)\u001b[0m Listening {\"listenAddress\": \":8082\"}\n20.09.08 03:23:47.966 \u001b[37m           processor.http\u001b[0m \u001b[32m(D)\u001b[0m Creating worker pool {\"num\": 2}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Creating listener socket {\"path\": \"/tmp/nuclio-rpc-btbfigu11teg00f8q550.sock\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Creating listener socket {\"path\": \"/tmp/nuclio-rpc-btbfigu11teg00f8q55g.sock\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python wrapper script path {\"path\": \"/opt/nuclio/_nuclio_wrapper.py\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python handler {\"handler\": \"main:handler\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python wrapper script path {\"path\": \"/opt/nuclio/_nuclio_wrapper.py\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python handler {\"handler\": \"main:handler\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python executable {\"path\": \"/usr/bin/python3\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python executable {\"path\": \"/usr/bin/python3\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Setting PYTHONPATH {\"value\": \"PYTHONPATH=/opt/nuclio\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Setting PYTHONPATH {\"value\": \"PYTHONPATH=/opt/nuclio\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Running wrapper {\"command\": \"/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --socket-path /tmp/nuclio-rpc-btbfigu11teg00f8q550.sock --platform-kind local --namespace nuclio --worker-id 1 --trigger-kind http --trigger-name myHttpTrigger\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Running wrapper {\"command\": \"/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --socket-path /tmp/nuclio-rpc-btbfigu11teg00f8q55g.sock --platform-kind local --namespace nuclio --worker-id 0 --trigger-kind http --trigger-name myHttpTrigger\"}\nl{\"datetime\": \"2020-09-08 03:23:48,039\", \"level\": \"error\", \"message\": \"Caught unhandled exception while initializing\", \"with\": {\"err\": \"No module named 'model_loader'\", \"traceback\": \"Traceback (most recent call last):\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 286, in run_wrapper\\n    args.trigger_name)\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 51, in __init__\\n    self._entrypoint = self._load_entrypoint_from_handler(handler)\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 190, in _load_entrypoint_from_handler\\n    module = __import__(module_name)\\n  File \\\"/opt/nuclio/main.py\\\", line 5, in <module>\\n    from model_handler import ModelHandler\\n  File \\\"/opt/nuclio/model_handler.py\\\", line 6, in <module>\\n    from model_loader import ModelLoader\\nModuleNotFoundError: No module named 'model_loader'\\n\"}}\nl{\"datetime\": \"2020-09-08 03:23:48,039\", \"level\": \"error\", \"message\": \"Caught unhandled exception while initializing\", \"with\": {\"err\": \"No module named 'model_loader'\", \"traceback\": \"Traceback (most recent call last):\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 286, in run_wrapper\\n    args.trigger_name)\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 51, in __init__\\n    self._entrypoint = self._load_entrypoint_from_handler(handler)\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 190, in _load_entrypoint_from_handler\\n    module = __import__(module_name)\\n  File \\\"/opt/nuclio/main.py\\\", line 5, in <module>\\n    from model_handler import ModelHandler\\n  File \\\"/opt/nuclio/model_handler.py\\\", line 6, in <module>\\n    from model_loader import ModelLoader\\nModuleNotFoundError: No module named 'model_loader'\\n\"}}\n20.09.08 03:23:48.046 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[31m(E)\u001b[0m Unexpected termination of child process {\"error\": null, \"status\": \"exit status 1\"}\n20.09.08 03:23:48.046 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[31m(E)\u001b[0m Unexpected termination of child process {\"error\": null, \"status\": \"exit status 1\"}\npanic: Wrapper process for worker 0 exited unexpectedly with: exit status 1\n\ngoroutine 53 [running]:\ngithub.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).watchWrapperProcess(0xc420363130)\n\t/go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:453 +0x5bb\ncreated by github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).startWrapper\n\t/go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:232 +0x1c8\npanic: Wrapper process for worker 1 exited unexpectedly with: exit status 1\n\ngoroutine 102 [running]:\ngithub.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).watchWrapperProcess(0xc42053e210)\n\t/go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:453 +0x5bb\ncreated by github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).startWrapper\n\t/go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:232 +0x1c8\n\n    .../nuclio/nuclio/pkg/platform/local/platform.go:569\nFunction wasn't ready in time. Logs:\n20.09.08 03:23:47.966 \u001b[37m                processor\u001b[0m \u001b[32m(D)\u001b[0m Read configuration {\"config\": \"{\\n    \\\"metadata\\\": {\\n        \\\"name\\\": \\\"openvino.omz.public.faster_rcnn_inception_v2_coco\\\",\\n        \\\"namespace\\\": \\\"nuclio\\\",\\n        \\\"labels\\\": {\\n            \\\"nuclio.io/project-name\\\": \\\"cvat\\\"\\n        },\\n        \\\"annotations\\\": {\\n            \\\"framework\\\": \\\"openvino\\\",\\n            \\\"name\\\": \\\"Faster RCNN\\\",\\n            \\\"spec\\\": \\\"[\\\\n  { \\\\\\\"id\\\\\\\": 1, \\\\\\\"name\\\\\\\": \\\\\\\"person\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 2, \\\\\\\"name\\\\\\\": \\\\\\\"bicycle\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 3, \\\\\\\"name\\\\\\\": \\\\\\\"car\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 4, \\\\\\\"name\\\\\\\": \\\\\\\"motorcycle\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 5, \\\\\\\"name\\\\\\\": \\\\\\\"airplane\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 6, \\\\\\\"name\\\\\\\": \\\\\\\"bus\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 7, \\\\\\\"name\\\\\\\": \\\\\\\"train\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 8, \\\\\\\"name\\\\\\\": \\\\\\\"truck\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\": 9, \\\\\\\"name\\\\\\\": \\\\\\\"boat\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":10, \\\\\\\"name\\\\\\\": \\\\\\\"traffic_light\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":11, \\\\\\\"name\\\\\\\": \\\\\\\"fire_hydrant\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":13, \\\\\\\"name\\\\\\\": \\\\\\\"stop_sign\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":14, \\\\\\\"name\\\\\\\": \\\\\\\"parking_meter\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":15, \\\\\\\"name\\\\\\\": \\\\\\\"bench\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":16, \\\\\\\"name\\\\\\\": \\\\\\\"bird\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":17, \\\\\\\"name\\\\\\\": \\\\\\\"cat\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":18, \\\\\\\"name\\\\\\\": \\\\\\\"dog\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":19, \\\\\\\"name\\\\\\\": \\\\\\\"horse\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":20, \\\\\\\"name\\\\\\\": \\\\\\\"sheep\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":21, \\\\\\\"name\\\\\\\": \\\\\\\"cow\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":22, \\\\\\\"name\\\\\\\": \\\\\\\"elephant\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":23, \\\\\\\"name\\\\\\\": \\\\\\\"bear\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":24, \\\\\\\"name\\\\\\\": \\\\\\\"zebra\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":25, \\\\\\\"name\\\\\\\": \\\\\\\"giraffe\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":27, \\\\\\\"name\\\\\\\": \\\\\\\"backpack\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":28, \\\\\\\"name\\\\\\\": \\\\\\\"umbrella\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":31, \\\\\\\"name\\\\\\\": \\\\\\\"handbag\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":32, \\\\\\\"name\\\\\\\": \\\\\\\"tie\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":33, \\\\\\\"name\\\\\\\": \\\\\\\"suitcase\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":34, \\\\\\\"name\\\\\\\": \\\\\\\"frisbee\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":35, \\\\\\\"name\\\\\\\": \\\\\\\"skis\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":36, \\\\\\\"name\\\\\\\": \\\\\\\"snowboard\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":37, \\\\\\\"name\\\\\\\": \\\\\\\"sports_ball\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":38, \\\\\\\"name\\\\\\\": \\\\\\\"kite\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":39, \\\\\\\"name\\\\\\\": \\\\\\\"baseball_bat\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":40, \\\\\\\"name\\\\\\\": \\\\\\\"baseball_glove\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":41, \\\\\\\"name\\\\\\\": \\\\\\\"skateboard\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":42, \\\\\\\"name\\\\\\\": \\\\\\\"surfboard\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":43, \\\\\\\"name\\\\\\\": \\\\\\\"tennis_racket\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":44, \\\\\\\"name\\\\\\\": \\\\\\\"bottle\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":46, \\\\\\\"name\\\\\\\": \\\\\\\"wine_glass\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":47, \\\\\\\"name\\\\\\\": \\\\\\\"cup\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":48, \\\\\\\"name\\\\\\\": \\\\\\\"fork\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":49, \\\\\\\"name\\\\\\\": \\\\\\\"knife\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":50, \\\\\\\"name\\\\\\\": \\\\\\\"spoon\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":51, \\\\\\\"name\\\\\\\": \\\\\\\"bowl\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":52, \\\\\\\"name\\\\\\\": \\\\\\\"banana\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":53, \\\\\\\"name\\\\\\\": \\\\\\\"apple\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":54, \\\\\\\"name\\\\\\\": \\\\\\\"sandwich\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":55, \\\\\\\"name\\\\\\\": \\\\\\\"orange\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":56, \\\\\\\"name\\\\\\\": \\\\\\\"broccoli\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":57, \\\\\\\"name\\\\\\\": \\\\\\\"carrot\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":58, \\\\\\\"name\\\\\\\": \\\\\\\"hot_dog\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":59, \\\\\\\"name\\\\\\\": \\\\\\\"pizza\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":60, \\\\\\\"name\\\\\\\": \\\\\\\"donut\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":61, \\\\\\\"name\\\\\\\": \\\\\\\"cake\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":62, \\\\\\\"name\\\\\\\": \\\\\\\"chair\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":63, \\\\\\\"name\\\\\\\": \\\\\\\"couch\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":64, \\\\\\\"name\\\\\\\": \\\\\\\"potted_plant\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":65, \\\\\\\"name\\\\\\\": \\\\\\\"bed\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":67, \\\\\\\"name\\\\\\\": \\\\\\\"dining_table\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":70, \\\\\\\"name\\\\\\\": \\\\\\\"toilet\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":72, \\\\\\\"name\\\\\\\": \\\\\\\"tv\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":73, \\\\\\\"name\\\\\\\": \\\\\\\"laptop\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":74, \\\\\\\"name\\\\\\\": \\\\\\\"mouse\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":75, \\\\\\\"name\\\\\\\": \\\\\\\"remote\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":76, \\\\\\\"name\\\\\\\": \\\\\\\"keyboard\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":77, \\\\\\\"name\\\\\\\": \\\\\\\"cell_phone\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":78, \\\\\\\"name\\\\\\\": \\\\\\\"microwave\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":79, \\\\\\\"name\\\\\\\": \\\\\\\"oven\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":80, \\\\\\\"name\\\\\\\": \\\\\\\"toaster\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":81, \\\\\\\"name\\\\\\\": \\\\\\\"sink\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":83, \\\\\\\"name\\\\\\\": \\\\\\\"refrigerator\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":84, \\\\\\\"name\\\\\\\": \\\\\\\"book\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":85, \\\\\\\"name\\\\\\\": \\\\\\\"clock\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":86, \\\\\\\"name\\\\\\\": \\\\\\\"vase\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":87, \\\\\\\"name\\\\\\\": \\\\\\\"scissors\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":88, \\\\\\\"name\\\\\\\": \\\\\\\"teddy_bear\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":89, \\\\\\\"name\\\\\\\": \\\\\\\"hair_drier\\\\\\\" },\\\\n  { \\\\\\\"id\\\\\\\":90, \\\\\\\"name\\\\\\\": \\\\\\\"toothbrush\\\\\\\" }\\\\n]\\\\n\\\",\\n            \\\"type\\\": \\\"detector\\\"\\n        }\\n    },\\n    \\\"spec\\\": {\\n        \\\"description\\\": \\\"Faster RCNN inception v2 COCO via Intel OpenVINO toolkit\\\",\\n        \\\"handler\\\": \\\"main:handler\\\",\\n        \\\"runtime\\\": \\\"python:3.6\\\",\\n        \\\"env\\\": [\\n            {\\n                \\\"name\\\": \\\"NUCLIO_PYTHON_EXE_PATH\\\",\\n                \\\"value\\\": \\\"/opt/nuclio/common/python3\\\"\\n            }\\n        ],\\n        \\\"resources\\\": {},\\n        \\\"image\\\": \\\"cvat/openvino.omz.public.faster_rcnn_inception_v2_coco:latest\\\",\\n        \\\"targetCPU\\\": 75,\\n        \\\"triggers\\\": {\\n            \\\"myHttpTrigger\\\": {\\n                \\\"class\\\": \\\"\\\",\\n                \\\"kind\\\": \\\"http\\\",\\n                \\\"name\\\": \\\"\\\",\\n                \\\"maxWorkers\\\": 2,\\n                \\\"workerAvailabilityTimeoutMilliseconds\\\": 10000,\\n                \\\"attributes\\\": {\\n                    \\\"maxRequestBodySize\\\": 33554432\\n                }\\n            }\\n        },\\n        \\\"volumes\\\": [\\n            {\\n                \\\"volume\\\": {\\n                    \\\"name\\\": \\\"volume-1\\\",\\n                    \\\"hostPath\\\": {\\n                        \\\"path\\\": \\\"/mnt/hdd/projects//cvat/serverless/openvino/common\\\"\\n                    }\\n                },\\n                \\\"volumeMount\\\": {\\n                    \\\"name\\\": \\\"volume-1\\\",\\n                    \\\"mountPath\\\": \\\"/opt/nuclio/common\\\"\\n                }\\n            }\\n        ],\\n        \\\"build\\\": {\\n            \\\"path\\\": \\\"cvat/serverless/openvino/omz/public/faster_rcnn_inception_v2_coco/nuclio/\\\",\\n            \\\"image\\\": \\\"cvat/openvino.omz.public.faster_rcnn_inception_v2_coco\\\",\\n            \\\"baseImage\\\": \\\"openvino/ubuntu18_dev:2020.2\\\",\\n            \\\"directives\\\": {\\n                \\\"preCopy\\\": [\\n                    {\\n                        \\\"kind\\\": \\\"USER\\\",\\n                        \\\"value\\\": \\\"root\\\"\\n                    },\\n                    {\\n                        \\\"kind\\\": \\\"WORKDIR\\\",\\n                        \\\"value\\\": \\\"/opt/nuclio\\\"\\n                    },\\n                    {\\n                        \\\"kind\\\": \\\"RUN\\\",\\n                        \\\"value\\\": \\\"ln -s /usr/bin/pip3 /usr/bin/pip\\\"\\n                    },\\n                    {\\n                        \\\"kind\\\": \\\"RUN\\\",\\n                        \\\"value\\\": \\\"/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/downloader.py --name faster_rcnn_inception_v2_coco -o /opt/nuclio/open_model_zoo\\\"\\n                    },\\n                    {\\n                        \\\"kind\\\": \\\"RUN\\\",\\n                        \\\"value\\\": \\\"/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name faster_rcnn_inception_v2_coco --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo\\\"\\n                    }\\n                ]\\n            },\\n            \\\"timestamp\\\": 1599535427\\n        },\\n        \\\"platform\\\": {\\n            \\\"attributes\\\": {\\n                \\\"restartPolicy\\\": {\\n                    \\\"maximumRetryCount\\\": 3,\\n                    \\\"name\\\": \\\"always\\\"\\n                }\\n            }\\n        },\\n        \\\"readinessTimeoutSeconds\\\": 60,\\n        \\\"eventTimeout\\\": \\\"30s\\\"\\n    },\\n    \\\"PlatformConfig\\\": null\\n}\", \"platformConfig\": \"{\\n    \\\"kind\\\": \\\"local\\\",\\n    \\\"webAdmin\\\": {\\n        \\\"enabled\\\": true,\\n        \\\"listenAddress\\\": \\\":8081\\\"\\n    },\\n    \\\"healthCheck\\\": {\\n        \\\"enabled\\\": true,\\n        \\\"listenAddress\\\": \\\":8082\\\"\\n    },\\n    \\\"logger\\\": {\\n        \\\"sinks\\\": {\\n            \\\"stdout\\\": {\\n                \\\"kind\\\": \\\"stdout\\\"\\n            }\\n        },\\n        \\\"system\\\": [\\n            {\\n                \\\"level\\\": \\\"debug\\\",\\n                \\\"sink\\\": \\\"stdout\\\"\\n            }\\n        ],\\n        \\\"functions\\\": [\\n            {\\n                \\\"level\\\": \\\"debug\\\",\\n                \\\"sink\\\": \\\"stdout\\\"\\n            }\\n        ]\\n    },\\n    \\\"metrics\\\": {},\\n    \\\"scaleToZero\\\": {},\\n    \\\"autoScale\\\": {}\\n}\"}\n20.09.08 03:23:47.966 \u001b[37mcessor.healthcheck.server\u001b[0m \u001b[34m(I)\u001b[0m Listening {\"listenAddress\": \":8082\"}\n20.09.08 03:23:47.966 \u001b[37m           processor.http\u001b[0m \u001b[32m(D)\u001b[0m Creating worker pool {\"num\": 2}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Creating listener socket {\"path\": \"/tmp/nuclio-rpc-btbfigu11teg00f8q550.sock\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Creating listener socket {\"path\": \"/tmp/nuclio-rpc-btbfigu11teg00f8q55g.sock\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python wrapper script path {\"path\": \"/opt/nuclio/_nuclio_wrapper.py\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python handler {\"handler\": \"main:handler\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python wrapper script path {\"path\": \"/opt/nuclio/_nuclio_wrapper.py\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python handler {\"handler\": \"main:handler\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python executable {\"path\": \"/usr/bin/python3\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Using Python executable {\"path\": \"/usr/bin/python3\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Setting PYTHONPATH {\"value\": \"PYTHONPATH=/opt/nuclio\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Setting PYTHONPATH {\"value\": \"PYTHONPATH=/opt/nuclio\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Running wrapper {\"command\": \"/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --socket-path /tmp/nuclio-rpc-btbfigu11teg00f8q550.sock --platform-kind local --namespace nuclio --worker-id 1 --trigger-kind http --trigger-name myHttpTrigger\"}\n20.09.08 03:23:47.967 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[32m(D)\u001b[0m Running wrapper {\"command\": \"/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --socket-path /tmp/nuclio-rpc-btbfigu11teg00f8q55g.sock --platform-kind local --namespace nuclio --worker-id 0 --trigger-kind http --trigger-name myHttpTrigger\"}\nl{\"datetime\": \"2020-09-08 03:23:48,039\", \"level\": \"error\", \"message\": \"Caught unhandled exception while initializing\", \"with\": {\"err\": \"No module named 'model_loader'\", \"traceback\": \"Traceback (most recent call last):\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 286, in run_wrapper\\n    args.trigger_name)\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 51, in __init__\\n    self._entrypoint = self._load_entrypoint_from_handler(handler)\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 190, in _load_entrypoint_from_handler\\n    module = __import__(module_name)\\n  File \\\"/opt/nuclio/main.py\\\", line 5, in <module>\\n    from model_handler import ModelHandler\\n  File \\\"/opt/nuclio/model_handler.py\\\", line 6, in <module>\\n    from model_loader import ModelLoader\\nModuleNotFoundError: No module named 'model_loader'\\n\"}}\nl{\"datetime\": \"2020-09-08 03:23:48,039\", \"level\": \"error\", \"message\": \"Caught unhandled exception while initializing\", \"with\": {\"err\": \"No module named 'model_loader'\", \"traceback\": \"Traceback (most recent call last):\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 286, in run_wrapper\\n    args.trigger_name)\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 51, in __init__\\n    self._entrypoint = self._load_entrypoint_from_handler(handler)\\n  File \\\"/opt/nuclio/_nuclio_wrapper.py\\\", line 190, in _load_entrypoint_from_handler\\n    module = __import__(module_name)\\n  File \\\"/opt/nuclio/main.py\\\", line 5, in <module>\\n    from model_handler import ModelHandler\\n  File \\\"/opt/nuclio/model_handler.py\\\", line 6, in <module>\\n    from model_loader import ModelLoader\\nModuleNotFoundError: No module named 'model_loader'\\n\"}}\n20.09.08 03:23:48.046 \u001b[37msor.http.w0.python.logger\u001b[0m \u001b[31m(E)\u001b[0m Unexpected termination of child process {\"error\": null, \"status\": \"exit status 1\"}\n20.09.08 03:23:48.046 \u001b[37msor.http.w1.python.logger\u001b[0m \u001b[31m(E)\u001b[0m Unexpected termination of child process {\"error\": null, \"status\": \"exit status 1\"}\npanic: Wrapper process for worker 0 exited unexpectedly with: exit status 1\n\ngoroutine 53 [running]:\ngithub.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).watchWrapperProcess(0xc420363130)\n\t/go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:453 +0x5bb\ncreated by github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).startWrapper\n\t/go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:232 +0x1c8\npanic: Wrapper process for worker 1 exited unexpectedly with: exit status 1\n\ngoroutine 102 [running]:\ngithub.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).watchWrapperProcess(0xc42053e210)\n\t/go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:453 +0x5bb\ncreated by github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).startWrapper\n\t/go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:232 +0x1c8\n", "errCauses": [{"error": "Container wasn't healthy in time", "errorVerbose": "\nError - Container wasn't healthy in time\n    .../nuclio/nuclio/pkg/dockerclient/shell.go:395\n\nCall stack:\nContainer wasn't healthy in time\n    .../nuclio/nuclio/pkg/dockerclient/shell.go:395\nContainer wasn't healthy in time", "errorCauses": [{}]}]}

Error - Container wasn't healthy in time
    .../nuclio/nuclio/pkg/dockerclient/shell.go:395

Call stack:
Container wasn't healthy in time
    .../nuclio/nuclio/pkg/dockerclient/shell.go:395
Function wasn't ready in time. Logs:
20.09.08 03:23:47.966                 processor (D) Read configuration {"config": "{\n    \"metadata\": {\n        \"name\": \"openvino.omz.public.faster_rcnn_inception_v2_coco\",\n        \"namespace\": \"nuclio\",\n        \"labels\": {\n            \"nuclio.io/project-name\": \"cvat\"\n        },\n        \"annotations\": {\n            \"framework\": \"openvino\",\n            \"name\": \"Faster RCNN\",\n            \"spec\": \"[\\n  { \\\"id\\\": 1, \\\"name\\\": \\\"person\\\" },\\n  { \\\"id\\\": 2, \\\"name\\\": \\\"bicycle\\\" },\\n  { \\\"id\\\": 3, \\\"name\\\": \\\"car\\\" },\\n  { \\\"id\\\": 4, \\\"name\\\": \\\"motorcycle\\\" },\\n  { \\\"id\\\": 5, \\\"name\\\": \\\"airplane\\\" },\\n  { \\\"id\\\": 6, \\\"name\\\": \\\"bus\\\" },\\n  { \\\"id\\\": 7, \\\"name\\\": \\\"train\\\" },\\n  { \\\"id\\\": 8, \\\"name\\\": \\\"truck\\\" },\\n  { \\\"id\\\": 9, \\\"name\\\": \\\"boat\\\" },\\n  { \\\"id\\\":10, \\\"name\\\": \\\"traffic_light\\\" },\\n  { \\\"id\\\":11, \\\"name\\\": \\\"fire_hydrant\\\" },\\n  { \\\"id\\\":13, \\\"name\\\": \\\"stop_sign\\\" },\\n  { \\\"id\\\":14, \\\"name\\\": \\\"parking_meter\\\" },\\n  { \\\"id\\\":15, \\\"name\\\": \\\"bench\\\" },\\n  { \\\"id\\\":16, \\\"name\\\": \\\"bird\\\" },\\n  { \\\"id\\\":17, \\\"name\\\": \\\"cat\\\" },\\n  { \\\"id\\\":18, \\\"name\\\": \\\"dog\\\" },\\n  { \\\"id\\\":19, \\\"name\\\": \\\"horse\\\" },\\n  { \\\"id\\\":20, \\\"name\\\": \\\"sheep\\\" },\\n  { \\\"id\\\":21, \\\"name\\\": \\\"cow\\\" },\\n  { \\\"id\\\":22, \\\"name\\\": \\\"elephant\\\" },\\n  { \\\"id\\\":23, \\\"name\\\": \\\"bear\\\" },\\n  { \\\"id\\\":24, \\\"name\\\": \\\"zebra\\\" },\\n  { \\\"id\\\":25, \\\"name\\\": \\\"giraffe\\\" },\\n  { \\\"id\\\":27, \\\"name\\\": \\\"backpack\\\" },\\n  { \\\"id\\\":28, \\\"name\\\": \\\"umbrella\\\" },\\n  { \\\"id\\\":31, \\\"name\\\": \\\"handbag\\\" },\\n  { \\\"id\\\":32, \\\"name\\\": \\\"tie\\\" },\\n  { \\\"id\\\":33, \\\"name\\\": \\\"suitcase\\\" },\\n  { \\\"id\\\":34, \\\"name\\\": \\\"frisbee\\\" },\\n  { \\\"id\\\":35, \\\"name\\\": \\\"skis\\\" },\\n  { \\\"id\\\":36, \\\"name\\\": \\\"snowboard\\\" },\\n  { \\\"id\\\":37, \\\"name\\\": \\\"sports_ball\\\" },\\n  { \\\"id\\\":38, \\\"name\\\": \\\"kite\\\" },\\n  { \\\"id\\\":39, \\\"name\\\": \\\"baseball_bat\\\" },\\n  { \\\"id\\\":40, \\\"name\\\": \\\"baseball_glove\\\" },\\n  { \\\"id\\\":41, \\\"name\\\": \\\"skateboard\\\" },\\n  { \\\"id\\\":42, \\\"name\\\": \\\"surfboard\\\" },\\n  { \\\"id\\\":43, \\\"name\\\": \\\"tennis_racket\\\" },\\n  { \\\"id\\\":44, \\\"name\\\": \\\"bottle\\\" },\\n  { \\\"id\\\":46, \\\"name\\\": \\\"wine_glass\\\" },\\n  { \\\"id\\\":47, \\\"name\\\": \\\"cup\\\" },\\n  { \\\"id\\\":48, \\\"name\\\": \\\"fork\\\" },\\n  { \\\"id\\\":49, \\\"name\\\": \\\"knife\\\" },\\n  { \\\"id\\\":50, \\\"name\\\": \\\"spoon\\\" },\\n  { \\\"id\\\":51, \\\"name\\\": \\\"bowl\\\" },\\n  { \\\"id\\\":52, \\\"name\\\": \\\"banana\\\" },\\n  { \\\"id\\\":53, \\\"name\\\": \\\"apple\\\" },\\n  { \\\"id\\\":54, \\\"name\\\": \\\"sandwich\\\" },\\n  { \\\"id\\\":55, \\\"name\\\": \\\"orange\\\" },\\n  { \\\"id\\\":56, \\\"name\\\": \\\"broccoli\\\" },\\n  { \\\"id\\\":57, \\\"name\\\": \\\"carrot\\\" },\\n  { \\\"id\\\":58, \\\"name\\\": \\\"hot_dog\\\" },\\n  { \\\"id\\\":59, \\\"name\\\": \\\"pizza\\\" },\\n  { \\\"id\\\":60, \\\"name\\\": \\\"donut\\\" },\\n  { \\\"id\\\":61, \\\"name\\\": \\\"cake\\\" },\\n  { \\\"id\\\":62, \\\"name\\\": \\\"chair\\\" },\\n  { \\\"id\\\":63, \\\"name\\\": \\\"couch\\\" },\\n  { \\\"id\\\":64, \\\"name\\\": \\\"potted_plant\\\" },\\n  { \\\"id\\\":65, \\\"name\\\": \\\"bed\\\" },\\n  { \\\"id\\\":67, \\\"name\\\": \\\"dining_table\\\" },\\n  { \\\"id\\\":70, \\\"name\\\": \\\"toilet\\\" },\\n  { \\\"id\\\":72, \\\"name\\\": \\\"tv\\\" },\\n  { \\\"id\\\":73, \\\"name\\\": \\\"laptop\\\" },\\n  { \\\"id\\\":74, \\\"name\\\": \\\"mouse\\\" },\\n  { \\\"id\\\":75, \\\"name\\\": \\\"remote\\\" },\\n  { \\\"id\\\":76, \\\"name\\\": \\\"keyboard\\\" },\\n  { \\\"id\\\":77, \\\"name\\\": \\\"cell_phone\\\" },\\n  { \\\"id\\\":78, \\\"name\\\": \\\"microwave\\\" },\\n  { \\\"id\\\":79, \\\"name\\\": \\\"oven\\\" },\\n  { \\\"id\\\":80, \\\"name\\\": \\\"toaster\\\" },\\n  { \\\"id\\\":81, \\\"name\\\": \\\"sink\\\" },\\n  { \\\"id\\\":83, \\\"name\\\": \\\"refrigerator\\\" },\\n  { \\\"id\\\":84, \\\"name\\\": \\\"book\\\" },\\n  { \\\"id\\\":85, \\\"name\\\": \\\"clock\\\" },\\n  { \\\"id\\\":86, \\\"name\\\": \\\"vase\\\" },\\n  { \\\"id\\\":87, \\\"name\\\": \\\"scissors\\\" },\\n  { \\\"id\\\":88, \\\"name\\\": \\\"teddy_bear\\\" },\\n  { \\\"id\\\":89, \\\"name\\\": \\\"hair_drier\\\" },\\n  { \\\"id\\\":90, \\\"name\\\": \\\"toothbrush\\\" }\\n]\\n\",\n            \"type\": \"detector\"\n        }\n    },\n    \"spec\": {\n        \"description\": \"Faster RCNN inception v2 COCO via Intel OpenVINO toolkit\",\n        \"handler\": \"main:handler\",\n        \"runtime\": \"python:3.6\",\n        \"env\": [\n            {\n                \"name\": \"NUCLIO_PYTHON_EXE_PATH\",\n                \"value\": \"/opt/nuclio/common/python3\"\n            }\n        ],\n        \"resources\": {},\n        \"image\": \"cvat/openvino.omz.public.faster_rcnn_inception_v2_coco:latest\",\n        \"targetCPU\": 75,\n        \"triggers\": {\n            \"myHttpTrigger\": {\n                \"class\": \"\",\n                \"kind\": \"http\",\n                \"name\": \"\",\n                \"maxWorkers\": 2,\n                \"workerAvailabilityTimeoutMilliseconds\": 10000,\n                \"attributes\": {\n                    \"maxRequestBodySize\": 33554432\n                }\n            }\n        },\n        \"volumes\": [\n            {\n                \"volume\": {\n                    \"name\": \"volume-1\",\n                    \"hostPath\": {\n                        \"path\": \"/mnt/hdd/projects//cvat/serverless/openvino/common\"\n                    }\n                },\n                \"volumeMount\": {\n                    \"name\": \"volume-1\",\n                    \"mountPath\": \"/opt/nuclio/common\"\n                }\n            }\n        ],\n        \"build\": {\n            \"path\": \"cvat/serverless/openvino/omz/public/faster_rcnn_inception_v2_coco/nuclio/\",\n            \"image\": \"cvat/openvino.omz.public.faster_rcnn_inception_v2_coco\",\n            \"baseImage\": \"openvino/ubuntu18_dev:2020.2\",\n            \"directives\": {\n                \"preCopy\": [\n                    {\n                        \"kind\": \"USER\",\n                        \"value\": \"root\"\n                    },\n                    {\n                        \"kind\": \"WORKDIR\",\n                        \"value\": \"/opt/nuclio\"\n                    },\n                    {\n                        \"kind\": \"RUN\",\n                        \"value\": \"ln -s /usr/bin/pip3 /usr/bin/pip\"\n                    },\n                    {\n                        \"kind\": \"RUN\",\n                        \"value\": \"/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/downloader.py --name faster_rcnn_inception_v2_coco -o /opt/nuclio/open_model_zoo\"\n                    },\n                    {\n                        \"kind\": \"RUN\",\n                        \"value\": \"/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name faster_rcnn_inception_v2_coco --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo\"\n                    }\n                ]\n            },\n            \"timestamp\": 1599535427\n        },\n        \"platform\": {\n            \"attributes\": {\n                \"restartPolicy\": {\n                    \"maximumRetryCount\": 3,\n                    \"name\": \"always\"\n                }\n            }\n        },\n        \"readinessTimeoutSeconds\": 60,\n        \"eventTimeout\": \"30s\"\n    },\n    \"PlatformConfig\": null\n}", "platformConfig": "{\n    \"kind\": \"local\",\n    \"webAdmin\": {\n        \"enabled\": true,\n        \"listenAddress\": \":8081\"\n    },\n    \"healthCheck\": {\n        \"enabled\": true,\n        \"listenAddress\": \":8082\"\n    },\n    \"logger\": {\n        \"sinks\": {\n            \"stdout\": {\n                \"kind\": \"stdout\"\n            }\n        },\n        \"system\": [\n            {\n                \"level\": \"debug\",\n                \"sink\": \"stdout\"\n            }\n        ],\n        \"functions\": [\n            {\n                \"level\": \"debug\",\n                \"sink\": \"stdout\"\n            }\n        ]\n    },\n    \"metrics\": {},\n    \"scaleToZero\": {},\n    \"autoScale\": {}\n}"}
20.09.08 03:23:47.966 cessor.healthcheck.server (I) Listening {"listenAddress": ":8082"}
20.09.08 03:23:47.966            processor.http (D) Creating worker pool {"num": 2}
20.09.08 03:23:47.967 sor.http.w1.python.logger (D) Creating listener socket {"path": "/tmp/nuclio-rpc-btbfigu11teg00f8q550.sock"}
20.09.08 03:23:47.967 sor.http.w0.python.logger (D) Creating listener socket {"path": "/tmp/nuclio-rpc-btbfigu11teg00f8q55g.sock"}
20.09.08 03:23:47.967 sor.http.w1.python.logger (D) Using Python wrapper script path {"path": "/opt/nuclio/_nuclio_wrapper.py"}
20.09.08 03:23:47.967 sor.http.w1.python.logger (D) Using Python handler {"handler": "main:handler"}
20.09.08 03:23:47.967 sor.http.w0.python.logger (D) Using Python wrapper script path {"path": "/opt/nuclio/_nuclio_wrapper.py"}
20.09.08 03:23:47.967 sor.http.w0.python.logger (D) Using Python handler {"handler": "main:handler"}
20.09.08 03:23:47.967 sor.http.w1.python.logger (D) Using Python executable {"path": "/usr/bin/python3"}
20.09.08 03:23:47.967 sor.http.w0.python.logger (D) Using Python executable {"path": "/usr/bin/python3"}
20.09.08 03:23:47.967 sor.http.w1.python.logger (D) Setting PYTHONPATH {"value": "PYTHONPATH=/opt/nuclio"}
20.09.08 03:23:47.967 sor.http.w0.python.logger (D) Setting PYTHONPATH {"value": "PYTHONPATH=/opt/nuclio"}
20.09.08 03:23:47.967 sor.http.w1.python.logger (D) Running wrapper {"command": "/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --socket-path /tmp/nuclio-rpc-btbfigu11teg00f8q550.sock --platform-kind local --namespace nuclio --worker-id 1 --trigger-kind http --trigger-name myHttpTrigger"}
20.09.08 03:23:47.967 sor.http.w0.python.logger (D) Running wrapper {"command": "/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --socket-path /tmp/nuclio-rpc-btbfigu11teg00f8q55g.sock --platform-kind local --namespace nuclio --worker-id 0 --trigger-kind http --trigger-name myHttpTrigger"}
l{"datetime": "2020-09-08 03:23:48,039", "level": "error", "message": "Caught unhandled exception while initializing", "with": {"err": "No module named 'model_loader'", "traceback": "Traceback (most recent call last):\n  File \"/opt/nuclio/_nuclio_wrapper.py\", line 286, in run_wrapper\n    args.trigger_name)\n  File \"/opt/nuclio/_nuclio_wrapper.py\", line 51, in __init__\n    self._entrypoint = self._load_entrypoint_from_handler(handler)\n  File \"/opt/nuclio/_nuclio_wrapper.py\", line 190, in _load_entrypoint_from_handler\n    module = __import__(module_name)\n  File \"/opt/nuclio/main.py\", line 5, in <module>\n    from model_handler import ModelHandler\n  File \"/opt/nuclio/model_handler.py\", line 6, in <module>\n    from model_loader import ModelLoader\nModuleNotFoundError: No module named 'model_loader'\n"}}
l{"datetime": "2020-09-08 03:23:48,039", "level": "error", "message": "Caught unhandled exception while initializing", "with": {"err": "No module named 'model_loader'", "traceback": "Traceback (most recent call last):\n  File \"/opt/nuclio/_nuclio_wrapper.py\", line 286, in run_wrapper\n    args.trigger_name)\n  File \"/opt/nuclio/_nuclio_wrapper.py\", line 51, in __init__\n    self._entrypoint = self._load_entrypoint_from_handler(handler)\n  File \"/opt/nuclio/_nuclio_wrapper.py\", line 190, in _load_entrypoint_from_handler\n    module = __import__(module_name)\n  File \"/opt/nuclio/main.py\", line 5, in <module>\n    from model_handler import ModelHandler\n  File \"/opt/nuclio/model_handler.py\", line 6, in <module>\n    from model_loader import ModelLoader\nModuleNotFoundError: No module named 'model_loader'\n"}}
20.09.08 03:23:48.046 sor.http.w0.python.logger (E) Unexpected termination of child process {"error": null, "status": "exit status 1"}
20.09.08 03:23:48.046 sor.http.w1.python.logger (E) Unexpected termination of child process {"error": null, "status": "exit status 1"}
panic: Wrapper process for worker 0 exited unexpectedly with: exit status 1

goroutine 53 [running]:
github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).watchWrapperProcess(0xc420363130)
    /go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:453 +0x5bb
created by github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).startWrapper
    /go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:232 +0x1c8
panic: Wrapper process for worker 1 exited unexpectedly with: exit status 1

goroutine 102 [running]:
github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).watchWrapperProcess(0xc42053e210)
    /go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:453 +0x5bb
created by github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).startWrapper
    /go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:232 +0x1c8

    .../nuclio/nuclio/pkg/platform/local/platform.go:569
Failed to deploy function
    .../nuclio/pkg/platform/abstract/platform.go:159

But looks like docker image has been created. $ docker images

REPOSITORY                                               TAG                 IMAGE ID            CREATED             SIZE
cvat/openvino.omz.public.faster_rcnn_inception_v2_coco   latest              c516252c49a2        About an hour ago   5.26GB
<none>                                                   <none>              d62a551be48e        About an hour ago   5.26GB
cvat_logstash                                            latest              99fa48c5ffeb        About an hour ago   674MB
cvat_kibana                                              latest              2c44bc28f11e        About an hour ago   493MB
cvat_elasticsearch                                       latest              1e73f00b99c5        About an hour ago   678MB
<none>                                                   <none>              1421ac098832        2 hours ago         5.26GB
cvat/openvino.omz.public.yolo-v3-tf                      latest              006c43758ff5        2 hours ago         5.53GB
cvat/ui                                                  latest              455986be759e        3 hours ago         36.7MB
<none>                                                   <none>              aae5091de29b        3 hours ago         995MB
cvat/server                                              latest              25a9bb716013        3 hours ago         3.15GB
quay.io/nuclio/handler-builder-python-onbuild            1.1.37-amd64        0d6bcafa17a2        43 hours ago        29.1MB
quay.io/nuclio/dashboard                                 stable-amd64        dafab320a3ec        43 hours ago        208MB
ubuntu                                                   20.04               4e2eef94cd6b        2 weeks ago         73.9MB
postgres                                                 10-alpine           eb55da5694a2        3 weeks ago         71.8MB
node                                                     lts-alpine          18f4bc975732        5 weeks ago         89.3MB
nginx                                                    stable-alpine       8c1bfa967ebf        8 weeks ago         21.5MB
quay.io/nuclio/dashboard                                 1.4.8-amd64         8aeef5e3ef33        2 months ago        224MB
redis                                                    4.0-alpine          e3dd0e49bca5        4 months ago        20.4MB
alpine                                                   3.11                f70734b6a266        4 months ago        5.61MB
openvino/ubuntu18_dev                                    2020.2              bf7a4dff2d97        4 months ago        5GB
docker.elastic.co/logstash/logstash-oss                  6.4.0               15e297c65be6        2 years ago         669MB
docker.elastic.co/kibana/kibana-oss                      6.4.0               c8986fcd3caa        2 years ago         493MB
docker.elastic.co/elasticsearch/elasticsearch-oss        6.4.0               a13ec2fa6275        2 years ago         678MB
quay.io/nuclio/uhttpc                                    0.0.1-amd64         5c59b3d31aa8        2 years ago         3.96MB

But no container: $docker ps

CONTAINER ID        IMAGE                                  COMMAND                  CREATED             STATUS              PORTS                            NAMES
2d42f4f1ad94        nginx:stable-alpine                    "/docker-entrypoint.…"   2 hours ago         Up 2 hours          0.0.0.0:8080->80/tcp             cvat_proxy
b23a214d233e        cvat/server                            "/usr/bin/supervisord"   2 hours ago         Up 2 hours          8080/tcp, 8443/tcp               cvat
c6d2b77f063b        cvat_kibana                            "/usr/local/bin/kiba…"   2 hours ago         Up 2 hours          5601/tcp                         cvat_kibana
e11df058c5c5        cvat_logstash                          "/usr/local/bin/dock…"   2 hours ago         Up 2 hours          5000/tcp, 5044/tcp, 9600/tcp     cvat_logstash
66fd480cfb25        cvat_elasticsearch                     "/usr/local/bin/dock…"   2 hours ago         Up 2 hours          9200/tcp, 9300/tcp               cvat_elasticsearch
f199152affe9        alpine:3.11                            "/bin/sh -c '/bin/sl…"   2 hours ago         Up 2 hours                                           nuclio-local-storage-reader
24db1754e117        cvat/ui                                "/docker-entrypoint.…"   2 hours ago         Up 2 hours          80/tcp                           cvat_ui
39036c30c54b        redis:4.0-alpine                       "docker-entrypoint.s…"   2 hours ago         Up 2 hours          6379/tcp                         cvat_redis
7439a3238d4b        postgres:10-alpine                     "docker-entrypoint.s…"   2 hours ago         Up 2 hours          5432/tcp                         cvat_db
fc11befa2e45        quay.io/nuclio/dashboard:1.4.8-amd64   "sh -c ./runner.sh"      2 hours ago         Up 2 hours          80/tcp, 0.0.0.0:8070->8070/tcp   nuclio

Any help would be highly appreciated.

cognitiveRobot commented 4 years ago

Working with the latest release of nuclio. https://github.com/nuclio/nuclio/releases/tag/1.4.17

varchanaiyer commented 4 years ago

Hello,

I am also facing a similar issue. I am using the 1.4.17 version of nuclio, but when I try to perform automatic sengmentation using the openvino yolo model, then I get an error message:

Error: Inference status for the task 1 is failed. requests.exceptions.HTTPError: 503 Server Error: Service Unavailable for url: http://nuclio:8070/api/function_invocations

Here are the logs:

docker_logs.txt

yolo-v3-tf-logs.txt

Thank you

WeiChihChern commented 4 years ago

@nmanovic Same issue, and fixed by using latest nuclio suggested by @cognitiveRobot. Please update the link for nuclio in the guide, that one is an older version which will cause problems for serverless functions.

veer5551 commented 4 years ago

Hello,

I am getting these errors while executing the first command itself nuctl create project cvat. Tried multiple versions and got different errors.

Error on nuctl version - 1.4.17

Error - exec: "/bin/sh": file does not exist
    /nuclio/pkg/cmdrunner/cmdrunner.go:124

Error on nuctl version - 1.3.17 and 1.1.37

Error - Failed to create auto platform
    .../nuclio/nuclio/pkg/platform/factory/factory.go:73

Running on Windows 10. Let me know If I am missing something here!

Thanks a lot!

cognitiveRobot commented 4 years ago

@varchanaiyer, after trying to perform automatic segmentation, immediately check docker logs nuclio-nuclio-openvino.omz.public.yolo-v3-tf (make sure the image name is correct in this command) and see what's the error message. Hope you will find the solution. If not then share your docker logs share, we will try.

cognitiveRobot commented 4 years ago

@veer5551, go to the location where you downloaded nuctl and check the filename. In my case, it's nuctl-1.4.17-linux-amd64, so I run ./nuctl-1.4.17-linux-amd64 .........

veer5551 commented 4 years ago

Hello @cognitiveRobot,

I renamed the downloaded files and ran accordingly. Still getting the same Errors.

Thanks!

cognitiveRobot commented 4 years ago

well, Error - exec: "/bin/sh": file does not exist means there is something wrong in your execution. It could be either you are not going to the directory where the nuctl is or you are not executing the command properly. Hope these give you some directions.

veer5551 commented 4 years ago

Followed the same procedure on a Linux system and everything was smooth. Models are deployed correctly and I am able to use them as well.

Need to check if this is an issue of Nuclio + WSL in Windows 10 combination problem. Maybe I'll report the issue there. If anyone here has got it working, please share the procedure/guide.

Alternatively, is it possible to deploy the models via the Nuclio UI Dashboard by loading the appropriate YAML and model handler? Tried doing it but got some function invocation error.

Thanks a lot!

nmanovic commented 4 years ago

@veer5551 , could you please check if basic examples from nuclio are working on your Windows machine and report problems to nuclio github if it doesn't work?

veer5551 commented 4 years ago

Reported the Issue on Nuclio github: Reference Link: https://github.com/nuclio/nuclio/issues/1821#issue-701006809

Thanks!

beep-love commented 3 years ago

I was having an error while executing:

./nuctl-1.3.31-linux-amd64 deploy --project-name cvat \ > --path serverless/openvino/dextr/nuclio \ > --volumepwd/serverless/openvino/common:/opt/nuclio/common \ > --platform local

Error - Container wasn't healthy in time .../nuclio/nuclio/pkg/dockerclient/shell.go:395

Call stack: Container wasn't healthy in time .../nuclio/nuclio/pkg/dockerclient/shell.go:395 Function wasn't ready in time. Logs:

as a first warning but the process kept continuing before getting terminated with the following error

`20.12.04 08:40:45.720 sor.http.w0.python.logger (E) Unexpected termination of child process {"error": null, "status": "exit status 1"} panic: Wrapper process for worker 0 exited unexpectedly with: exit status 1

goroutine 22 [running]: github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(AbstractRuntime).watchWrapperProcess(0xc4205160b0) /go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:453 +0x5bb created by github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(AbstractRuntime).startWrapper /go/src/github.com/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:232 +0x1c8

.../nuclio/nuclio/pkg/platform/local/platform.go:573

Failed to deploy function .../nuclio/pkg/platform/abstract/platform.go:172 ` I checked the GitHub repo and found that the URL does not exist but the file is rather with URL https://github.com/nuclio/nuclio/tree/1.3.x/pkg instead of github.com/nuclio/nuclio/pkg/ ......

Can you please help on this issue @cognitiveRobot @nmanovic ?

cognitiveRobot commented 3 years ago

@beep-love looks like you are trying to use an older version of nuclio. I did the same mistake, then when I used https://github.com/nuclio/nuclio/releases/tag/1.4.17 this, it worked.

beep-love commented 3 years ago

Yeah. It was the same issue. Mine worked with v 1.5.7.

cvat-ai / cvat