Closed yaoguang97 closed 2 years ago
@yaoguang97 , thanks for the report. I can reproduce the issue on our side. I will try to fix before 2.0.0 release.
@nmanovic, I pulled the latest code, rebuilt the YOLOv5 model, and the same error occurred.
@yaoguang97 , could you please send logs for the function? https://openvinotoolkit.github.io/cvat/docs/manual/advanced/serverless-tutorial/#troubleshooting
@nmanovic ,Hello, this is the log about the container.
22.02.18 03:05:03.843 processor (I) Starting processor {"version": "Label: 1.5.16, Git commit: ae43a6a560c2bec42d7ccfdf6e8e11a1e3cc3774, OS: linux, Arch: amd64, Go version: go1.14.3"}
22.02.18 03:05:03.843 processor (D) Read configuration {"config": "{\n \"metadata\": {\n \"name\": \"ultralytics-yolov5\",\n \"namespace\": \"nuclio\",\n \"labels\": {\n \"nuclio.io/project-name\": \"cvat\"\n },\n \"annotations\": {\n \"framework\": \"pytorch\",\n \"name\": \"YOLO v5\",\n \"spec\": \"[\\n { \\\"id\\\": 0, \\\"name\\\": \\\"person\\\" },\\n { \\\"id\\\": 1, \\\"name\\\": \\\"bicycle\\\" },\\n { \\\"id\\\": 2, \\\"name\\\": \\\"car\\\" },\\n { \\\"id\\\": 3, \\\"name\\\": \\\"motorbike\\\" },\\n { \\\"id\\\": 4, \\\"name\\\": \\\"aeroplane\\\" },\\n { \\\"id\\\": 5, \\\"name\\\": \\\"bus\\\" },\\n { \\\"id\\\": 6, \\\"name\\\": \\\"train\\\" },\\n { \\\"id\\\": 7, \\\"name\\\": \\\"truck\\\" },\\n { \\\"id\\\": 8, \\\"name\\\": \\\"boat\\\" },\\n { \\\"id\\\": 9, \\\"name\\\": \\\"traffic light\\\" },\\n { \\\"id\\\": 10, \\\"name\\\": \\\"fire hydrant\\\" },\\n { \\\"id\\\": 11, \\\"name\\\": \\\"stop sign\\\" },\\n { \\\"id\\\": 12, \\\"name\\\": \\\"parking meter\\\" },\\n { \\\"id\\\": 13, \\\"name\\\": \\\"bench\\\" },\\n { \\\"id\\\": 14, \\\"name\\\": \\\"bird\\\" },\\n { \\\"id\\\": 15, \\\"name\\\": \\\"cat\\\" },\\n { \\\"id\\\": 16, \\\"name\\\": \\\"dog\\\" },\\n { \\\"id\\\": 17, \\\"name\\\": \\\"horse\\\" },\\n { \\\"id\\\": 18, \\\"name\\\": \\\"sheep\\\" },\\n { \\\"id\\\": 19, \\\"name\\\": \\\"cow\\\" },\\n { \\\"id\\\": 20, \\\"name\\\": \\\"elephant\\\" },\\n { \\\"id\\\": 21, \\\"name\\\": \\\"bear\\\" },\\n { \\\"id\\\": 22, \\\"name\\\": \\\"zebra\\\" },\\n { \\\"id\\\": 23, \\\"name\\\": \\\"giraffe\\\" },\\n { \\\"id\\\": 24, \\\"name\\\": \\\"backpack\\\" },\\n { \\\"id\\\": 25, \\\"name\\\": \\\"umbrella\\\" },\\n { \\\"id\\\": 26, \\\"name\\\": \\\"handbag\\\" },\\n { \\\"id\\\": 27, \\\"name\\\": \\\"tie\\\" },\\n { \\\"id\\\": 28, \\\"name\\\": \\\"suitcase\\\" },\\n { \\\"id\\\": 29, \\\"name\\\": \\\"frisbee\\\" },\\n { \\\"id\\\": 30, \\\"name\\\": \\\"skis\\\" },\\n { \\\"id\\\": 31, \\\"name\\\": \\\"snowboard\\\" },\\n { \\\"id\\\": 32, \\\"name\\\": \\\"sports ball\\\" },\\n { \\\"id\\\": 33, \\\"name\\\": \\\"kite\\\" },\\n { \\\"id\\\": 34, \\\"name\\\": \\\"baseball bat\\\" },\\n { \\\"id\\\": 35, \\\"name\\\": \\\"baseball glove\\\" },\\n { \\\"id\\\": 36, \\\"name\\\": \\\"skateboard\\\" },\\n { \\\"id\\\": 37, \\\"name\\\": \\\"surfboard\\\" },\\n { \\\"id\\\": 38, \\\"name\\\": \\\"tennis racket\\\" },\\n { \\\"id\\\": 39, \\\"name\\\": \\\"bottle\\\" },\\n { \\\"id\\\": 40, \\\"name\\\": \\\"wine glass\\\" },\\n { \\\"id\\\": 41, \\\"name\\\": \\\"cup\\\" },\\n { \\\"id\\\": 42, \\\"name\\\": \\\"fork\\\" },\\n { \\\"id\\\": 43, \\\"name\\\": \\\"knife\\\" },\\n { \\\"id\\\": 44, \\\"name\\\": \\\"spoon\\\" },\\n { \\\"id\\\": 45, \\\"name\\\": \\\"bowl\\\" },\\n { \\\"id\\\": 46, \\\"name\\\": \\\"banana\\\" },\\n { \\\"id\\\": 47, \\\"name\\\": \\\"apple\\\" },\\n { \\\"id\\\": 48, \\\"name\\\": \\\"sandwich\\\" },\\n { \\\"id\\\": 49, \\\"name\\\": \\\"orange\\\" },\\n { \\\"id\\\": 50, \\\"name\\\": \\\"broccoli\\\" },\\n { \\\"id\\\": 51, \\\"name\\\": \\\"carrot\\\" },\\n { \\\"id\\\": 52, \\\"name\\\": \\\"hot dog\\\" },\\n { \\\"id\\\": 53, \\\"name\\\": \\\"pizza\\\" },\\n { \\\"id\\\": 54, \\\"name\\\": \\\"donut\\\" },\\n { \\\"id\\\": 55, \\\"name\\\": \\\"cake\\\" },\\n { \\\"id\\\": 56, \\\"name\\\": \\\"chair\\\" },\\n { \\\"id\\\": 57, \\\"name\\\": \\\"sofa\\\" },\\n { \\\"id\\\": 58, \\\"name\\\": \\\"pottedplant\\\" },\\n { \\\"id\\\": 59, \\\"name\\\": \\\"bed\\\" },\\n { \\\"id\\\": 60, \\\"name\\\": \\\"diningtable\\\" },\\n { \\\"id\\\": 61, \\\"name\\\": \\\"toilet\\\" },\\n { \\\"id\\\": 62, \\\"name\\\": \\\"tvmonitor\\\" },\\n { \\\"id\\\": 63, \\\"name\\\": \\\"laptop\\\" },\\n { \\\"id\\\": 64, \\\"name\\\": \\\"mouse\\\" },\\n { \\\"id\\\": 65, \\\"name\\\": \\\"remote\\\" },\\n { \\\"id\\\": 66, \\\"name\\\": \\\"keyboard\\\" },\\n { \\\"id\\\": 67, \\\"name\\\": \\\"cell phone\\\" },\\n { \\\"id\\\": 68, \\\"name\\\": \\\"microwave\\\" },\\n { \\\"id\\\": 69, \\\"name\\\": \\\"oven\\\" },\\n { \\\"id\\\": 70, \\\"name\\\": \\\"toaster\\\" },\\n { \\\"id\\\": 71, \\\"name\\\": \\\"sink\\\" },\\n { \\\"id\\\": 72, \\\"name\\\": \\\"refrigerator\\\" },\\n { \\\"id\\\": 73, \\\"name\\\": \\\"book\\\" },\\n { \\\"id\\\": 74, \\\"name\\\": \\\"clock\\\" },\\n { \\\"id\\\": 75, \\\"name\\\": \\\"vase\\\" },\\n { \\\"id\\\": 76, \\\"name\\\": \\\"scissors\\\" },\\n { \\\"id\\\": 77, \\\"name\\\": \\\"teddy bear\\\" },\\n { \\\"id\\\": 78, \\\"name\\\": \\\"hair drier\\\" },\\n { \\\"id\\\": 79, \\\"name\\\": \\\"toothbrush\\\" }\\n]\\n\",\n \"type\": \"detector\"\n }\n },\n \"spec\": {\n \"description\": \"YOLO v5 via pytorch hub\",\n \"handler\": \"main:handler\",\n \"runtime\": \"python:3.6\",\n \"resources\": {},\n \"image\": \"cvat/ultralytics-yolov5:latest\",\n \"targetCPU\": 75,\n \"triggers\": {\n \"myHttpTrigger\": {\n \"class\": \"\",\n \"kind\": \"http\",\n \"name\": \"myHttpTrigger\",\n \"maxWorkers\": 2,\n \"workerAvailabilityTimeoutMilliseconds\": 10000,\n \"attributes\": {\n \"maxRequestBodySize\": 33554432\n }\n }\n },\n \"volumes\": [\n {\n \"volume\": {\n \"name\": \"volume-1\",\n \"hostPath\": {\n \"path\": \"/home/yg/work/cvat/serverless/common\"\n }\n },\n \"volumeMount\": {\n \"name\": \"volume-1\",\n \"mountPath\": \"/opt/nuclio/common\"\n }\n }\n ],\n \"build\": {\n \"image\": \"cvat/ultralytics-yolov5\",\n \"baseImage\": \"ultralytics/yolov5:latest-cpu\",\n \"directives\": {\n \"preCopy\": [\n {\n \"kind\": \"USER\",\n \"value\": \"root\"\n },\n {\n \"kind\": \"RUN\",\n \"value\": \"apt update \\u0026\\u0026 apt install --no-install-recommends -y libglib2.0-0\"\n },\n {\n \"kind\": \"WORKDIR\",\n \"value\": \"/opt/nuclio\"\n }\n ]\n },\n \"codeEntryType\": \"image\",\n \"timestamp\": 1645153502\n },\n \"platform\": {\n \"attributes\": {\n \"mountMode\": \"volume\",\n \"restartPolicy\": {\n \"maximumRetryCount\": 3,\n \"name\": \"always\"\n }\n }\n },\n \"readinessTimeoutSeconds\": 60,\n \"securityContext\": {},\n \"eventTimeout\": \"30s\"\n },\n \"PlatformConfig\": null\n}", "platformConfig": "{\n \"kind\": \"local\",\n \"webAdmin\": {\n \"enabled\": true,\n \"listenAddress\": \":8081\"\n },\n \"healthCheck\": {\n \"enabled\": true,\n \"listenAddress\": \":8082\"\n },\n \"logger\": {\n \"sinks\": {\n \"stdout\": {\n \"kind\": \"stdout\"\n }\n },\n \"system\": [\n {\n \"level\": \"debug\",\n \"sink\": \"stdout\"\n }\n ],\n \"functions\": [\n {\n \"level\": \"debug\",\n \"sink\": \"stdout\"\n }\n ]\n },\n \"metrics\": {},\n \"scaleToZero\": {},\n \"autoScale\": {},\n \"cronTriggerCreationMode\": \"processor\",\n \"ingressConfig\": {},\n \"kube\": {\n \"defaultServiceType\": \"ClusterIP\"\n },\n \"imageRegistryOverrides\": {}\n}"}
22.02.18 03:05:03.843 cessor.healthcheck.server (I) Listening {"listenAddress": ":8082"}
22.02.18 03:05:03.843 processor.http (D) Creating worker pool {"num": 2}
22.02.18 03:05:03.843 sor.http.w1.python.logger (D) Creating listener socket {"path": "/tmp/nuclio-rpc-c87gpnscjqrjg335rio0.sock"}
22.02.18 03:05:03.844 sor.http.w1.python.logger (D) Using Python wrapper script path {"path": "/opt/nuclio/_nuclio_wrapper.py"}
22.02.18 03:05:03.844 sor.http.w1.python.logger (D) Using Python handler {"handler": "main:handler"}
22.02.18 03:05:03.844 sor.http.w1.python.logger (D) Using Python executable {"path": "/usr/bin/python3"}
22.02.18 03:05:03.844 sor.http.w1.python.logger (D) Setting PYTHONPATH {"value": "PYTHONPATH=/opt/nuclio"}
22.02.18 03:05:03.844 sor.http.w1.python.logger (D) Running wrapper {"command": "/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --socket-path /tmp/nuclio-rpc-c87gpnscjqrjg335rio0.sock --platform-kind local --namespace nuclio --worker-id 1 --trigger-kind http --trigger-name myHttpTrigger"}
22.02.18 03:05:03.844 sor.http.w0.python.logger (D) Creating listener socket {"path": "/tmp/nuclio-rpc-c87gpnscjqrjg335riog.sock"}
22.02.18 03:05:03.844 sor.http.w0.python.logger (D) Using Python wrapper script path {"path": "/opt/nuclio/_nuclio_wrapper.py"}
22.02.18 03:05:03.844 sor.http.w0.python.logger (D) Using Python handler {"handler": "main:handler"}
22.02.18 03:05:03.844 sor.http.w0.python.logger (D) Using Python executable {"path": "/usr/bin/python3"}
22.02.18 03:05:03.844 sor.http.w0.python.logger (D) Setting PYTHONPATH {"value": "PYTHONPATH=/opt/nuclio"}
22.02.18 03:05:03.844 sor.http.w0.python.logger (D) Running wrapper {"command": "/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --socket-path /tmp/nuclio-rpc-c87gpnscjqrjg335riog.sock --platform-kind local --namespace nuclio --worker-id 0 --trigger-kind http --trigger-name myHttpTrigger"}
l{"datetime": "2022-02-18 03:05:05,338", "level": "info", "message": "Init context... 0%", "with": {"worker_id": "1"}}
22.02.18 03:05:05.340 sor.http.w1.python.logger (I) Wrapper connected {"wid": 1, "pid": 10}
22.02.18 03:05:05.340 sor.http.w1.python.logger (D) Waiting for start
22.02.18 03:05:05.377 sor.http.w0.python.logger (I) Wrapper connected {"wid": 0, "pid": 11}
22.02.18 03:05:05.377 sor.http.w0.python.logger (D) Waiting for start
l{"datetime": "2022-02-18 03:05:05,377", "level": "info", "message": "Init context... 0%", "with": {"worker_id": "0"}}
l{"datetime": "2022-02-18 03:05:24,719", "level": "error", "message": "Exception raised while running init_context", "with": {"worker_id": "1"}}
l{"datetime": "2022-02-18 03:05:24,739", "level": "error", "message": "Caught unhandled exception while initializing", "with": {"err": "[Errno 104] Connection reset by peer", "traceback": "Traceback (most recent call last):\n File \"/opt/nuclio/_nuclio_wrapper.py\", line 343, in run_wrapper\n wrapper_instance = Wrapper(root_logger,\n File \"/opt/nuclio/_nuclio_wrapper.py\", line 80, in __init__\n getattr(entrypoint_module, 'init_context')(self._context)\n File \"/opt/nuclio/main.py\", line 11, in init_context\n model = torch.hub.load('ultralytics/yolov5', 'yolov5s') # or yolov5m, yolov5l, yolov5x, custom\n File \"/usr/local/lib/python3.8/dist-packages/torch/hub.py\", line 397, in load\n repo_or_dir = _get_cache_or_reload(repo_or_dir, force_reload, verbose, skip_validation)\n File \"/usr/local/lib/python3.8/dist-packages/torch/hub.py\", line 165, in _get_cache_or_reload\n repo_owner, repo_name, branch = _parse_repo_info(github)\n File \"/usr/local/lib/python3.8/dist-packages/torch/hub.py\", line 119, in _parse_repo_info\n with urlopen(f\"https://github.com/{repo_owner}/{repo_name}/tree/main/\"):\n File \"/usr/lib/python3.8/urllib/request.py\", line 222, in urlopen\n return opener.open(url, data, timeout)\n File \"/usr/lib/python3.8/urllib/request.py\", line 525, in open\n response = self._open(req, data)\n File \"/usr/lib/python3.8/urllib/request.py\", line 542, in _open\n result = self._call_chain(self.handle_open, protocol, protocol +\n File \"/usr/lib/python3.8/urllib/request.py\", line 502, in _call_chain\n result = func(*args)\n File \"/usr/lib/python3.8/urllib/request.py\", line 1397, in https_open\n return self.do_open(http.client.HTTPSConnection, req,\n File \"/usr/lib/python3.8/urllib/request.py\", line 1358, in do_open\n r = h.getresponse()\n File \"/usr/lib/python3.8/http/client.py\", line 1344, in getresponse\n response.begin()\n File \"/usr/lib/python3.8/http/client.py\", line 307, in begin\n version, status, reason = self._read_status()\n File \"/usr/lib/python3.8/http/client.py\", line 268, in _read_status\n line = str(self.fp.readline(_MAXLINE + 1), \"iso-8859-1\")\n File \"/usr/lib/python3.8/socket.py\", line 669, in readinto\n return self._sock.recv_into(b)\n File \"/usr/lib/python3.8/ssl.py\", line 1241, in recv_into\n return self.read(nbytes, buffer)\n File \"/usr/lib/python3.8/ssl.py\", line 1099, in read\n return self._sslobj.read(len, buffer)\nConnectionResetError: [Errno 104] Connection reset by peer\n", "worker_id": "1"}}
22.02.18 03:05:24.739 sor.http.w1.python.logger (W) Failed to read from connection {"err": "EOF"}
22.02.18 03:05:24.871 sor.http.w1.python.logger (E) Unexpected termination of child process {"error": null, "status": "exit status 1"}
panic: Wrapper process for worker 1 exited unexpectedly with: exit status 1
goroutine 22 [running]:
github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).watchWrapperProcess(0xc000270000)
/nuclio/pkg/processor/runtime/rpc/abstract.go:464 +0x576
created by github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).startWrapper
/nuclio/pkg/processor/runtime/rpc/abstract.go:236 +0x1b5
22.02.18 03:05:25.331 processor (I) Starting processor {"version": "Label: 1.5.16, Git commit: ae43a6a560c2bec42d7ccfdf6e8e11a1e3cc3774, OS: linux, Arch: amd64, Go version: go1.14.3"}
22.02.18 03:05:25.331 processor (D) Read configuration {"config": "{\n \"metadata\": {\n \"name\": \"ultralytics-yolov5\",\n \"namespace\": \"nuclio\",\n \"labels\": {\n \"nuclio.io/project-name\": \"cvat\"\n },\n \"annotations\": {\n \"framework\": \"pytorch\",\n \"name\": \"YOLO v5\",\n \"spec\": \"[\\n { \\\"id\\\": 0, \\\"name\\\": \\\"person\\\" },\\n { \\\"id\\\": 1, \\\"name\\\": \\\"bicycle\\\" },\\n { \\\"id\\\": 2, \\\"name\\\": \\\"car\\\" },\\n { \\\"id\\\": 3, \\\"name\\\": \\\"motorbike\\\" },\\n { \\\"id\\\": 4, \\\"name\\\": \\\"aeroplane\\\" },\\n { \\\"id\\\": 5, \\\"name\\\": \\\"bus\\\" },\\n { \\\"id\\\": 6, \\\"name\\\": \\\"train\\\" },\\n { \\\"id\\\": 7, \\\"name\\\": \\\"truck\\\" },\\n { \\\"id\\\": 8, \\\"name\\\": \\\"boat\\\" },\\n { \\\"id\\\": 9, \\\"name\\\": \\\"traffic light\\\" },\\n { \\\"id\\\": 10, \\\"name\\\": \\\"fire hydrant\\\" },\\n { \\\"id\\\": 11, \\\"name\\\": \\\"stop sign\\\" },\\n { \\\"id\\\": 12, \\\"name\\\": \\\"parking meter\\\" },\\n { \\\"id\\\": 13, \\\"name\\\": \\\"bench\\\" },\\n { \\\"id\\\": 14, \\\"name\\\": \\\"bird\\\" },\\n { \\\"id\\\": 15, \\\"name\\\": \\\"cat\\\" },\\n { \\\"id\\\": 16, \\\"name\\\": \\\"dog\\\" },\\n { \\\"id\\\": 17, \\\"name\\\": \\\"horse\\\" },\\n { \\\"id\\\": 18, \\\"name\\\": \\\"sheep\\\" },\\n { \\\"id\\\": 19, \\\"name\\\": \\\"cow\\\" },\\n { \\\"id\\\": 20, \\\"name\\\": \\\"elephant\\\" },\\n { \\\"id\\\": 21, \\\"name\\\": \\\"bear\\\" },\\n { \\\"id\\\": 22, \\\"name\\\": \\\"zebra\\\" },\\n { \\\"id\\\": 23, \\\"name\\\": \\\"giraffe\\\" },\\n { \\\"id\\\": 24, \\\"name\\\": \\\"backpack\\\" },\\n { \\\"id\\\": 25, \\\"name\\\": \\\"umbrella\\\" },\\n { \\\"id\\\": 26, \\\"name\\\": \\\"handbag\\\" },\\n { \\\"id\\\": 27, \\\"name\\\": \\\"tie\\\" },\\n { \\\"id\\\": 28, \\\"name\\\": \\\"suitcase\\\" },\\n { \\\"id\\\": 29, \\\"name\\\": \\\"frisbee\\\" },\\n { \\\"id\\\": 30, \\\"name\\\": \\\"skis\\\" },\\n { \\\"id\\\": 31, \\\"name\\\": \\\"snowboard\\\" },\\n { \\\"id\\\": 32, \\\"name\\\": \\\"sports ball\\\" },\\n { \\\"id\\\": 33, \\\"name\\\": \\\"kite\\\" },\\n { \\\"id\\\": 34, \\\"name\\\": \\\"baseball bat\\\" },\\n { \\\"id\\\": 35, \\\"name\\\": \\\"baseball glove\\\" },\\n { \\\"id\\\": 36, \\\"name\\\": \\\"skateboard\\\" },\\n { \\\"id\\\": 37, \\\"name\\\": \\\"surfboard\\\" },\\n { \\\"id\\\": 38, \\\"name\\\": \\\"tennis racket\\\" },\\n { \\\"id\\\": 39, \\\"name\\\": \\\"bottle\\\" },\\n { \\\"id\\\": 40, \\\"name\\\": \\\"wine glass\\\" },\\n { \\\"id\\\": 41, \\\"name\\\": \\\"cup\\\" },\\n { \\\"id\\\": 42, \\\"name\\\": \\\"fork\\\" },\\n { \\\"id\\\": 43, \\\"name\\\": \\\"knife\\\" },\\n { \\\"id\\\": 44, \\\"name\\\": \\\"spoon\\\" },\\n { \\\"id\\\": 45, \\\"name\\\": \\\"bowl\\\" },\\n { \\\"id\\\": 46, \\\"name\\\": \\\"banana\\\" },\\n { \\\"id\\\": 47, \\\"name\\\": \\\"apple\\\" },\\n { \\\"id\\\": 48, \\\"name\\\": \\\"sandwich\\\" },\\n { \\\"id\\\": 49, \\\"name\\\": \\\"orange\\\" },\\n { \\\"id\\\": 50, \\\"name\\\": \\\"broccoli\\\" },\\n { \\\"id\\\": 51, \\\"name\\\": \\\"carrot\\\" },\\n { \\\"id\\\": 52, \\\"name\\\": \\\"hot dog\\\" },\\n { \\\"id\\\": 53, \\\"name\\\": \\\"pizza\\\" },\\n { \\\"id\\\": 54, \\\"name\\\": \\\"donut\\\" },\\n { \\\"id\\\": 55, \\\"name\\\": \\\"cake\\\" },\\n { \\\"id\\\": 56, \\\"name\\\": \\\"chair\\\" },\\n { \\\"id\\\": 57, \\\"name\\\": \\\"sofa\\\" },\\n { \\\"id\\\": 58, \\\"name\\\": \\\"pottedplant\\\" },\\n { \\\"id\\\": 59, \\\"name\\\": \\\"bed\\\" },\\n { \\\"id\\\": 60, \\\"name\\\": \\\"diningtable\\\" },\\n { \\\"id\\\": 61, \\\"name\\\": \\\"toilet\\\" },\\n { \\\"id\\\": 62, \\\"name\\\": \\\"tvmonitor\\\" },\\n { \\\"id\\\": 63, \\\"name\\\": \\\"laptop\\\" },\\n { \\\"id\\\": 64, \\\"name\\\": \\\"mouse\\\" },\\n { \\\"id\\\": 65, \\\"name\\\": \\\"remote\\\" },\\n { \\\"id\\\": 66, \\\"name\\\": \\\"keyboard\\\" },\\n { \\\"id\\\": 67, \\\"name\\\": \\\"cell phone\\\" },\\n { \\\"id\\\": 68, \\\"name\\\": \\\"microwave\\\" },\\n { \\\"id\\\": 69, \\\"name\\\": \\\"oven\\\" },\\n { \\\"id\\\": 70, \\\"name\\\": \\\"toaster\\\" },\\n { \\\"id\\\": 71, \\\"name\\\": \\\"sink\\\" },\\n { \\\"id\\\": 72, \\\"name\\\": \\\"refrigerator\\\" },\\n { \\\"id\\\": 73, \\\"name\\\": \\\"book\\\" },\\n { \\\"id\\\": 74, \\\"name\\\": \\\"clock\\\" },\\n { \\\"id\\\": 75, \\\"name\\\": \\\"vase\\\" },\\n { \\\"id\\\": 76, \\\"name\\\": \\\"scissors\\\" },\\n { \\\"id\\\": 77, \\\"name\\\": \\\"teddy bear\\\" },\\n { \\\"id\\\": 78, \\\"name\\\": \\\"hair drier\\\" },\\n { \\\"id\\\": 79, \\\"name\\\": \\\"toothbrush\\\" }\\n]\\n\",\n \"type\": \"detector\"\n }\n },\n \"spec\": {\n \"description\": \"YOLO v5 via pytorch hub\",\n \"handler\": \"main:handler\",\n \"runtime\": \"python:3.6\",\n \"resources\": {},\n \"image\": \"cvat/ultralytics-yolov5:latest\",\n \"targetCPU\": 75,\n \"triggers\": {\n \"myHttpTrigger\": {\n \"class\": \"\",\n \"kind\": \"http\",\n \"name\": \"myHttpTrigger\",\n \"maxWorkers\": 2,\n \"workerAvailabilityTimeoutMilliseconds\": 10000,\n \"attributes\": {\n \"maxRequestBodySize\": 33554432\n }\n }\n },\n \"volumes\": [\n {\n \"volume\": {\n \"name\": \"volume-1\",\n \"hostPath\": {\n \"path\": \"/home/yg/work/cvat/serverless/common\"\n }\n },\n \"volumeMount\": {\n \"name\": \"volume-1\",\n \"mountPath\": \"/opt/nuclio/common\"\n }\n }\n ],\n \"build\": {\n \"image\": \"cvat/ultralytics-yolov5\",\n \"baseImage\": \"ultralytics/yolov5:latest-cpu\",\n \"directives\": {\n \"preCopy\": [\n {\n \"kind\": \"USER\",\n \"value\": \"root\"\n },\n {\n \"kind\": \"RUN\",\n \"value\": \"apt update \\u0026\\u0026 apt install --no-install-recommends -y libglib2.0-0\"\n },\n {\n \"kind\": \"WORKDIR\",\n \"value\": \"/opt/nuclio\"\n }\n ]\n },\n \"codeEntryType\": \"image\",\n \"timestamp\": 1645153502\n },\n \"platform\": {\n \"attributes\": {\n \"mountMode\": \"volume\",\n \"restartPolicy\": {\n \"maximumRetryCount\": 3,\n \"name\": \"always\"\n }\n }\n },\n \"readinessTimeoutSeconds\": 60,\n \"securityContext\": {},\n \"eventTimeout\": \"30s\"\n },\n \"PlatformConfig\": null\n}", "platformConfig": "{\n \"kind\": \"local\",\n \"webAdmin\": {\n \"enabled\": true,\n \"listenAddress\": \":8081\"\n },\n \"healthCheck\": {\n \"enabled\": true,\n \"listenAddress\": \":8082\"\n },\n \"logger\": {\n \"sinks\": {\n \"stdout\": {\n \"kind\": \"stdout\"\n }\n },\n \"system\": [\n {\n \"level\": \"debug\",\n \"sink\": \"stdout\"\n }\n ],\n \"functions\": [\n {\n \"level\": \"debug\",\n \"sink\": \"stdout\"\n }\n ]\n },\n \"metrics\": {},\n \"scaleToZero\": {},\n \"autoScale\": {},\n \"cronTriggerCreationMode\": \"processor\",\n \"ingressConfig\": {},\n \"kube\": {\n \"defaultServiceType\": \"ClusterIP\"\n },\n \"imageRegistryOverrides\": {}\n}"}
22.02.18 03:05:25.331 cessor.healthcheck.server (I) Listening {"listenAddress": ":8082"}
22.02.18 03:05:25.331 processor.http (D) Creating worker pool {"num": 2}
22.02.18 03:05:25.331 sor.http.w1.python.logger (D) Creating listener socket {"path": "/tmp/nuclio-rpc-c87gptccjqrjg32q7kf0.sock"}
22.02.18 03:05:25.331 sor.http.w0.python.logger (D) Creating listener socket {"path": "/tmp/nuclio-rpc-c87gptccjqrjg32q7kfg.sock"}
22.02.18 03:05:25.331 sor.http.w1.python.logger (D) Using Python wrapper script path {"path": "/opt/nuclio/_nuclio_wrapper.py"}
22.02.18 03:05:25.331 sor.http.w1.python.logger (D) Using Python handler {"handler": "main:handler"}
22.02.18 03:05:25.331 sor.http.w0.python.logger (D) Using Python wrapper script path {"path": "/opt/nuclio/_nuclio_wrapper.py"}
22.02.18 03:05:25.331 sor.http.w1.python.logger (D) Using Python executable {"path": "/usr/bin/python3"}
22.02.18 03:05:25.331 sor.http.w0.python.logger (D) Using Python handler {"handler": "main:handler"}
22.02.18 03:05:25.331 sor.http.w1.python.logger (D) Setting PYTHONPATH {"value": "PYTHONPATH=/opt/nuclio"}
22.02.18 03:05:25.331 sor.http.w1.python.logger (D) Running wrapper {"command": "/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --socket-path /tmp/nuclio-rpc-c87gptccjqrjg32q7kf0.sock --platform-kind local --namespace nuclio --worker-id 1 --trigger-kind http --trigger-name myHttpTrigger"}
22.02.18 03:05:25.331 sor.http.w0.python.logger (D) Using Python executable {"path": "/usr/bin/python3"}
22.02.18 03:05:25.331 sor.http.w0.python.logger (D) Setting PYTHONPATH {"value": "PYTHONPATH=/opt/nuclio"}
22.02.18 03:05:25.331 sor.http.w0.python.logger (D) Running wrapper {"command": "/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --socket-path /tmp/nuclio-rpc-c87gptccjqrjg32q7kfg.sock --platform-kind local --namespace nuclio --worker-id 0 --trigger-kind http --trigger-name myHttpTrigger"}
22.02.18 03:05:26.590 sor.http.w0.python.logger (I) Wrapper connected {"wid": 0, "pid": 13}
22.02.18 03:05:26.591 sor.http.w0.python.logger (D) Waiting for start
l{"datetime": "2022-02-18 03:05:26,591", "level": "info", "message": "Init context... 0%", "with": {"worker_id": "0"}}
22.02.18 03:05:26.634 sor.http.w1.python.logger (I) Wrapper connected {"wid": 1, "pid": 12}
l{"datetime": "2022-02-18 03:05:26,634", "level": "info", "message": "Init context... 0%", "with": {"worker_id": "1"}}
22.02.18 03:05:26.634 sor.http.w1.python.logger (D) Waiting for start
Downloading: "https://github.com/ultralytics/yolov5/archive/master.zip" to /root/.cache/torch/hub/master.zip
Downloading: "https://github.com/ultralytics/yolov5/archive/master.zip" to /root/.cache/torch/hub/master.zip
@yaoguang97 ,
From your logs I see that model = torch.hub.load('ultralytics/yolov5', 'yolov5s')
leads to an exception. It tries to download the model locally and cannot. Please check your network settings.
l{"datetime": "2022-02-18 03:05:24,739", "level": "error", "message": "Caught unhandled exception while initializing", "with": {"err": "[Errno 104] Connection reset by peer", "traceback": "Traceback (most recent call last):\n File \"/opt/nuclio/_nuclio_wrapper.py\", line 343, in run_wrapper\n wrapper_instance = Wrapper(root_logger,\n File \"/opt/nuclio/_nuclio_wrapper.py\", line 80, in __init__\n getattr(entrypoint_module, 'init_context')(self._context)\n File \"/opt/nuclio/main.py\", line 11, in init_context\n model = torch.hub.load('ultralytics/yolov5', 'yolov5s') # or yolov5m, yolov5l, yolov5x, custom\n File \"/usr/local/lib/python3.8/dist-packages/torch/hub.py\", line 397, in load\n repo_or_dir = _get_cache_or_reload(repo_or_dir, force_reload, verbose, skip_validation)\n File \"/usr/local/lib/python3.8/dist-packages/torch/hub.py\", line 165, in _get_cache_or_reload\n repo_owner, repo_name, branch = _parse_repo_info(github)\n File \"/usr/local/lib/python3.8/dist-packages/torch/hub.py\", line 119, in _parse_repo_info\n with urlopen(f\"https://github.com/{repo_owner}/{repo_name}/tree/main/\"):\n File \"/usr/lib/python3.8/urllib/request.py\", line 222, in urlopen\n return opener.open(url, data, timeout)\n File \"/usr/lib/python3.8/urllib/request.py\", line 525, in open\n response = self._open(req, data)\n File \"/usr/lib/python3.8/urllib/request.py\", line 542, in _open\n result = self._call_chain(self.handle_open, protocol, protocol +\n File \"/usr/lib/python3.8/urllib/request.py\", line 502, in _call_chain\n result = func(*args)\n File \"/usr/lib/python3.8/urllib/request.py\", line 1397, in https_open\n return self.do_open(http.client.HTTPSConnection, req,\n File \"/usr/lib/python3.8/urllib/request.py\", line 1358, in do_open\n r = h.getresponse()\n File \"/usr/lib/python3.8/http/client.py\", line 1344, in getresponse\n response.begin()\n File \"/usr/lib/python3.8/http/client.py\", line 307, in begin\n version, status, reason = self._read_status()\n File \"/usr/lib/python3.8/http/client.py\", line 268, in _read_status\n line = str(self.fp.readline(_MAXLINE + 1), \"iso-8859-1\")\n File \"/usr/lib/python3.8/socket.py\", line 669, in readinto\n return self._sock.recv_into(b)\n File \"/usr/lib/python3.8/ssl.py\", line 1241, in recv_into\n return self.read(nbytes, buffer)\n File \"/usr/lib/python3.8/ssl.py\", line 1099, in read\n return self._sslobj.read(len, buffer)\nConnectionResetError: [Errno 104] Connection reset by peer\n", "worker_id": "1"}}
22.02.18 03:05:24.739 sor.http.w1.python.logger (W) Failed to read from connection {"err": "EOF"}
Any progress on this? For me, it fails as well. Network connection is OK.
Help for that @nmanovic
@yasakova-anastasia , could you please provide the current status?
i have the same problem did some one fix it
Environment:ubuntu20.04,nuctl1.5.16