search

pytorch / serve

Serve, optimize and scale PyTorch models in production
https://pytorch.org/serve/
Apache License 2.0
4.22k stars 863 forks source link

Real-time Object Detection #3030

Closed hungtrieu07 closed 7 months ago

hungtrieu07 commented 8 months ago

Hi guys, I have a question: Can TorchServe run detect task in real-time? I developed a surveillance camera app written in Python with input are camera, inside the app, I send the frame from camera to TorchServe with multi-thread. When TorchServe responses with object bounding box and score, I will draw them on frame. But the result is too slow, even when I use a GPU RTX 3060, the FPS about 4-5 FPS. The response of TorchServe is about 30-50 ms.

namannandan commented 8 months ago

@hungtrieu07 To help improve throughput, consider using batching. Here's an example which shows how to batch frames for near real-time video inference: https://github.com/pytorch/serve/blob/master/examples/image_classifier/near_real_time_video/README.md

hungtrieu07 commented 8 months ago

@hungtrieu07 To help improve throughput, consider using batching. Here's an example which shows how to batch frames for near real-time video inference: https://github.com/pytorch/serve/blob/master/examples/image_classifier/near_real_time_video/README.md

Thank you, this is a good idea, I will work on it now.

hungtrieu07 commented 8 months ago

Hi @namannandan , thanks for your information. Now I'm working on my model (YOLOv8), but they are in ONNX format with dynamic batching. Do we have any example about this?

This is my handler:

import base64
import onnxruntime
import torch
from torchvision.transforms.functional import to_tensor
from ts.torch_handler.base_handler import BaseHandler
from PIL import Image
import io
from ultils_v8 import *

class FaceDetection(BaseHandler):

    def initialize(self, context):
        self._context = context
        self.initialized = False

        # Load the ONNX model
        model_dir = context.system_properties.get("model_dir")
        model_file_path = model_dir + "/yolov8n-face-batch.onnx"  # Adjust 'model.onnx' to your model's filename
        self.session = onnxruntime.InferenceSession(model_file_path)
        self.initialized = True

    def preprocess(self, data):
        images = []

        for row in data:
            image = row.get("data") or row.get("body")

            if isinstance(image, str):
                # if the image is a string of bytesarray.
                image = base64.b64decode(image)

            # If the image is sent as bytesarray
            if isinstance(image, (bytearray, bytes)):
                image = Image.open(io.BytesIO(image))
            else:
                print(image.shape)
                # if the image is a list
                image = torch.FloatTensor(image)

            image = image.resize((640, 640))
            image = to_tensor(image)
            images.append(image)

        return torch.stack(images)  # Create a batch

    def inference(self, batch_data):
        # ONNX runtime expects numpy arrays
        batch_data = batch_data.numpy().astype(np.float32)
        inputs = {self.session.get_inputs()[0].name: batch_data}
        outputs = self.session.run(None, inputs)
        return outputs

    def postprocess(self, inference_output):
        # just processing the result, not done yet
        results = []

        for batch in inference_output:
            # print(batch)
            # print(batch.shape)
            bbox_coords, confidences = batch[:4], batch[4]
            print(bbox_coords)
            print(confidences)

        return results
lxning commented 8 months ago

@hungtrieu07 onnx is auto-detected by basehandler. (see code). You only need make sure onnx is installed.

hungtrieu07 commented 7 months ago

Hi @lxning, I'm using BaseHandler as you tell above, but I saw this error when run inference an image: AttributeError: 'NoneType' object has no attribute 'model_yaml_config'

My model are converted to ONNX format, and using BaseHandler class as handler file.

This is my full ts_log.log:

2024-04-09T10:34:16,954 [WARN ] main org.pytorch.serve.util.ConfigManager - Your torchserve instance can access any URL to load models. When deploying to production, make sure to limit the set of allowed_urls in config.properties
2024-04-09T10:34:16,954 [WARN ] main org.pytorch.serve.util.ConfigManager - Your torchserve instance can access any URL to load models. When deploying to production, make sure to limit the set of allowed_urls in config.properties
2024-04-09T10:34:16,958 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager - Initializing plugins manager...
2024-04-09T10:34:16,958 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager - Initializing plugins manager...
2024-04-09T10:34:17,041 [INFO ] main org.pytorch.serve.metrics.configuration.MetricConfiguration - Successfully loaded metrics configuration from /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml
2024-04-09T10:34:17,041 [INFO ] main org.pytorch.serve.metrics.configuration.MetricConfiguration - Successfully loaded metrics configuration from /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml
2024-04-09T10:34:17,213 [INFO ] main org.pytorch.serve.ModelServer - 
Torchserve version: 0.9.0
TS Home: /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages
Current directory: /mnt/d/dev/api_torchserve_lancs
Temp directory: /tmp
Metrics config path: /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml
Number of GPUs: 0
Number of CPUs: 8
Max heap size: 1970 M
Python executable: /home/hungtrieu07/miniconda3/envs/api_lanc/bin/python
Config file: config.properties
Inference address: http://0.0.0.0:8080
Management address: http://0.0.0.0:8081
Metrics address: http://0.0.0.0:8082
Model Store: /mnt/d/dev/api_torchserve_lancs/model-store
Initial Models: all
Log dir: /mnt/d/dev/api_torchserve_lancs/logs
Metrics dir: /mnt/d/dev/api_torchserve_lancs/logs
Netty threads: 0
Netty client threads: 0
Default workers per model: 8
Blacklist Regex: N/A
Maximum Response Size: 6553500
Maximum Request Size: 6553500
Limit Maximum Image Pixels: true
Prefer direct buffer: false
Allowed Urls: [file://.*|http(s)?://.*]
Custom python dependency for model allowed: true
Enable metrics API: true
Metrics mode: log
Disable system metrics: false
Workflow Store: /mnt/d/dev/api_torchserve_lancs/model-store
Model config: {"FaceDetection": {"1.0": {"defaultVersion": true,"marName": "FaceDetection.mar","minWorkers": 1,"maxWorkers": 1,"batchSize": 256,"maxBatchDelay": 100,"responseTimeout": 120}},"FaceExpression": {"1.0": {"defaultVersion": true,"marName": "FaceExpression.mar","minWorkers": 1,"maxWorkers": 4,"batchSize": 256,"maxBatchDelay": 100,"responseTimeout": 120}},"FaceRecognition": {"1.0": {"defaultVersion": true,"marName": "FaceRecognition.mar","minWorkers": 1,"maxWorkers": 4,"batchSize": 256,"maxBatchDelay": 100,"responseTimeout": 120}},"HumanPose": {"1.0": {"defaultVersion": true,"marName": "HumanPose.mar","minWorkers": 1,"maxWorkers": 4,"batchSize": 256,"maxBatchDelay": 100,"responseTimeout": 120}},"ActionRecognition": {"1.0": {"defaultVersion": true,"marName": "ActionRecognition.mar","minWorkers": 1,"maxWorkers": 4,"batchSize": 256,"maxBatchDelay": 100,"responseTimeout": 120}}}
2024-04-09T10:34:17,213 [INFO ] main org.pytorch.serve.ModelServer - 
Torchserve version: 0.9.0
TS Home: /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages
Current directory: /mnt/d/dev/api_torchserve_lancs
Temp directory: /tmp
Metrics config path: /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml
Number of GPUs: 0
Number of CPUs: 8
Max heap size: 1970 M
Python executable: /home/hungtrieu07/miniconda3/envs/api_lanc/bin/python
Config file: config.properties
Inference address: http://0.0.0.0:8080
Management address: http://0.0.0.0:8081
Metrics address: http://0.0.0.0:8082
Model Store: /mnt/d/dev/api_torchserve_lancs/model-store
Initial Models: all
Log dir: /mnt/d/dev/api_torchserve_lancs/logs
Metrics dir: /mnt/d/dev/api_torchserve_lancs/logs
Netty threads: 0
Netty client threads: 0
Default workers per model: 8
Blacklist Regex: N/A
Maximum Response Size: 6553500
Maximum Request Size: 6553500
Limit Maximum Image Pixels: true
Prefer direct buffer: false
Allowed Urls: [file://.*|http(s)?://.*]
Custom python dependency for model allowed: true
Enable metrics API: true
Metrics mode: log
Disable system metrics: false
Workflow Store: /mnt/d/dev/api_torchserve_lancs/model-store
Model config: {"FaceDetection": {"1.0": {"defaultVersion": true,"marName": "FaceDetection.mar","minWorkers": 1,"maxWorkers": 1,"batchSize": 256,"maxBatchDelay": 100,"responseTimeout": 120}},"FaceExpression": {"1.0": {"defaultVersion": true,"marName": "FaceExpression.mar","minWorkers": 1,"maxWorkers": 4,"batchSize": 256,"maxBatchDelay": 100,"responseTimeout": 120}},"FaceRecognition": {"1.0": {"defaultVersion": true,"marName": "FaceRecognition.mar","minWorkers": 1,"maxWorkers": 4,"batchSize": 256,"maxBatchDelay": 100,"responseTimeout": 120}},"HumanPose": {"1.0": {"defaultVersion": true,"marName": "HumanPose.mar","minWorkers": 1,"maxWorkers": 4,"batchSize": 256,"maxBatchDelay": 100,"responseTimeout": 120}},"ActionRecognition": {"1.0": {"defaultVersion": true,"marName": "ActionRecognition.mar","minWorkers": 1,"maxWorkers": 4,"batchSize": 256,"maxBatchDelay": 100,"responseTimeout": 120}}}
2024-04-09T10:34:17,227 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager -  Loading snapshot serializer plugin...
2024-04-09T10:34:17,227 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager -  Loading snapshot serializer plugin...
2024-04-09T10:34:17,238 [DEBUG] main org.pytorch.serve.ModelServer - Loading models from model store: ActionRecognition.mar
2024-04-09T10:34:17,238 [DEBUG] main org.pytorch.serve.ModelServer - Loading models from model store: ActionRecognition.mar
2024-04-09T10:34:17,483 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model ActionRecognition
2024-04-09T10:34:17,483 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model ActionRecognition
2024-04-09T10:34:17,484 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model ActionRecognition
2024-04-09T10:34:17,484 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model ActionRecognition
2024-04-09T10:34:17,484 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model ActionRecognition loaded.
2024-04-09T10:34:17,484 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model ActionRecognition loaded.
2024-04-09T10:34:17,485 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: ActionRecognition, count: 1
2024-04-09T10:34:17,485 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: ActionRecognition, count: 1
2024-04-09T10:34:17,499 [DEBUG] W-9000-ActionRecognition_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:34:17,499 [DEBUG] main org.pytorch.serve.ModelServer - Loading models from model store: FaceDetection.mar
2024-04-09T10:34:17,499 [DEBUG] W-9000-ActionRecognition_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:34:17,499 [DEBUG] main org.pytorch.serve.ModelServer - Loading models from model store: FaceDetection.mar
2024-04-09T10:34:19,198 [INFO ] W-9000-ActionRecognition_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=91650
2024-04-09T10:34:19,200 [INFO ] W-9000-ActionRecognition_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2024-04-09T10:34:19,206 [INFO ] W-9000-ActionRecognition_1.0-stdout MODEL_LOG - Successfully loaded /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2024-04-09T10:34:19,207 [INFO ] W-9000-ActionRecognition_1.0-stdout MODEL_LOG - [PID]91650
2024-04-09T10:34:19,208 [INFO ] W-9000-ActionRecognition_1.0-stdout MODEL_LOG - Torch worker started.
2024-04-09T10:34:19,209 [DEBUG] W-9000-ActionRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-ActionRecognition_1.0 State change null -> WORKER_STARTED
2024-04-09T10:34:19,209 [DEBUG] W-9000-ActionRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-ActionRecognition_1.0 State change null -> WORKER_STARTED
2024-04-09T10:34:19,209 [INFO ] W-9000-ActionRecognition_1.0-stdout MODEL_LOG - Python runtime: 3.8.19
2024-04-09T10:34:19,230 [INFO ] W-9000-ActionRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2024-04-09T10:34:19,230 [INFO ] W-9000-ActionRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2024-04-09T10:34:19,319 [INFO ] W-9000-ActionRecognition_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2024-04-09T10:34:19,323 [INFO ] W-9000-ActionRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633659323
2024-04-09T10:34:19,323 [INFO ] W-9000-ActionRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633659323
2024-04-09T10:34:19,367 [INFO ] W-9000-ActionRecognition_1.0-stdout MODEL_LOG - model_name: ActionRecognition, batchSize: 256
2024-04-09T10:34:20,012 [INFO ] W-9000-ActionRecognition_1.0-stdout MODEL_LOG - ONNX enabled
2024-04-09T10:34:20,013 [INFO ] W-9000-ActionRecognition_1.0-stdout MODEL_LOG - Torch TensorRT not enabled
2024-04-09T10:34:20,485 [DEBUG] W-9000-ActionRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:34:20,485 [DEBUG] W-9000-ActionRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:34:20,485 [INFO ] W-9000-ActionRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1117
2024-04-09T10:34:20,485 [INFO ] W-9000-ActionRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1117
2024-04-09T10:34:20,486 [DEBUG] W-9000-ActionRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-ActionRecognition_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:34:20,486 [DEBUG] W-9000-ActionRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-ActionRecognition_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:34:20,488 [INFO ] W-9000-ActionRecognition_1.0 TS_METRICS - WorkerLoadTime.Milliseconds:2991.0|#WorkerName:W-9000-ActionRecognition_1.0,Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633660
2024-04-09T10:34:20,489 [INFO ] W-9000-ActionRecognition_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:49.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633660
2024-04-09T10:34:20,717 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model FaceDetection
2024-04-09T10:34:20,717 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model FaceDetection
2024-04-09T10:34:20,718 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model FaceDetection
2024-04-09T10:34:20,718 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model FaceDetection
2024-04-09T10:34:20,719 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model FaceDetection loaded.
2024-04-09T10:34:20,719 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model FaceDetection loaded.
2024-04-09T10:34:20,719 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: FaceDetection, count: 1
2024-04-09T10:34:20,719 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: FaceDetection, count: 1
2024-04-09T10:34:20,721 [DEBUG] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9001, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:34:20,721 [DEBUG] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9001, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:34:20,722 [DEBUG] main org.pytorch.serve.ModelServer - Loading models from model store: FaceExpression.mar
2024-04-09T10:34:20,722 [DEBUG] main org.pytorch.serve.ModelServer - Loading models from model store: FaceExpression.mar
2024-04-09T10:34:22,192 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9001, pid=91696
2024-04-09T10:34:22,194 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9001
2024-04-09T10:34:22,203 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - Successfully loaded /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2024-04-09T10:34:22,204 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - [PID]91696
2024-04-09T10:34:22,204 [DEBUG] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9001-FaceDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:34:22,204 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - Torch worker started.
2024-04-09T10:34:22,204 [DEBUG] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9001-FaceDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:34:22,205 [INFO ] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9001
2024-04-09T10:34:22,205 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - Python runtime: 3.8.19
2024-04-09T10:34:22,205 [INFO ] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9001
2024-04-09T10:34:22,208 [INFO ] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633662208
2024-04-09T10:34:22,208 [INFO ] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633662208
2024-04-09T10:34:22,208 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9001.
2024-04-09T10:34:22,224 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - model_name: FaceDetection, batchSize: 256
2024-04-09T10:34:22,394 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - ONNX enabled
2024-04-09T10:34:22,394 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - Torch TensorRT not enabled
2024-04-09T10:34:22,474 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - Succesfully setup ort session
2024-04-09T10:34:22,475 [DEBUG] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:34:22,475 [DEBUG] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:34:22,475 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - '/tmp/models/514d97655cd546e9ae0fbdbd23c166bc/index_to_name.json' is missing. Inference output will not include class name.
2024-04-09T10:34:22,476 [INFO ] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 251
2024-04-09T10:34:22,476 [INFO ] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 251
2024-04-09T10:34:22,477 [DEBUG] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9001-FaceDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:34:22,477 [DEBUG] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9001-FaceDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:34:22,477 [INFO ] W-9001-FaceDetection_1.0 TS_METRICS - WorkerLoadTime.Milliseconds:1757.0|#WorkerName:W-9001-FaceDetection_1.0,Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633662
2024-04-09T10:34:22,478 [INFO ] W-9001-FaceDetection_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:19.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633662
2024-04-09T10:34:26,884 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model FaceExpression
2024-04-09T10:34:26,884 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model FaceExpression
2024-04-09T10:34:26,885 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model FaceExpression
2024-04-09T10:34:26,885 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model FaceExpression
2024-04-09T10:34:26,885 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model FaceExpression loaded.
2024-04-09T10:34:26,885 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model FaceExpression loaded.
2024-04-09T10:34:26,886 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: FaceExpression, count: 1
2024-04-09T10:34:26,886 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: FaceExpression, count: 1
2024-04-09T10:34:26,887 [DEBUG] W-9002-FaceExpression_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9002, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:34:26,887 [DEBUG] W-9002-FaceExpression_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9002, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:34:26,888 [DEBUG] main org.pytorch.serve.ModelServer - Loading models from model store: FaceRecognition.mar
2024-04-09T10:34:26,888 [DEBUG] main org.pytorch.serve.ModelServer - Loading models from model store: FaceRecognition.mar
2024-04-09T10:34:28,040 [INFO ] W-9002-FaceExpression_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9002, pid=91765
2024-04-09T10:34:28,041 [INFO ] W-9002-FaceExpression_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9002
2024-04-09T10:34:28,047 [INFO ] W-9002-FaceExpression_1.0-stdout MODEL_LOG - Successfully loaded /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2024-04-09T10:34:28,049 [INFO ] W-9002-FaceExpression_1.0-stdout MODEL_LOG - [PID]91765
2024-04-09T10:34:28,049 [DEBUG] W-9002-FaceExpression_1.0 org.pytorch.serve.wlm.WorkerThread - W-9002-FaceExpression_1.0 State change null -> WORKER_STARTED
2024-04-09T10:34:28,049 [INFO ] W-9002-FaceExpression_1.0-stdout MODEL_LOG - Torch worker started.
2024-04-09T10:34:28,049 [DEBUG] W-9002-FaceExpression_1.0 org.pytorch.serve.wlm.WorkerThread - W-9002-FaceExpression_1.0 State change null -> WORKER_STARTED
2024-04-09T10:34:28,050 [INFO ] W-9002-FaceExpression_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9002
2024-04-09T10:34:28,050 [INFO ] W-9002-FaceExpression_1.0-stdout MODEL_LOG - Python runtime: 3.8.19
2024-04-09T10:34:28,050 [INFO ] W-9002-FaceExpression_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9002
2024-04-09T10:34:28,053 [INFO ] W-9002-FaceExpression_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633668053
2024-04-09T10:34:28,053 [INFO ] W-9002-FaceExpression_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633668053
2024-04-09T10:34:28,053 [INFO ] W-9002-FaceExpression_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9002.
2024-04-09T10:34:28,069 [INFO ] W-9002-FaceExpression_1.0-stdout MODEL_LOG - model_name: FaceExpression, batchSize: 256
2024-04-09T10:34:28,512 [INFO ] W-9002-FaceExpression_1.0-stdout MODEL_LOG - ONNX enabled
2024-04-09T10:34:28,513 [INFO ] W-9002-FaceExpression_1.0-stdout MODEL_LOG - Torch TensorRT not enabled
2024-04-09T10:34:28,623 [DEBUG] W-9002-FaceExpression_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:34:28,623 [DEBUG] W-9002-FaceExpression_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:34:28,624 [INFO ] W-9002-FaceExpression_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 554
2024-04-09T10:34:28,624 [INFO ] W-9002-FaceExpression_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 554
2024-04-09T10:34:28,624 [DEBUG] W-9002-FaceExpression_1.0 org.pytorch.serve.wlm.WorkerThread - W-9002-FaceExpression_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:34:28,624 [DEBUG] W-9002-FaceExpression_1.0 org.pytorch.serve.wlm.WorkerThread - W-9002-FaceExpression_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:34:28,625 [INFO ] W-9002-FaceExpression_1.0 TS_METRICS - WorkerLoadTime.Milliseconds:1739.0|#WorkerName:W-9002-FaceExpression_1.0,Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633668
2024-04-09T10:34:28,626 [INFO ] W-9002-FaceExpression_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:19.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633668
2024-04-09T10:34:56,181 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model FaceRecognition
2024-04-09T10:34:56,181 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model FaceRecognition
2024-04-09T10:34:56,181 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model FaceRecognition
2024-04-09T10:34:56,181 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model FaceRecognition
2024-04-09T10:34:56,182 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model FaceRecognition loaded.
2024-04-09T10:34:56,182 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model FaceRecognition loaded.
2024-04-09T10:34:56,182 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: FaceRecognition, count: 1
2024-04-09T10:34:56,182 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: FaceRecognition, count: 1
2024-04-09T10:34:56,184 [DEBUG] W-9003-FaceRecognition_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9003, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:34:56,184 [DEBUG] W-9003-FaceRecognition_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9003, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:34:56,185 [DEBUG] main org.pytorch.serve.ModelServer - Loading models from model store: HumanPose.mar
2024-04-09T10:34:56,185 [DEBUG] main org.pytorch.serve.ModelServer - Loading models from model store: HumanPose.mar
2024-04-09T10:34:57,432 [INFO ] W-9003-FaceRecognition_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9003, pid=91890
2024-04-09T10:34:57,434 [INFO ] W-9003-FaceRecognition_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9003
2024-04-09T10:34:57,443 [INFO ] W-9003-FaceRecognition_1.0-stdout MODEL_LOG - Successfully loaded /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2024-04-09T10:34:57,446 [INFO ] W-9003-FaceRecognition_1.0-stdout MODEL_LOG - [PID]91890
2024-04-09T10:34:57,447 [DEBUG] W-9003-FaceRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - W-9003-FaceRecognition_1.0 State change null -> WORKER_STARTED
2024-04-09T10:34:57,447 [INFO ] W-9003-FaceRecognition_1.0-stdout MODEL_LOG - Torch worker started.
2024-04-09T10:34:57,447 [DEBUG] W-9003-FaceRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - W-9003-FaceRecognition_1.0 State change null -> WORKER_STARTED
2024-04-09T10:34:57,448 [INFO ] W-9003-FaceRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9003
2024-04-09T10:34:57,448 [INFO ] W-9003-FaceRecognition_1.0-stdout MODEL_LOG - Python runtime: 3.8.19
2024-04-09T10:34:57,448 [INFO ] W-9003-FaceRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9003
2024-04-09T10:34:57,450 [INFO ] W-9003-FaceRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633697450
2024-04-09T10:34:57,450 [INFO ] W-9003-FaceRecognition_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9003.
2024-04-09T10:34:57,450 [INFO ] W-9003-FaceRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633697450
2024-04-09T10:34:57,478 [INFO ] W-9003-FaceRecognition_1.0-stdout MODEL_LOG - model_name: FaceRecognition, batchSize: 256
2024-04-09T10:34:57,942 [INFO ] W-9003-FaceRecognition_1.0-stdout MODEL_LOG - ONNX enabled
2024-04-09T10:34:57,942 [INFO ] W-9003-FaceRecognition_1.0-stdout MODEL_LOG - Torch TensorRT not enabled
2024-04-09T10:34:59,763 [DEBUG] W-9003-FaceRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:34:59,763 [DEBUG] W-9003-FaceRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:34:59,764 [INFO ] W-9003-FaceRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2286
2024-04-09T10:34:59,764 [INFO ] W-9003-FaceRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2286
2024-04-09T10:34:59,764 [DEBUG] W-9003-FaceRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - W-9003-FaceRecognition_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:34:59,764 [DEBUG] W-9003-FaceRecognition_1.0 org.pytorch.serve.wlm.WorkerThread - W-9003-FaceRecognition_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:34:59,765 [INFO ] W-9003-FaceRecognition_1.0 TS_METRICS - WorkerLoadTime.Milliseconds:3582.0|#WorkerName:W-9003-FaceRecognition_1.0,Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633699
2024-04-09T10:34:59,765 [INFO ] W-9003-FaceRecognition_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:29.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633699
2024-04-09T10:35:05,420 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model HumanPose
2024-04-09T10:35:05,420 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model HumanPose
2024-04-09T10:35:05,421 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model HumanPose
2024-04-09T10:35:05,421 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model HumanPose
2024-04-09T10:35:05,421 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model HumanPose loaded.
2024-04-09T10:35:05,421 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model HumanPose loaded.
2024-04-09T10:35:05,422 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: HumanPose, count: 1
2024-04-09T10:35:05,422 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: HumanPose, count: 1
2024-04-09T10:35:05,423 [DEBUG] W-9004-HumanPose_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9004, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:05,423 [DEBUG] W-9004-HumanPose_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9004, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:05,424 [DEBUG] main org.pytorch.serve.ModelServer - Loading models from model store: PhoneDetection.mar
2024-04-09T10:35:05,424 [DEBUG] main org.pytorch.serve.ModelServer - Loading models from model store: PhoneDetection.mar
2024-04-09T10:35:07,297 [INFO ] W-9004-HumanPose_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9004, pid=91965
2024-04-09T10:35:07,298 [INFO ] W-9004-HumanPose_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9004
2024-04-09T10:35:07,310 [INFO ] W-9004-HumanPose_1.0-stdout MODEL_LOG - Successfully loaded /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2024-04-09T10:35:07,311 [INFO ] W-9004-HumanPose_1.0-stdout MODEL_LOG - [PID]91965
2024-04-09T10:35:07,311 [DEBUG] W-9004-HumanPose_1.0 org.pytorch.serve.wlm.WorkerThread - W-9004-HumanPose_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:07,311 [INFO ] W-9004-HumanPose_1.0-stdout MODEL_LOG - Torch worker started.
2024-04-09T10:35:07,311 [DEBUG] W-9004-HumanPose_1.0 org.pytorch.serve.wlm.WorkerThread - W-9004-HumanPose_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:07,312 [INFO ] W-9004-HumanPose_1.0-stdout MODEL_LOG - Python runtime: 3.8.19
2024-04-09T10:35:07,312 [INFO ] W-9004-HumanPose_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9004
2024-04-09T10:35:07,312 [INFO ] W-9004-HumanPose_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9004
2024-04-09T10:35:07,314 [INFO ] W-9004-HumanPose_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633707314
2024-04-09T10:35:07,314 [INFO ] W-9004-HumanPose_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633707314
2024-04-09T10:35:07,315 [INFO ] W-9004-HumanPose_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9004.
2024-04-09T10:35:07,423 [INFO ] W-9004-HumanPose_1.0-stdout MODEL_LOG - model_name: HumanPose, batchSize: 256
2024-04-09T10:35:08,072 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model PhoneDetection
2024-04-09T10:35:08,072 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model PhoneDetection
2024-04-09T10:35:08,072 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model PhoneDetection
2024-04-09T10:35:08,072 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model PhoneDetection
2024-04-09T10:35:08,073 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model PhoneDetection loaded.
2024-04-09T10:35:08,073 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model PhoneDetection loaded.
2024-04-09T10:35:08,074 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: PhoneDetection, count: 8
2024-04-09T10:35:08,074 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: PhoneDetection, count: 8
2024-04-09T10:35:08,077 [DEBUG] W-9005-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9005, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:08,077 [DEBUG] W-9007-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9007, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:08,077 [DEBUG] W-9007-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9007, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:08,077 [DEBUG] W-9006-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9006, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:08,077 [DEBUG] W-9005-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9005, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:08,080 [DEBUG] W-9008-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9008, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:08,077 [DEBUG] W-9006-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9006, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:08,080 [DEBUG] W-9008-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9008, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:08,083 [DEBUG] W-9010-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9010, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:08,083 [DEBUG] W-9009-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9009, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:08,084 [DEBUG] W-9011-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9011, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:08,088 [DEBUG] W-9012-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9012, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:08,084 [DEBUG] W-9011-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9011, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:08,083 [DEBUG] W-9010-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9010, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:08,093 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: EpollServerSocketChannel.
2024-04-09T10:35:08,088 [DEBUG] W-9012-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9012, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:08,093 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: EpollServerSocketChannel.
2024-04-09T10:35:08,083 [DEBUG] W-9009-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/hungtrieu07/miniconda3/envs/api_lanc/bin/python, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9009, --metrics-config, /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2024-04-09T10:35:08,216 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://0.0.0.0:8080
2024-04-09T10:35:08,216 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://0.0.0.0:8080
2024-04-09T10:35:08,217 [INFO ] main org.pytorch.serve.ModelServer - Initialize Management server with: EpollServerSocketChannel.
2024-04-09T10:35:08,217 [INFO ] main org.pytorch.serve.ModelServer - Initialize Management server with: EpollServerSocketChannel.
2024-04-09T10:35:08,282 [INFO ] main org.pytorch.serve.ModelServer - Management API bind to: http://0.0.0.0:8081
2024-04-09T10:35:08,282 [INFO ] main org.pytorch.serve.ModelServer - Management API bind to: http://0.0.0.0:8081
2024-04-09T10:35:08,284 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: EpollServerSocketChannel.
2024-04-09T10:35:08,284 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: EpollServerSocketChannel.
2024-04-09T10:35:08,290 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://0.0.0.0:8082
2024-04-09T10:35:08,290 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://0.0.0.0:8082
2024-04-09T10:35:09,160 [INFO ] W-9004-HumanPose_1.0-stdout MODEL_LOG - ONNX enabled
2024-04-09T10:35:09,162 [INFO ] W-9004-HumanPose_1.0-stdout MODEL_LOG - Torch TensorRT not enabled
2024-04-09T10:35:09,568 [WARN ] pool-3-thread-1 org.pytorch.serve.metrics.MetricCollector - worker pid is not available yet.
2024-04-09T10:35:09,568 [WARN ] pool-3-thread-1 org.pytorch.serve.metrics.MetricCollector - worker pid is not available yet.
2024-04-09T10:35:09,790 [INFO ] pool-3-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633709
2024-04-09T10:35:09,793 [INFO ] pool-3-thread-1 TS_METRICS - DiskAvailable.Gigabytes:924.9861183166504|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633709
2024-04-09T10:35:09,794 [INFO ] pool-3-thread-1 TS_METRICS - DiskUsage.Gigabytes:30.652191162109375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633709
2024-04-09T10:35:09,795 [INFO ] pool-3-thread-1 TS_METRICS - DiskUtilization.Percent:3.2|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633709
2024-04-09T10:35:09,796 [INFO ] pool-3-thread-1 TS_METRICS - MemoryAvailable.Megabytes:3258.49609375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633709
2024-04-09T10:35:09,797 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUsed.Megabytes:4309.60546875|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633709
2024-04-09T10:35:09,798 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUtilization.Percent:58.6|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633709
2024-04-09T10:35:10,132 [DEBUG] W-9004-HumanPose_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:10,132 [DEBUG] W-9004-HumanPose_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:10,135 [INFO ] W-9004-HumanPose_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2709
2024-04-09T10:35:10,135 [INFO ] W-9004-HumanPose_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2709
2024-04-09T10:35:10,136 [DEBUG] W-9004-HumanPose_1.0 org.pytorch.serve.wlm.WorkerThread - W-9004-HumanPose_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:10,136 [DEBUG] W-9004-HumanPose_1.0 org.pytorch.serve.wlm.WorkerThread - W-9004-HumanPose_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:10,137 [INFO ] W-9004-HumanPose_1.0 TS_METRICS - WorkerLoadTime.Milliseconds:4715.0|#WorkerName:W-9004-HumanPose_1.0,Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633710
2024-04-09T10:35:10,138 [INFO ] W-9004-HumanPose_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:115.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633710
2024-04-09T10:35:11,187 [INFO ] W-9007-PhoneDetection_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9007, pid=92001
2024-04-09T10:35:11,188 [INFO ] W-9007-PhoneDetection_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9007
2024-04-09T10:35:11,202 [INFO ] W-9007-PhoneDetection_1.0-stdout MODEL_LOG - Successfully loaded /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2024-04-09T10:35:11,204 [INFO ] W-9007-PhoneDetection_1.0-stdout MODEL_LOG - [PID]92001
2024-04-09T10:35:11,209 [DEBUG] W-9007-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9007-PhoneDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:11,209 [DEBUG] W-9007-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9007-PhoneDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:11,213 [INFO ] W-9007-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9007
2024-04-09T10:35:11,213 [INFO ] W-9007-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9007
2024-04-09T10:35:11,210 [INFO ] W-9007-PhoneDetection_1.0-stdout MODEL_LOG - Torch worker started.
2024-04-09T10:35:11,216 [INFO ] W-9007-PhoneDetection_1.0-stdout MODEL_LOG - Python runtime: 3.8.19
2024-04-09T10:35:11,226 [INFO ] W-9007-PhoneDetection_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9007.
2024-04-09T10:35:11,230 [INFO ] W-9007-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633711230
2024-04-09T10:35:11,230 [INFO ] W-9007-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633711230
2024-04-09T10:35:11,295 [INFO ] W-9007-PhoneDetection_1.0-stdout MODEL_LOG - model_name: PhoneDetection, batchSize: 1
2024-04-09T10:35:11,305 [INFO ] W-9010-PhoneDetection_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9010, pid=92008
2024-04-09T10:35:11,308 [INFO ] W-9010-PhoneDetection_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9010
2024-04-09T10:35:11,326 [INFO ] W-9010-PhoneDetection_1.0-stdout MODEL_LOG - Successfully loaded /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2024-04-09T10:35:11,328 [INFO ] W-9010-PhoneDetection_1.0-stdout MODEL_LOG - [PID]92008
2024-04-09T10:35:11,329 [DEBUG] W-9010-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9010-PhoneDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:11,329 [DEBUG] W-9010-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9010-PhoneDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:11,329 [INFO ] W-9010-PhoneDetection_1.0-stdout MODEL_LOG - Torch worker started.
2024-04-09T10:35:11,336 [INFO ] W-9011-PhoneDetection_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9011, pid=92007
2024-04-09T10:35:11,340 [INFO ] W-9010-PhoneDetection_1.0-stdout MODEL_LOG - Python runtime: 3.8.19
2024-04-09T10:35:11,332 [INFO ] W-9010-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9010
2024-04-09T10:35:11,332 [INFO ] W-9010-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9010
2024-04-09T10:35:11,343 [INFO ] W-9011-PhoneDetection_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9011
2024-04-09T10:35:11,361 [INFO ] W-9011-PhoneDetection_1.0-stdout MODEL_LOG - Successfully loaded /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2024-04-09T10:35:11,366 [INFO ] W-9010-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633711366
2024-04-09T10:35:11,366 [INFO ] W-9010-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633711366
2024-04-09T10:35:11,364 [INFO ] W-9011-PhoneDetection_1.0-stdout MODEL_LOG - [PID]92007
2024-04-09T10:35:11,373 [DEBUG] W-9011-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9011-PhoneDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:11,373 [DEBUG] W-9011-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9011-PhoneDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:11,367 [INFO ] W-9010-PhoneDetection_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9010.
2024-04-09T10:35:11,385 [INFO ] W-9009-PhoneDetection_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9009, pid=92010
2024-04-09T10:35:11,373 [INFO ] W-9011-PhoneDetection_1.0-stdout MODEL_LOG - Torch worker started.
2024-04-09T10:35:11,397 [INFO ] W-9009-PhoneDetection_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9009
2024-04-09T10:35:11,397 [INFO ] W-9011-PhoneDetection_1.0-stdout MODEL_LOG - Python runtime: 3.8.19
2024-04-09T10:35:11,386 [INFO ] W-9011-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9011
2024-04-09T10:35:11,386 [INFO ] W-9011-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9011
2024-04-09T10:35:11,406 [INFO ] W-9009-PhoneDetection_1.0-stdout MODEL_LOG - Successfully loaded /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2024-04-09T10:35:11,420 [INFO ] W-9009-PhoneDetection_1.0-stdout MODEL_LOG - [PID]92010
2024-04-09T10:35:11,425 [INFO ] W-9011-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633711424
2024-04-09T10:35:11,425 [INFO ] W-9011-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633711424
2024-04-09T10:35:11,420 [INFO ] W-9012-PhoneDetection_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9012, pid=92009
2024-04-09T10:35:11,421 [INFO ] W-9010-PhoneDetection_1.0-stdout MODEL_LOG - model_name: PhoneDetection, batchSize: 1
2024-04-09T10:35:11,422 [INFO ] W-9011-PhoneDetection_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9011.
2024-04-09T10:35:11,428 [DEBUG] W-9009-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9009-PhoneDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:11,434 [INFO ] W-9012-PhoneDetection_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9012
2024-04-09T10:35:11,422 [INFO ] W-9009-PhoneDetection_1.0-stdout MODEL_LOG - Torch worker started.
2024-04-09T10:35:11,450 [INFO ] W-9012-PhoneDetection_1.0-stdout MODEL_LOG - Successfully loaded /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2024-04-09T10:35:11,450 [INFO ] W-9009-PhoneDetection_1.0-stdout MODEL_LOG - Python runtime: 3.8.19
2024-04-09T10:35:11,434 [INFO ] W-9006-PhoneDetection_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9006, pid=92005
2024-04-09T10:35:11,428 [DEBUG] W-9009-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9009-PhoneDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:11,451 [INFO ] W-9012-PhoneDetection_1.0-stdout MODEL_LOG - [PID]92009
2024-04-09T10:35:11,451 [INFO ] W-9006-PhoneDetection_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9006
2024-04-09T10:35:11,452 [INFO ] W-9009-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9009
2024-04-09T10:35:11,452 [DEBUG] W-9012-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9012-PhoneDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:11,452 [INFO ] W-9012-PhoneDetection_1.0-stdout MODEL_LOG - Torch worker started.
2024-04-09T10:35:11,452 [INFO ] W-9009-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9009
2024-04-09T10:35:11,452 [DEBUG] W-9012-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9012-PhoneDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:11,453 [INFO ] W-9012-PhoneDetection_1.0-stdout MODEL_LOG - Python runtime: 3.8.19
2024-04-09T10:35:11,459 [INFO ] W-9009-PhoneDetection_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9009.
2024-04-09T10:35:11,460 [INFO ] W-9012-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9012
2024-04-09T10:35:11,460 [INFO ] W-9012-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9012
2024-04-09T10:35:11,457 [INFO ] W-9006-PhoneDetection_1.0-stdout MODEL_LOG - Successfully loaded /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2024-04-09T10:35:11,459 [INFO ] W-9009-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633711459
2024-04-09T10:35:11,457 [INFO ] W-9011-PhoneDetection_1.0-stdout MODEL_LOG - model_name: PhoneDetection, batchSize: 1
2024-04-09T10:35:11,462 [INFO ] W-9006-PhoneDetection_1.0-stdout MODEL_LOG - [PID]92005
2024-04-09T10:35:11,463 [INFO ] W-9012-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633711463
2024-04-09T10:35:11,459 [INFO ] W-9009-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633711459
2024-04-09T10:35:11,463 [DEBUG] W-9006-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9006-PhoneDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:11,463 [DEBUG] W-9006-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9006-PhoneDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:11,466 [INFO ] W-9006-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9006
2024-04-09T10:35:11,466 [INFO ] W-9006-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9006
2024-04-09T10:35:11,463 [INFO ] W-9006-PhoneDetection_1.0-stdout MODEL_LOG - Torch worker started.
2024-04-09T10:35:11,463 [INFO ] W-9012-PhoneDetection_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9012.
2024-04-09T10:35:11,463 [INFO ] W-9012-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633711463
2024-04-09T10:35:11,471 [INFO ] W-9006-PhoneDetection_1.0-stdout MODEL_LOG - Python runtime: 3.8.19
2024-04-09T10:35:11,480 [INFO ] W-9006-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633711480
2024-04-09T10:35:11,480 [INFO ] W-9006-PhoneDetection_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9006.
2024-04-09T10:35:11,480 [INFO ] W-9006-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633711480
2024-04-09T10:35:11,476 [INFO ] W-9008-PhoneDetection_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9008, pid=92006
2024-04-09T10:35:11,481 [INFO ] W-9008-PhoneDetection_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9008
2024-04-09T10:35:11,524 [INFO ] W-9008-PhoneDetection_1.0-stdout MODEL_LOG - Successfully loaded /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2024-04-09T10:35:11,532 [INFO ] W-9009-PhoneDetection_1.0-stdout MODEL_LOG - model_name: PhoneDetection, batchSize: 1
2024-04-09T10:35:11,587 [INFO ] W-9008-PhoneDetection_1.0-stdout MODEL_LOG - [PID]92006
2024-04-09T10:35:11,601 [DEBUG] W-9008-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9008-PhoneDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:11,601 [DEBUG] W-9008-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9008-PhoneDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:11,630 [INFO ] W-9008-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9008
2024-04-09T10:35:11,630 [INFO ] W-9008-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9008
2024-04-09T10:35:11,600 [INFO ] W-9008-PhoneDetection_1.0-stdout MODEL_LOG - Torch worker started.
2024-04-09T10:35:11,617 [INFO ] W-9012-PhoneDetection_1.0-stdout MODEL_LOG - model_name: PhoneDetection, batchSize: 1
2024-04-09T10:35:11,648 [INFO ] W-9008-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633711648
2024-04-09T10:35:11,648 [INFO ] W-9008-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633711648
2024-04-09T10:35:11,646 [INFO ] W-9008-PhoneDetection_1.0-stdout MODEL_LOG - Python runtime: 3.8.19
2024-04-09T10:35:11,657 [INFO ] W-9006-PhoneDetection_1.0-stdout MODEL_LOG - model_name: PhoneDetection, batchSize: 1
2024-04-09T10:35:11,690 [INFO ] W-9008-PhoneDetection_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9008.
2024-04-09T10:35:11,713 [INFO ] W-9005-PhoneDetection_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9005, pid=92000
2024-04-09T10:35:11,727 [INFO ] W-9005-PhoneDetection_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9005
2024-04-09T10:35:11,753 [INFO ] W-9005-PhoneDetection_1.0-stdout MODEL_LOG - Successfully loaded /home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2024-04-09T10:35:11,756 [INFO ] W-9005-PhoneDetection_1.0-stdout MODEL_LOG - [PID]92000
2024-04-09T10:35:11,759 [DEBUG] W-9005-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9005-PhoneDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:11,759 [DEBUG] W-9005-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9005-PhoneDetection_1.0 State change null -> WORKER_STARTED
2024-04-09T10:35:11,762 [INFO ] W-9005-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9005
2024-04-09T10:35:11,762 [INFO ] W-9005-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9005
2024-04-09T10:35:11,759 [INFO ] W-9005-PhoneDetection_1.0-stdout MODEL_LOG - Torch worker started.
2024-04-09T10:35:11,764 [INFO ] W-9005-PhoneDetection_1.0-stdout MODEL_LOG - Python runtime: 3.8.19
2024-04-09T10:35:11,781 [INFO ] W-9005-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633711781
2024-04-09T10:35:11,781 [INFO ] W-9005-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1712633711781
2024-04-09T10:35:11,780 [INFO ] W-9005-PhoneDetection_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9005.
2024-04-09T10:35:11,807 [INFO ] W-9008-PhoneDetection_1.0-stdout MODEL_LOG - model_name: PhoneDetection, batchSize: 1
2024-04-09T10:35:11,908 [INFO ] W-9005-PhoneDetection_1.0-stdout MODEL_LOG - model_name: PhoneDetection, batchSize: 1
2024-04-09T10:35:12,593 [INFO ] W-9011-PhoneDetection_1.0-stdout MODEL_LOG - ONNX enabled
2024-04-09T10:35:12,594 [INFO ] W-9011-PhoneDetection_1.0-stdout MODEL_LOG - Torch TensorRT not enabled
2024-04-09T10:35:12,598 [INFO ] W-9007-PhoneDetection_1.0-stdout MODEL_LOG - ONNX enabled
2024-04-09T10:35:12,599 [INFO ] W-9007-PhoneDetection_1.0-stdout MODEL_LOG - Torch TensorRT not enabled
2024-04-09T10:35:12,600 [INFO ] W-9010-PhoneDetection_1.0-stdout MODEL_LOG - ONNX enabled
2024-04-09T10:35:12,600 [INFO ] W-9010-PhoneDetection_1.0-stdout MODEL_LOG - Torch TensorRT not enabled
2024-04-09T10:35:12,639 [INFO ] W-9009-PhoneDetection_1.0-stdout MODEL_LOG - ONNX enabled
2024-04-09T10:35:12,640 [INFO ] W-9009-PhoneDetection_1.0-stdout MODEL_LOG - Torch TensorRT not enabled
2024-04-09T10:35:12,844 [INFO ] W-9012-PhoneDetection_1.0-stdout MODEL_LOG - ONNX enabled
2024-04-09T10:35:12,847 [INFO ] W-9012-PhoneDetection_1.0-stdout MODEL_LOG - Torch TensorRT not enabled
2024-04-09T10:35:12,851 [INFO ] W-9008-PhoneDetection_1.0-stdout MODEL_LOG - ONNX enabled
2024-04-09T10:35:12,852 [INFO ] W-9008-PhoneDetection_1.0-stdout MODEL_LOG - Torch TensorRT not enabled
2024-04-09T10:35:12,898 [INFO ] W-9006-PhoneDetection_1.0-stdout MODEL_LOG - ONNX enabled
2024-04-09T10:35:12,901 [INFO ] W-9006-PhoneDetection_1.0-stdout MODEL_LOG - Torch TensorRT not enabled
2024-04-09T10:35:12,917 [DEBUG] W-9011-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:12,917 [DEBUG] W-9011-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:12,924 [INFO ] W-9011-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1464
2024-04-09T10:35:12,924 [INFO ] W-9011-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1464
2024-04-09T10:35:12,925 [DEBUG] W-9011-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9011-PhoneDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:12,925 [DEBUG] W-9011-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9011-PhoneDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:12,925 [INFO ] W-9011-PhoneDetection_1.0 TS_METRICS - WorkerLoadTime.Milliseconds:4846.0|#WorkerName:W-9011-PhoneDetection_1.0,Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633712
2024-04-09T10:35:12,927 [INFO ] W-9011-PhoneDetection_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:39.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633712
2024-04-09T10:35:12,935 [DEBUG] W-9010-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:12,935 [DEBUG] W-9010-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:12,941 [INFO ] W-9010-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1511
2024-04-09T10:35:12,941 [INFO ] W-9010-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1511
2024-04-09T10:35:12,944 [DEBUG] W-9010-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9010-PhoneDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:12,944 [DEBUG] W-9010-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9010-PhoneDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:12,945 [INFO ] W-9010-PhoneDetection_1.0 TS_METRICS - WorkerLoadTime.Milliseconds:4866.0|#WorkerName:W-9010-PhoneDetection_1.0,Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633712
2024-04-09T10:35:12,947 [INFO ] W-9010-PhoneDetection_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:70.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633712
2024-04-09T10:35:12,981 [INFO ] W-9005-PhoneDetection_1.0-stdout MODEL_LOG - ONNX enabled
2024-04-09T10:35:12,982 [INFO ] W-9005-PhoneDetection_1.0-stdout MODEL_LOG - Torch TensorRT not enabled
2024-04-09T10:35:13,012 [DEBUG] W-9009-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:13,012 [DEBUG] W-9009-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:13,013 [INFO ] W-9009-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1488
2024-04-09T10:35:13,013 [INFO ] W-9009-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1488
2024-04-09T10:35:13,013 [DEBUG] W-9009-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9009-PhoneDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:13,013 [DEBUG] W-9009-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9009-PhoneDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:13,014 [INFO ] W-9009-PhoneDetection_1.0 TS_METRICS - WorkerLoadTime.Milliseconds:4936.0|#WorkerName:W-9009-PhoneDetection_1.0,Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633713
2024-04-09T10:35:13,015 [INFO ] W-9009-PhoneDetection_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:68.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633713
2024-04-09T10:35:13,071 [DEBUG] W-9007-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:13,071 [DEBUG] W-9007-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:13,071 [INFO ] W-9007-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1780
2024-04-09T10:35:13,071 [INFO ] W-9007-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1780
2024-04-09T10:35:13,072 [DEBUG] W-9007-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9007-PhoneDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:13,072 [DEBUG] W-9007-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9007-PhoneDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:13,072 [INFO ] W-9007-PhoneDetection_1.0 TS_METRICS - WorkerLoadTime.Milliseconds:4997.0|#WorkerName:W-9007-PhoneDetection_1.0,Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633713
2024-04-09T10:35:13,073 [INFO ] W-9007-PhoneDetection_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:63.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633713
2024-04-09T10:35:13,203 [DEBUG] W-9012-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:13,203 [DEBUG] W-9012-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:13,203 [INFO ] W-9012-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1585
2024-04-09T10:35:13,203 [INFO ] W-9012-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1585
2024-04-09T10:35:13,203 [DEBUG] W-9012-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9012-PhoneDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:13,203 [DEBUG] W-9012-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9012-PhoneDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:13,204 [INFO ] W-9012-PhoneDetection_1.0 TS_METRICS - WorkerLoadTime.Milliseconds:5122.0|#WorkerName:W-9012-PhoneDetection_1.0,Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633713
2024-04-09T10:35:13,205 [INFO ] W-9012-PhoneDetection_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:156.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633713
2024-04-09T10:35:13,217 [DEBUG] W-9008-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:13,217 [DEBUG] W-9008-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:13,217 [INFO ] W-9008-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1404
2024-04-09T10:35:13,217 [INFO ] W-9008-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1404
2024-04-09T10:35:13,221 [DEBUG] W-9008-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9008-PhoneDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:13,221 [DEBUG] W-9008-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9008-PhoneDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:13,222 [INFO ] W-9008-PhoneDetection_1.0 TS_METRICS - WorkerLoadTime.Milliseconds:5146.0|#WorkerName:W-9008-PhoneDetection_1.0,Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633713
2024-04-09T10:35:13,223 [INFO ] W-9008-PhoneDetection_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:171.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633713
2024-04-09T10:35:13,229 [DEBUG] W-9006-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:13,229 [DEBUG] W-9006-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:13,230 [INFO ] W-9006-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1552
2024-04-09T10:35:13,230 [INFO ] W-9006-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1552
2024-04-09T10:35:13,230 [DEBUG] W-9006-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9006-PhoneDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:13,230 [DEBUG] W-9006-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9006-PhoneDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:13,231 [INFO ] W-9006-PhoneDetection_1.0 TS_METRICS - WorkerLoadTime.Milliseconds:5155.0|#WorkerName:W-9006-PhoneDetection_1.0,Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633713
2024-04-09T10:35:13,231 [INFO ] W-9006-PhoneDetection_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:199.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633713
2024-04-09T10:35:13,248 [DEBUG] W-9005-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:13,248 [DEBUG] W-9005-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:35:13,249 [INFO ] W-9005-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1341
2024-04-09T10:35:13,249 [INFO ] W-9005-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1341
2024-04-09T10:35:13,249 [DEBUG] W-9005-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9005-PhoneDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:13,249 [DEBUG] W-9005-PhoneDetection_1.0 org.pytorch.serve.wlm.WorkerThread - W-9005-PhoneDetection_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2024-04-09T10:35:13,250 [INFO ] W-9005-PhoneDetection_1.0 TS_METRICS - WorkerLoadTime.Milliseconds:5176.0|#WorkerName:W-9005-PhoneDetection_1.0,Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633713
2024-04-09T10:35:13,250 [INFO ] W-9005-PhoneDetection_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:128.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633713
2024-04-09T10:36:09,567 [INFO ] pool-3-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633769
2024-04-09T10:36:09,568 [INFO ] pool-3-thread-1 TS_METRICS - DiskAvailable.Gigabytes:924.986083984375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633769
2024-04-09T10:36:09,568 [INFO ] pool-3-thread-1 TS_METRICS - DiskUsage.Gigabytes:30.652225494384766|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633769
2024-04-09T10:36:09,569 [INFO ] pool-3-thread-1 TS_METRICS - DiskUtilization.Percent:3.2|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633769
2024-04-09T10:36:09,570 [INFO ] pool-3-thread-1 TS_METRICS - MemoryAvailable.Megabytes:1889.4765625|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633769
2024-04-09T10:36:09,570 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUsed.Megabytes:5678.94921875|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633769
2024-04-09T10:36:09,571 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUtilization.Percent:76.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633769
2024-04-09T10:37:09,563 [INFO ] pool-3-thread-2 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633829
2024-04-09T10:37:09,565 [INFO ] pool-3-thread-2 TS_METRICS - DiskAvailable.Gigabytes:924.986083984375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633829
2024-04-09T10:37:09,566 [INFO ] pool-3-thread-2 TS_METRICS - DiskUsage.Gigabytes:30.652225494384766|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633829
2024-04-09T10:37:09,567 [INFO ] pool-3-thread-2 TS_METRICS - DiskUtilization.Percent:3.2|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633829
2024-04-09T10:37:09,567 [INFO ] pool-3-thread-2 TS_METRICS - MemoryAvailable.Megabytes:1905.0078125|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633829
2024-04-09T10:37:09,568 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUsed.Megabytes:5663.77734375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633829
2024-04-09T10:37:09,569 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUtilization.Percent:75.8|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633829
2024-04-09T10:38:09,561 [INFO ] pool-3-thread-2 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633889
2024-04-09T10:38:09,562 [INFO ] pool-3-thread-2 TS_METRICS - DiskAvailable.Gigabytes:924.986083984375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633889
2024-04-09T10:38:09,563 [INFO ] pool-3-thread-2 TS_METRICS - DiskUsage.Gigabytes:30.652225494384766|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633889
2024-04-09T10:38:09,565 [INFO ] pool-3-thread-2 TS_METRICS - DiskUtilization.Percent:3.2|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633889
2024-04-09T10:38:09,566 [INFO ] pool-3-thread-2 TS_METRICS - MemoryAvailable.Megabytes:1911.93359375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633889
2024-04-09T10:38:09,568 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUsed.Megabytes:5656.8515625|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633889
2024-04-09T10:38:09,569 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUtilization.Percent:75.7|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633889
2024-04-09T10:39:09,560 [INFO ] pool-3-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633949
2024-04-09T10:39:09,560 [INFO ] pool-3-thread-1 TS_METRICS - DiskAvailable.Gigabytes:924.986083984375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633949
2024-04-09T10:39:09,561 [INFO ] pool-3-thread-1 TS_METRICS - DiskUsage.Gigabytes:30.652225494384766|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633949
2024-04-09T10:39:09,562 [INFO ] pool-3-thread-1 TS_METRICS - DiskUtilization.Percent:3.2|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633949
2024-04-09T10:39:09,563 [INFO ] pool-3-thread-1 TS_METRICS - MemoryAvailable.Megabytes:1905.84375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633949
2024-04-09T10:39:09,563 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUsed.Megabytes:5662.94921875|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633949
2024-04-09T10:39:09,564 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUtilization.Percent:75.8|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633949
2024-04-09T10:39:45,692 [INFO ] pool-2-thread-14 ACCESS_LOG - /[0:0:0:0:0:0:0:1%0]:47326 "GET /ping HTTP/1.1" 200 5
2024-04-09T10:39:45,693 [INFO ] pool-2-thread-14 TS_METRICS - Requests2XX.Count:1.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633985
2024-04-09T10:39:49,653 [INFO ] epollEventLoopGroup-3-2 ACCESS_LOG - /[0:0:0:0:0:0:0:1%0]:40140 "GET /models HTTP/1.1" 200 4
2024-04-09T10:39:49,654 [INFO ] epollEventLoopGroup-3-2 TS_METRICS - Requests2XX.Count:1.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712633989
2024-04-09T10:40:09,564 [INFO ] pool-3-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634009
2024-04-09T10:40:09,565 [INFO ] pool-3-thread-1 TS_METRICS - DiskAvailable.Gigabytes:924.986083984375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634009
2024-04-09T10:40:09,566 [INFO ] pool-3-thread-1 TS_METRICS - DiskUsage.Gigabytes:30.652225494384766|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634009
2024-04-09T10:40:09,566 [INFO ] pool-3-thread-1 TS_METRICS - DiskUtilization.Percent:3.2|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634009
2024-04-09T10:40:09,567 [INFO ] pool-3-thread-1 TS_METRICS - MemoryAvailable.Megabytes:1857.796875|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634009
2024-04-09T10:40:09,568 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUsed.Megabytes:5710.99609375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634009
2024-04-09T10:40:09,568 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUtilization.Percent:76.4|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634009
2024-04-09T10:40:38,197 [INFO ] epollEventLoopGroup-3-3 TS_METRICS - ts_inference_requests_total.Count:1.0|#model_name:FaceDetection,model_version:default|#hostname:DESKTOP-UR1F133,timestamp:1712634038
2024-04-09T10:40:38,299 [INFO ] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd PREDICT to backend at: 1712634038299
2024-04-09T10:40:38,299 [INFO ] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd PREDICT to backend at: 1712634038299
2024-04-09T10:40:38,301 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - Backend received inference at: 1712634038
2024-04-09T10:40:38,302 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - Invoking custom service failed.
2024-04-09T10:40:38,302 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - Traceback (most recent call last):
2024-04-09T10:40:38,303 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG -   File "/home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/service.py", line 134, in predict
2024-04-09T10:40:38,304 [INFO ] W-9001-FaceDetection_1.0 ACCESS_LOG - /[0:0:0:0:0:0:0:1%0]:49728 "PUT /predictions/FaceDetection HTTP/1.1" 503 109
2024-04-09T10:40:38,304 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG -     ret = self._entry_point(input_batch, self.context)
2024-04-09T10:40:38,304 [INFO ] W-9001-FaceDetection_1.0 TS_METRICS - Requests5XX.Count:1.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634038
2024-04-09T10:40:38,304 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG -   File "/tmp/models/514d97655cd546e9ae0fbdbd23c166bc/my_handler.py", line 13, in handle
2024-04-09T10:40:38,305 [DEBUG] W-9001-FaceDetection_1.0 org.pytorch.serve.job.Job - Waiting time ns: 100447902, Inference time ns: 107102902
2024-04-09T10:40:38,305 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG -     data = _service.preprocess(data)
2024-04-09T10:40:38,305 [DEBUG] W-9001-FaceDetection_1.0 org.pytorch.serve.job.Job - Waiting time ns: 100447902, Inference time ns: 107102902
2024-04-09T10:40:38,308 [DEBUG] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:40:38,308 [DEBUG] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:40:38,308 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG -   File "/home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/handler_utils/timer.py", line 27, in wrap_func
2024-04-09T10:40:38,309 [INFO ] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2024-04-09T10:40:38,309 [INFO ] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2024-04-09T10:40:38,309 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG -     "handler" in self.context.model_yaml_config
2024-04-09T10:40:38,310 [INFO ] W-9001-FaceDetection_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:9.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634038
2024-04-09T10:40:38,311 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - AttributeError: 'NoneType' object has no attribute 'model_yaml_config'
2024-04-09T10:41:09,615 [INFO ] pool-3-thread-2 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634069
2024-04-09T10:41:09,616 [INFO ] pool-3-thread-2 TS_METRICS - DiskAvailable.Gigabytes:924.9860763549805|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634069
2024-04-09T10:41:09,617 [INFO ] pool-3-thread-2 TS_METRICS - DiskUsage.Gigabytes:30.652233123779297|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634069
2024-04-09T10:41:09,618 [INFO ] pool-3-thread-2 TS_METRICS - DiskUtilization.Percent:3.2|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634069
2024-04-09T10:41:09,619 [INFO ] pool-3-thread-2 TS_METRICS - MemoryAvailable.Megabytes:1696.00390625|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634069
2024-04-09T10:41:09,619 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUsed.Megabytes:5872.79296875|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634069
2024-04-09T10:41:09,620 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUtilization.Percent:78.5|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634069
2024-04-09T10:42:09,595 [INFO ] pool-3-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634129
2024-04-09T10:42:09,596 [INFO ] pool-3-thread-1 TS_METRICS - DiskAvailable.Gigabytes:924.9860687255859|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634129
2024-04-09T10:42:09,597 [INFO ] pool-3-thread-1 TS_METRICS - DiskUsage.Gigabytes:30.652240753173828|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634129
2024-04-09T10:42:09,598 [INFO ] pool-3-thread-1 TS_METRICS - DiskUtilization.Percent:3.2|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634129
2024-04-09T10:42:09,599 [INFO ] pool-3-thread-1 TS_METRICS - MemoryAvailable.Megabytes:1615.3359375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634129
2024-04-09T10:42:09,600 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUsed.Megabytes:5953.15234375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634129
2024-04-09T10:42:09,600 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUtilization.Percent:79.5|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634129
2024-04-09T10:42:44,565 [INFO ] epollEventLoopGroup-3-4 TS_METRICS - ts_inference_requests_total.Count:1.0|#model_name:FaceDetection,model_version:default|#hostname:DESKTOP-UR1F133,timestamp:1712634164
2024-04-09T10:42:44,666 [INFO ] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd PREDICT to backend at: 1712634164666
2024-04-09T10:42:44,666 [INFO ] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd PREDICT to backend at: 1712634164666
2024-04-09T10:42:44,668 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - Backend received inference at: 1712634164
2024-04-09T10:42:44,668 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - Invoking custom service failed.
2024-04-09T10:42:44,668 [INFO ] W-9001-FaceDetection_1.0 ACCESS_LOG - /[0:0:0:0:0:0:0:1%0]:50760 "PUT /predictions/FaceDetection HTTP/1.1" 503 103
2024-04-09T10:42:44,669 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - Traceback (most recent call last):
2024-04-09T10:42:44,669 [INFO ] W-9001-FaceDetection_1.0 TS_METRICS - Requests5XX.Count:1.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634164
2024-04-09T10:42:44,669 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG -   File "/home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/service.py", line 134, in predict
2024-04-09T10:42:44,670 [DEBUG] W-9001-FaceDetection_1.0 org.pytorch.serve.job.Job - Waiting time ns: 100287301, Inference time ns: 103736101
2024-04-09T10:42:44,670 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG -     ret = self._entry_point(input_batch, self.context)
2024-04-09T10:42:44,670 [DEBUG] W-9001-FaceDetection_1.0 org.pytorch.serve.job.Job - Waiting time ns: 100287301, Inference time ns: 103736101
2024-04-09T10:42:44,670 [DEBUG] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:42:44,670 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG -   File "/tmp/models/514d97655cd546e9ae0fbdbd23c166bc/my_handler.py", line 13, in handle
2024-04-09T10:42:44,670 [DEBUG] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2024-04-09T10:42:44,672 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG -     data = _service.preprocess(data)
2024-04-09T10:42:44,672 [INFO ] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2024-04-09T10:42:44,672 [INFO ] W-9001-FaceDetection_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2024-04-09T10:42:44,673 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG -   File "/home/hungtrieu07/miniconda3/envs/api_lanc/lib/python3.8/site-packages/ts/handler_utils/timer.py", line 27, in wrap_func
2024-04-09T10:42:44,673 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG -     "handler" in self.context.model_yaml_config
2024-04-09T10:42:44,673 [INFO ] W-9001-FaceDetection_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:6.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634164
2024-04-09T10:42:44,674 [INFO ] W-9001-FaceDetection_1.0-stdout MODEL_LOG - AttributeError: 'NoneType' object has no attribute 'model_yaml_config'
2024-04-09T10:43:09,589 [INFO ] pool-3-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634189
2024-04-09T10:43:09,590 [INFO ] pool-3-thread-1 TS_METRICS - DiskAvailable.Gigabytes:924.9860649108887|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634189
2024-04-09T10:43:09,591 [INFO ] pool-3-thread-1 TS_METRICS - DiskUsage.Gigabytes:30.652244567871094|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634189
2024-04-09T10:43:09,591 [INFO ] pool-3-thread-1 TS_METRICS - DiskUtilization.Percent:3.2|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634189
2024-04-09T10:43:09,592 [INFO ] pool-3-thread-1 TS_METRICS - MemoryAvailable.Megabytes:1773.9140625|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634189
2024-04-09T10:43:09,593 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUsed.Megabytes:5794.8828125|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634189
2024-04-09T10:43:09,594 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUtilization.Percent:77.5|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634189
2024-04-09T10:44:09,561 [INFO ] pool-3-thread-2 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634249
2024-04-09T10:44:09,562 [INFO ] pool-3-thread-2 TS_METRICS - DiskAvailable.Gigabytes:924.9860649108887|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634249
2024-04-09T10:44:09,563 [INFO ] pool-3-thread-2 TS_METRICS - DiskUsage.Gigabytes:30.652244567871094|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634249
2024-04-09T10:44:09,563 [INFO ] pool-3-thread-2 TS_METRICS - DiskUtilization.Percent:3.2|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634249
2024-04-09T10:44:09,564 [INFO ] pool-3-thread-2 TS_METRICS - MemoryAvailable.Megabytes:1769.26953125|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634249
2024-04-09T10:44:09,565 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUsed.Megabytes:5799.52734375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634249
2024-04-09T10:44:09,566 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUtilization.Percent:77.5|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634249
2024-04-09T10:45:09,589 [INFO ] pool-3-thread-2 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634309
2024-04-09T10:45:09,590 [INFO ] pool-3-thread-2 TS_METRICS - DiskAvailable.Gigabytes:924.9860649108887|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634309
2024-04-09T10:45:09,591 [INFO ] pool-3-thread-2 TS_METRICS - DiskUsage.Gigabytes:30.652244567871094|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634309
2024-04-09T10:45:09,592 [INFO ] pool-3-thread-2 TS_METRICS - DiskUtilization.Percent:3.2|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634309
2024-04-09T10:45:09,593 [INFO ] pool-3-thread-2 TS_METRICS - MemoryAvailable.Megabytes:1771.234375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634309
2024-04-09T10:45:09,594 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUsed.Megabytes:5797.55859375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634309
2024-04-09T10:45:09,595 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUtilization.Percent:77.5|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634309
2024-04-09T10:46:09,572 [INFO ] pool-3-thread-2 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634369
2024-04-09T10:46:09,573 [INFO ] pool-3-thread-2 TS_METRICS - DiskAvailable.Gigabytes:924.9860649108887|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634369
2024-04-09T10:46:09,574 [INFO ] pool-3-thread-2 TS_METRICS - DiskUsage.Gigabytes:30.652244567871094|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634369
2024-04-09T10:46:09,575 [INFO ] pool-3-thread-2 TS_METRICS - DiskUtilization.Percent:3.2|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634369
2024-04-09T10:46:09,575 [INFO ] pool-3-thread-2 TS_METRICS - MemoryAvailable.Megabytes:1755.60546875|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634369
2024-04-09T10:46:09,576 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUsed.Megabytes:5813.19140625|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634369
2024-04-09T10:46:09,577 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUtilization.Percent:77.7|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634369
2024-04-09T10:47:09,563 [INFO ] pool-3-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634429
2024-04-09T10:47:09,564 [INFO ] pool-3-thread-1 TS_METRICS - DiskAvailable.Gigabytes:924.9860649108887|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634429
2024-04-09T10:47:09,565 [INFO ] pool-3-thread-1 TS_METRICS - DiskUsage.Gigabytes:30.652244567871094|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634429
2024-04-09T10:47:09,566 [INFO ] pool-3-thread-1 TS_METRICS - DiskUtilization.Percent:3.2|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634429
2024-04-09T10:47:09,567 [INFO ] pool-3-thread-1 TS_METRICS - MemoryAvailable.Megabytes:1751.53515625|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634429
2024-04-09T10:47:09,567 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUsed.Megabytes:5817.26171875|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634429
2024-04-09T10:47:09,568 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUtilization.Percent:77.8|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634429
2024-04-09T10:48:09,594 [INFO ] pool-3-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634489
2024-04-09T10:48:09,595 [INFO ] pool-3-thread-1 TS_METRICS - DiskAvailable.Gigabytes:924.9860649108887|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634489
2024-04-09T10:48:09,596 [INFO ] pool-3-thread-1 TS_METRICS - DiskUsage.Gigabytes:30.652244567871094|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634489
2024-04-09T10:48:09,597 [INFO ] pool-3-thread-1 TS_METRICS - DiskUtilization.Percent:3.2|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634489
2024-04-09T10:48:09,598 [INFO ] pool-3-thread-1 TS_METRICS - MemoryAvailable.Megabytes:1750.94921875|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634489
2024-04-09T10:48:09,598 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUsed.Megabytes:5817.84765625|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634489
2024-04-09T10:48:09,599 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUtilization.Percent:77.8|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634489
2024-04-09T10:49:09,567 [INFO ] pool-3-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634549
2024-04-09T10:49:09,568 [INFO ] pool-3-thread-1 TS_METRICS - DiskAvailable.Gigabytes:924.9859657287598|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634549
2024-04-09T10:49:09,568 [INFO ] pool-3-thread-1 TS_METRICS - DiskUsage.Gigabytes:30.65234375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634549
2024-04-09T10:49:09,569 [INFO ] pool-3-thread-1 TS_METRICS - DiskUtilization.Percent:3.2|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634549
2024-04-09T10:49:09,569 [INFO ] pool-3-thread-1 TS_METRICS - MemoryAvailable.Megabytes:1616.15234375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634549
2024-04-09T10:49:09,570 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUsed.Megabytes:5952.6484375|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634549
2024-04-09T10:49:09,570 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUtilization.Percent:79.5|#Level:Host|#hostname:DESKTOP-UR1F133,timestamp:1712634549
agunapal commented 7 months ago

@hungtrieu07 You can refer to this example.

https://github.com/pytorch/serve/blob/master/test/pytest/test_onnx.py#L49

You can inherit the base handler initialize method.

hungtrieu07 commented 7 months ago

@hungtrieu07 You can refer to this example.

https://github.com/pytorch/serve/blob/master/test/pytest/test_onnx.py#L49

You can inherit the base handler initialize method.

It worked. Thanks for your help.