search

pytorch / serve

Serve, optimize and scale PyTorch models in production
https://pytorch.org/serve/
Apache License 2.0
4.13k stars 832 forks source link

Backend process failed #2770

Open naveenjr opened 9 months ago

naveenjr commented 9 months ago

🐛 Describe the bug

Command i have used to create model file `torch-model-archiver --model-name yolo_tiny --version 1.0 --model-file model.pth --handler handler.py

torchserve --start --model-store model_store --models yolo_tiny.mar --no-config-snapshot` this is my handler.py

`from ts.torch_handler.base_handler import BaseHandler from transformers import YolosImageProcessor, YolosForObjectDetection from PIL import Image import torch import requests print("Transformers version %s", transformers.version) class YolosHandler(BaseHandler): def init(self): super().init() self.initialized = False self.model = None self.processor = None

def initialize(self, context):
    """Initialize model and processor"""
    properties = context.system_properties
    model_dir = properties.get("model_dir")

    # Initialize the model and processor
    self.model = YolosForObjectDetection.from_pretrained(model_dir)
    self.processor = YolosImageProcessor.from_pretrained(model_dir)

def preprocess(self, data):
    """Preprocess the input data"""
    # Assuming the input is in the form of a JSON file with an image URL
    image_url = data[0].get("data") or data[0].get("body")
    if isinstance(image_url, (bytes, bytearray)):
        image_url = image_url.decode('utf-8')

    # Load the image from the URL
    image = Image.open(requests.get(image_url, stream=True).raw)

    # Process image
    inputs = self.processor(images=image, return_tensors="pt")
    return inputs

def inference(self, inputs):
    """Run inference on the preprocessed data"""
    with torch.no_grad():
        outputs = self.model(**inputs)
    return outputs

def postprocess(self, outputs):
    """Postprocess the inference output"""
    logits = outputs.logits
    bboxes = outputs.pred_boxes
    target_sizes = torch.tensor([outputs.input_sizes.tolist()])
    results = self.processor.post_process_object_detection(outputs, threshold=0.9, target_sizes=target_sizes)[0]

    response = []
    for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
        box = [round(i, 2) for i in box.tolist()]
        response.append(
            f"Detected {self.model.config.id2label[label.item()]} with confidence "
            f"{round(score.item(), 3)} at location {box}"
        )
    return response

`

Error logs

2023-11-07T05:11:39,015 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-11-07T05:11:39,021 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Successfully loaded /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml. 2023-11-07T05:11:39,022 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - [PID]19364 2023-11-07T05:11:39,023 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Torch worker started. 2023-11-07T05:11:39,023 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Python runtime: 3.10.11 2023-11-07T05:11:39,092 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-11-07T05:11:39,159 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - model_name: yolo_tiny, batchSize: 1 2023-11-07T05:11:40,049 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Backend worker process died. 2023-11-07T05:11:40,049 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-11-07T05:11:40,050 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 221, in 2023-11-07T05:11:40,050 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - worker.run_server() 2023-11-07T05:11:40,050 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 189, in run_server 2023-11-07T05:11:40,051 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket) 2023-11-07T05:11:40,052 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 154, in handle_connection 2023-11-07T05:11:40,052 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg) 2023-11-07T05:11:40,052 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 118, in load_model 2023-11-07T05:11:40,055 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service = model_loader.load( 2023-11-07T05:11:40,055 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_loader.py", line 135, in load 2023-11-07T05:11:40,056 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - initialize_fn(service.context) 2023-11-07T05:11:40,056 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/tmp/models/002de4ebcfb34c45ac2fa3619db6b020/handler.py", line 20, in initialize 2023-11-07T05:11:40,056 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.model = YolosForObjectDetection.from_pretrained(model_dir) 2023-11-07T05:11:40,056 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2662, in from_pretrained 2023-11-07T05:11:40,057 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - config, model_kwargs = cls.config_class.from_pretrained( 2023-11-07T05:11:40,057 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/configuration_utils.py", line 591, in from_pretrained 2023-11-07T05:11:40,057 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, kwargs) 2023-11-07T05:11:40,058 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/configuration_utils.py", line 620, in get_config_dict 2023-11-07T05:11:40,058 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, kwargs) 2023-11-07T05:11:40,058 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/configuration_utils.py", line 675, in _get_config_dict 2023-11-07T05:11:40,059 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - resolved_config_file = cached_file( 2023-11-07T05:11:40,059 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/utils/hub.py", line 400, in cached_file 2023-11-07T05:11:40,060 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - raise EnvironmentError( 2023-11-07T05:11:40,060 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - OSError: /tmp/models/002de4ebcfb34c45ac2fa3619db6b020 does not appear to have a file named config.json. Checkout 'https://huggingface.co//tmp/models/002de4ebcfb34c45ac2fa3619db6b020/main' for available files. 2023-11-07T05:11:42,589 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-11-07T05:11:42,594 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Successfully loaded /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml. 2023-11-07T05:11:42,595 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - [PID]19725 2023-11-07T05:11:42,600 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Torch worker started. 2023-11-07T05:11:42,600 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Python runtime: 3.10.11 2023-11-07T05:11:42,604 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-11-07T05:11:42,632 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - model_name: yolo_tiny, batchSize: 1 2023-11-07T05:11:43,522 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Backend worker process died. 2023-11-07T05:11:43,523 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-11-07T05:11:43,524 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 221, in 2023-11-07T05:11:43,524 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - worker.run_server() 2023-11-07T05:11:43,525 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 189, in run_server 2023-11-07T05:11:43,526 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket) 2023-11-07T05:11:43,527 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 154, in handle_connection 2023-11-07T05:11:43,527 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg) 2023-11-07T05:11:43,528 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 118, in load_model 2023-11-07T05:11:43,528 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service = model_loader.load( 2023-11-07T05:11:43,528 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_loader.py", line 135, in load 2023-11-07T05:11:43,528 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - initialize_fn(service.context) 2023-11-07T05:11:43,528 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/tmp/models/002de4ebcfb34c45ac2fa3619db6b020/handler.py", line 20, in initialize 2023-11-07T05:11:43,529 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.model = YolosForObjectDetection.from_pretrained(model_dir) 2023-11-07T05:11:46,036 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-11-07T05:11:46,041 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Successfully loaded /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml. 2023-11-07T05:11:46,041 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - [PID]19733 2023-11-07T05:11:46,042 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Torch worker started. 2023-11-07T05:11:46,043 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Python runtime: 3.10.11 2023-11-07T05:11:46,051 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-11-07T05:11:46,073 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - model_name: yolo_tiny, batchSize: 1 2023-11-07T05:11:46,904 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Backend worker process died. 2023-11-07T05:11:46,905 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-11-07T05:11:46,905 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 221, in 2023-11-07T05:11:46,905 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - worker.run_server() 2023-11-07T05:11:46,906 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 189, in run_server 2023-11-07T05:11:46,906 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket) 2023-11-07T05:11:46,906 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 154, in handle_connection 2023-11-07T05:11:46,906 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg) 2023-11-07T05:11:46,907 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 118, in load_model 2023-11-07T05:11:46,908 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service = model_loader.load( 2023-11-07T05:11:46,908 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec

Installation instructions

No

Model Packaing

`torch-model-archiver --model-name yolo_tiny --version 1.0 --model-file model.pth --handler handler.py

config.properties

metrics_address=http://0.0.0.0:8082 job_queue_size=1000 model_store=/home/model-server/model-store enable_envvars_config=true install_py_dep_per_model=true

inference_address=http://127.0.0.1:8090 management_address=http://127.0.0.1:8091

Versions

TorchServe Version is 0.7.1

Repro instructions

the model is taken from (https://huggingface.co/hustvl/yolos-tiny)

model saved by model_path = '/home/ec2-user/Serving/yolo_tiny/model.pth' # Correct the variable name here torch.save(model.state_dict(), model_path)

Possible Solution

No response

lxning commented 9 months ago

@naveenjr The error "OSError: /tmp/models/002de4ebcfb34c45ac2fa3619db6b020 does not appear to have a file named config.json. Checkout 'https://huggingface.co//tmp/models/002de4ebcfb34c45ac2fa3619db6b020/main' for available files."

Please make sure if the model is completely downloaded from HF and stored in model artifacts.

naveenjr commented 9 months ago

thanks @lxning.

now i have added the config

`model.config.to_json_file("config.json")

and torch-model-archiver --model-name yolo_tiny --version 1.0 --serialized-file model/model.pth --handler handler.py --extra-files model/config.json --export-path model_store -f

but the logs are a little strange

log file is ####################################### (pytorch_p310) [ec2-user@ip-10-194-163-225 yolo_tiny]$ WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. 2023-11-08T06:55:18,834 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager - Initializing plugins manager... 2023-11-08T06:55:18,944 [INFO ] main org.pytorch.serve.ModelServer - Torchserve version: 0.7.1 TS Home: /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages Current directory: /home/ec2-user/Serving/yolo_tiny Temp directory: /tmp Metrics config path: /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml Number of GPUs: 1 Number of CPUs: 4 Max heap size: 3924 M Python executable: /home/ec2-user/anaconda3/envs/pytorch_p310/bin/python Config file: config.properties Inference address: http://127.0.0.1:8090 Management address: http://127.0.0.1:8091 Metrics address: http://0.0.0.0:8082 Model Store: /home/ec2-user/Serving/yolo_tiny/model_store Initial Models: yolo_tiny.mar Log dir: /home/ec2-user/Serving/yolo_tiny/logs Metrics dir: /home/ec2-user/Serving/yolo_tiny/logs Netty threads: 0 Netty client threads: 0 Default workers per model: 1 Blacklist Regex: N/A Maximum Response Size: 6553500 Maximum Request Size: 6553500 Limit Maximum Image Pixels: true Prefer direct buffer: false Allowed Urls: [file://.|http(s)?://.] Custom python dependency for model allowed: true Metrics report format: prometheus Enable metrics API: true Workflow Store: /home/ec2-user/Serving/yolo_tiny/model_store Model config: N/A 2023-11-08T06:55:18,952 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager - Loading snapshot serializer plugin... 2023-11-08T06:55:18,990 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: yolo_tiny.mar 2023-11-08T06:55:19,589 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model yolo_tiny 2023-11-08T06:55:19,589 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model yolo_tiny 2023-11-08T06:55:19,590 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model yolo_tiny loaded. 2023-11-08T06:55:19,590 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: yolo_tiny, count: 1 2023-11-08T06:55:19,600 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: EpollServerSocketChannel. 2023-11-08T06:55:19,599 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/ec2-user/anaconda3/envs/pytorch_p310/bin/python, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml] 2023-11-08T06:55:19,693 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://127.0.0.1:8090 2023-11-08T06:55:19,693 [INFO ] main org.pytorch.serve.ModelServer - Initialize Management server with: EpollServerSocketChannel. 2023-11-08T06:55:19,700 [INFO ] main org.pytorch.serve.ModelServer - Management API bind to: http://127.0.0.1:8091 2023-11-08T06:55:19,700 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: EpollServerSocketChannel. 2023-11-08T06:55:19,701 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://0.0.0.0:8082 Model server started. 2023-11-08T06:55:19,943 [WARN ] pool-3-thread-1 org.pytorch.serve.metrics.MetricCollector - worker pid is not available yet. 2023-11-08T06:55:20,484 [INFO ] pool-3-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426520 2023-11-08T06:55:20,484 [INFO ] pool-3-thread-1 TS_METRICS - DiskAvailable.Gigabytes:67.23215103149414|#Level:Host|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426520 2023-11-08T06:55:20,485 [INFO ] pool-3-thread-1 TS_METRICS - DiskUsage.Gigabytes:112.75611114501953|#Level:Host|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426520 2023-11-08T06:55:20,485 [INFO ] pool-3-thread-1 TS_METRICS - DiskUtilization.Percent:62.6|#Level:Host|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426520 2023-11-08T06:55:20,485 [INFO ] pool-3-thread-1 TS_METRICS - GPUMemoryUtilization.Percent:0.0|#Level:Host,device_id:0|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426520 2023-11-08T06:55:20,485 [INFO ] pool-3-thread-1 TS_METRICS - GPUMemoryUsed.Megabytes:0|#Level:Host,device_id:0|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426520 2023-11-08T06:55:20,486 [INFO ] pool-3-thread-1 TS_METRICS - GPUUtilization.Percent:0|#Level:Host,device_id:0|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426520 2023-11-08T06:55:20,486 [INFO ] pool-3-thread-1 TS_METRICS - MemoryAvailable.Megabytes:13257.59375|#Level:Host|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426520 2023-11-08T06:55:20,486 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUsed.Megabytes:2105.1953125|#Level:Host|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426520 2023-11-08T06:55:20,486 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUtilization.Percent:15.5|#Level:Host|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426520 2023-11-08T06:55:21,262 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-11-08T06:55:21,272 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Successfully loaded /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml. 2023-11-08T06:55:21,273 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - [PID]21272 2023-11-08T06:55:21,273 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Torch worker started. 2023-11-08T06:55:21,273 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Python runtime: 3.10.11 2023-11-08T06:55:21,274 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change null -> WORKER_STARTED 2023-11-08T06:55:21,284 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000 2023-11-08T06:55:21,322 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-11-08T06:55:21,332 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req. to backend at: 1699426521330 2023-11-08T06:55:21,390 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - model_name: yolo_tiny, batchSize: 1 2023-11-08T06:55:22,294 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Backend worker process died. 2023-11-08T06:55:22,295 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-11-08T06:55:22,295 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 221, in 2023-11-08T06:55:22,296 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - worker.run_server() 2023-11-08T06:55:22,301 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 189, in run_server 2023-11-08T06:55:22,303 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket) 2023-11-08T06:55:22,304 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 154, in handle_connection 2023-11-08T06:55:22,306 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg) 2023-11-08T06:55:22,306 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 118, in load_model 2023-11-08T06:55:22,308 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service = model_loader.load( 2023-11-08T06:55:22,308 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_loader.py", line 135, in load 2023-11-08T06:55:22,304 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED 2023-11-08T06:55:22,309 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - initialize_fn(service.context) 2023-11-08T06:55:22,309 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED 2023-11-08T06:55:22,322 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/tmp/models/e0675cae7c07449daa30bcd78ecd3a99/handler.py", line 21, in initialize 2023-11-08T06:55:22,323 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.model = YolosForObjectDetection.from_pretrained(model_dir) 2023-11-08T06:55:22,324 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2864, in from_pretrained 2023-11-08T06:55:22,325 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - raise EnvironmentError( 2023-11-08T06:55:22,326 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - OSError: Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory /tmp/models/e0675cae7c07449daa30bcd78ecd3a99. 2023-11-08T06:55:22,310 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died. java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432) ~[?:?] at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:191) [model-server.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:829) [?:?] 2023-11-08T06:55:22,354 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: yolo_tiny, error: Worker died. 2023-11-08T06:55:22,355 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STARTED -> WORKER_STOPPED 2023-11-08T06:55:22,356 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:55:22,357 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:55:22,358 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 1 seconds. 2023-11-08T06:55:22,373 [INFO ] W-9000-yolo_tiny_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:55:22,373 [INFO ] W-9000-yolo_tiny_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:55:23,359 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/ec2-user/anaconda3/envs/pytorch_p310/bin/python, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml] 2023-11-08T06:55:24,806 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-11-08T06:55:24,811 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Successfully loaded /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml. 2023-11-08T06:55:24,811 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - [PID]21633 2023-11-08T06:55:24,812 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Torch worker started. 2023-11-08T06:55:24,812 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Python runtime: 3.10.11 2023-11-08T06:55:24,812 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STOPPED -> WORKER_STARTED 2023-11-08T06:55:24,813 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000 2023-11-08T06:55:24,815 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-11-08T06:55:24,819 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req. to backend at: 1699426524819 2023-11-08T06:55:24,841 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - model_name: yolo_tiny, batchSize: 1 2023-11-08T06:55:25,687 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Backend worker process died. 2023-11-08T06:55:25,687 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-11-08T06:55:25,687 [INFO ] epollEventLoopGroup-5-2 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED 2023-11-08T06:55:25,689 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 221, in 2023-11-08T06:55:25,690 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - worker.run_server() 2023-11-08T06:55:25,690 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 189, in run_server 2023-11-08T06:55:25,691 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket) 2023-11-08T06:55:25,691 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED 2023-11-08T06:55:25,692 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 154, in handle_connection 2023-11-08T06:55:25,692 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg) 2023-11-08T06:55:25,692 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 118, in load_model 2023-11-08T06:55:25,692 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died. java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432) ~[?:?] at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:191) [model-server.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:829) [?:?] 2023-11-08T06:55:25,692 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: yolo_tiny, error: Worker died. 2023-11-08T06:55:25,692 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service = model_loader.load( 2023-11-08T06:55:25,693 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STARTED -> WORKER_STOPPED 2023-11-08T06:55:25,693 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_loader.py", line 135, in load 2023-11-08T06:55:25,693 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:55:25,694 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:55:25,694 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 1 seconds. 2023-11-08T06:55:25,695 [INFO ] W-9000-yolo_tiny_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:55:25,708 [INFO ] W-9000-yolo_tiny_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:55:26,695 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/ec2-user/anaconda3/envs/pytorch_p310/bin/python, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml] 2023-11-08T06:55:28,131 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-11-08T06:55:28,135 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Successfully loaded /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml. 2023-11-08T06:55:28,136 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - [PID]21642 2023-11-08T06:55:28,137 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Torch worker started. 2023-11-08T06:55:28,137 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STOPPED -> WORKER_STARTED 2023-11-08T06:55:28,137 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000 2023-11-08T06:55:28,137 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Python runtime: 3.10.11 2023-11-08T06:55:28,140 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req. to backend at: 1699426528140 2023-11-08T06:55:28,141 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-11-08T06:55:28,166 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - model_name: yolo_tiny, batchSize: 1 2023-11-08T06:55:28,947 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Backend worker process died. 2023-11-08T06:55:28,947 [INFO ] epollEventLoopGroup-5-3 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED 2023-11-08T06:55:28,948 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-11-08T06:55:28,948 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 221, in 2023-11-08T06:55:28,948 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED 2023-11-08T06:55:28,949 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - worker.run_server() 2023-11-08T06:55:28,949 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died. java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432) ~[?:?] at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:191) [model-server.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:829) [?:?] 2023-11-08T06:55:28,950 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 189, in run_server 2023-11-08T06:55:28,950 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket) 2023-11-08T06:55:28,950 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: yolo_tiny, error: Worker died. 2023-11-08T06:55:28,950 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 154, in handle_connection 2023-11-08T06:55:28,950 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STARTED -> WORKER_STOPPED 2023-11-08T06:55:28,950 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg) 2023-11-08T06:55:28,951 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 118, in load_model 2023-11-08T06:55:28,951 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:55:28,951 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:55:28,951 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service = model_loader.load( 2023-11-08T06:55:28,951 [INFO ] W-9000-yolo_tiny_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:55:28,951 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 2 seconds. 2023-11-08T06:55:28,967 [INFO ] W-9000-yolo_tiny_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:55:30,952 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/ec2-user/anaconda3/envs/pytorch_p310/bin/python, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml] 2023-11-08T06:55:32,387 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-11-08T06:55:32,391 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Successfully loaded /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml. 2023-11-08T06:55:32,392 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - [PID]21650 2023-11-08T06:55:32,392 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Torch worker started. 2023-11-08T06:55:32,392 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STOPPED -> WORKER_STARTED 2023-11-08T06:55:32,393 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000 2023-11-08T06:55:32,393 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Python runtime: 3.10.11 2023-11-08T06:55:32,396 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req. to backend at: 1699426532395 2023-11-08T06:55:32,398 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-11-08T06:55:32,417 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - model_name: yolo_tiny, batchSize: 1 2023-11-08T06:55:33,199 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Backend worker process died. 2023-11-08T06:55:33,200 [INFO ] epollEventLoopGroup-5-4 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED 2023-11-08T06:55:33,201 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-11-08T06:55:33,201 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 221, in 2023-11-08T06:55:33,202 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED 2023-11-08T06:55:33,202 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - worker.run_server() 2023-11-08T06:55:33,202 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died. java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432) ~[?:?] at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:191) [model-server.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:829) [?:?] 2023-11-08T06:55:33,203 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: yolo_tiny, error: Worker died. 2023-11-08T06:55:33,203 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STARTED -> WORKER_STOPPED 2023-11-08T06:55:33,203 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:55:33,203 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:55:33,204 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 3 seconds. 2023-11-08T06:55:33,204 [INFO ] W-9000-yolo_tiny_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:55:33,218 [INFO ] W-9000-yolo_tiny_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:55:36,204 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/ec2-user/anaconda3/envs/pytorch_p310/bin/python, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml] 2023-11-08T06:55:37,651 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-11-08T06:55:37,656 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Successfully loaded /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml. 2023-11-08T06:55:37,656 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - [PID]21699 2023-11-08T06:55:37,657 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Torch worker started. 2023-11-08T06:55:37,657 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Python runtime: 3.10.11 2023-11-08T06:55:37,657 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STOPPED -> WORKER_STARTED 2023-11-08T06:55:37,657 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000 2023-11-08T06:55:37,659 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-11-08T06:55:37,659 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req. to backend at: 1699426537659 2023-11-08T06:55:37,680 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - model_name: yolo_tiny, batchSize: 1 2023-11-08T06:55:38,472 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Backend worker process died. 2023-11-08T06:55:38,472 [INFO ] epollEventLoopGroup-5-5 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED 2023-11-08T06:55:38,473 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-11-08T06:55:38,473 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 221, in 2023-11-08T06:55:38,473 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED 2023-11-08T06:55:38,473 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - worker.run_server() 2023-11-08T06:55:38,473 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 189, in run_server 2023-11-08T06:55:38,474 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket) 2023-11-08T06:55:38,474 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 154, in handle_connection 2023-11-08T06:55:38,473 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died. java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432) ~[?:?] at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:191) [model-server.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:829) [?:?] 2023-11-08T06:55:38,474 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg) 2023-11-08T06:55:38,474 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 118, in load_model 2023-11-08T06:55:38,474 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: yolo_tiny, error: Worker died. 2023-11-08T06:55:38,474 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service = model_loader.load( 2023-11-08T06:55:38,475 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STARTED -> WORKER_STOPPED 2023-11-08T06:55:38,475 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:55:38,475 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:55:38,476 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 5 seconds. 2023-11-08T06:55:38,475 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_loader.py", line 135, in load 2023-11-08T06:55:38,476 [INFO ] W-9000-yolo_tiny_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:55:38,489 [INFO ] W-9000-yolo_tiny_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:55:43,476 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/ec2-user/anaconda3/envs/pytorch_p310/bin/python, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml] 2023-11-08T06:55:44,980 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-11-08T06:55:44,987 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Successfully loaded /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml. 2023-11-08T06:55:44,988 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - [PID]21774 2023-11-08T06:55:44,988 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Torch worker started. 2023-11-08T06:55:44,988 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STOPPED -> WORKER_STARTED 2023-11-08T06:55:44,988 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Python runtime: 3.10.11 2023-11-08T06:55:44,988 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000 2023-11-08T06:55:44,992 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-11-08T06:55:44,999 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req. to backend at: 1699426544999 2023-11-08T06:55:45,033 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - model_name: yolo_tiny, batchSize: 1 2023-11-08T06:55:45,941 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Backend worker process died. 2023-11-08T06:55:45,942 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-11-08T06:55:45,942 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 221, in 2023-11-08T06:55:45,942 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - worker.run_server() 2023-11-08T06:55:45,942 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 189, in run_server 2023-11-08T06:55:45,942 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket) 2023-11-08T06:55:45,942 [INFO ] epollEventLoopGroup-5-6 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED 2023-11-08T06:55:45,942 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 154, in handle_connection 2023-11-08T06:55:45,943 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg) 2023-11-08T06:55:45,943 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED 2023-11-08T06:55:45,943 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 118, in load_model 2023-11-08T06:55:45,943 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service = model_loader.load( 2023-11-08T06:55:45,943 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_loader.py", line 135, in load 2023-11-08T06:55:45,943 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - initialize_fn(service.context) 2023-11-08T06:55:45,943 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/tmp/models/e0675cae7c07449daa30bcd78ecd3a99/handler.py", line 21, in initialize 2023-11-08T06:55:45,943 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died. java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432) ~[?:?] at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:191) [model-server.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:829) [?:?] 2023-11-08T06:55:45,944 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.model = YolosForObjectDetection.from_pretrained(model_dir) 2023-11-08T06:55:45,944 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2864, in from_pretrained 2023-11-08T06:55:45,944 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: yolo_tiny, error: Worker died. 2023-11-08T06:55:45,944 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - raise EnvironmentError( 2023-11-08T06:55:45,944 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STARTED -> WORKER_STOPPED 2023-11-08T06:55:45,945 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:55:45,945 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:55:45,945 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 8 seconds. 2023-11-08T06:55:45,948 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - OSError: Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory /tmp/models/e0675cae7c07449daa30bcd78ecd3a99. 2023-11-08T06:55:45,948 [INFO ] W-9000-yolo_tiny_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:55:45,968 [INFO ] W-9000-yolo_tiny_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:55:53,946 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/ec2-user/anaconda3/envs/pytorch_p310/bin/python, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml] 2023-11-08T06:55:55,342 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-11-08T06:55:55,347 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Successfully loaded /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml. 2023-11-08T06:55:55,347 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - [PID]21889 2023-11-08T06:55:55,347 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Torch worker started. 2023-11-08T06:55:55,347 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Python runtime: 3.10.11 2023-11-08T06:55:55,347 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STOPPED -> WORKER_STARTED 2023-11-08T06:55:55,347 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000 2023-11-08T06:55:55,349 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req. to backend at: 1699426555349 2023-11-08T06:55:55,355 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-11-08T06:55:55,381 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - model_name: yolo_tiny, batchSize: 1 2023-11-08T06:55:56,156 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Backend worker process died. 2023-11-08T06:55:56,157 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-11-08T06:55:56,157 [INFO ] epollEventLoopGroup-5-7 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED 2023-11-08T06:55:56,158 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 221, in 2023-11-08T06:55:56,158 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - worker.run_server() 2023-11-08T06:55:56,158 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED 2023-11-08T06:55:56,158 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 189, in run_server 2023-11-08T06:55:56,159 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket) 2023-11-08T06:55:56,159 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 154, in handle_connection 2023-11-08T06:55:56,159 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg) 2023-11-08T06:55:56,158 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died. java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432) ~[?:?] at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:191) [model-server.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:829) [?:?] 2023-11-08T06:55:56,159 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: yolo_tiny, error: Worker died. 2023-11-08T06:55:56,159 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STARTED -> WORKER_STOPPED 2023-11-08T06:55:56,160 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:55:56,160 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:55:56,160 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 13 seconds. 2023-11-08T06:55:56,159 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 118, in load_model 2023-11-08T06:55:56,161 [INFO ] W-9000-yolo_tiny_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:55:56,176 [INFO ] W-9000-yolo_tiny_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:56:09,161 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/ec2-user/anaconda3/envs/pytorch_p310/bin/python, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml] 2023-11-08T06:56:10,617 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-11-08T06:56:10,624 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Successfully loaded /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml. 2023-11-08T06:56:10,625 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - [PID]22032 2023-11-08T06:56:10,625 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Torch worker started. 2023-11-08T06:56:10,625 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STOPPED -> WORKER_STARTED 2023-11-08T06:56:10,626 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000 2023-11-08T06:56:10,627 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Python runtime: 3.10.11 2023-11-08T06:56:10,628 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req. to backend at: 1699426570628 2023-11-08T06:56:10,635 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-11-08T06:56:10,661 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - model_name: yolo_tiny, batchSize: 1 2023-11-08T06:56:11,428 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Backend worker process died. 2023-11-08T06:56:11,428 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-11-08T06:56:11,428 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 221, in 2023-11-08T06:56:11,429 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - worker.run_server() 2023-11-08T06:56:11,429 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 189, in run_server 2023-11-08T06:56:11,429 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket) 2023-11-08T06:56:11,429 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 154, in handle_connection 2023-11-08T06:56:11,429 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg) 2023-11-08T06:56:11,429 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 118, in load_model 2023-11-08T06:56:11,429 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service = model_loader.load( 2023-11-08T06:56:11,429 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_loader.py", line 135, in load 2023-11-08T06:56:11,430 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - initialize_fn(service.context) 2023-11-08T06:56:11,430 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/tmp/models/e0675cae7c07449daa30bcd78ecd3a99/handler.py", line 21, in initialize 2023-11-08T06:56:11,429 [INFO ] epollEventLoopGroup-5-8 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED 2023-11-08T06:56:11,430 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.model = YolosForObjectDetection.from_pretrained(model_dir) 2023-11-08T06:56:11,430 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2864, in from_pretrained 2023-11-08T06:56:11,430 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - raise EnvironmentError( 2023-11-08T06:56:11,430 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - OSError: Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory /tmp/models/e0675cae7c07449daa30bcd78ecd3a99. 2023-11-08T06:56:11,430 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED 2023-11-08T06:56:11,431 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died. java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432) ~[?:?] at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:191) [model-server.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:829) [?:?] 2023-11-08T06:56:11,431 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: yolo_tiny, error: Worker died. 2023-11-08T06:56:11,431 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STARTED -> WORKER_STOPPED 2023-11-08T06:56:11,432 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:56:11,432 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:56:11,432 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 21 seconds. 2023-11-08T06:56:11,446 [INFO ] W-9000-yolo_tiny_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:56:11,446 [INFO ] W-9000-yolo_tiny_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:56:20,451 [INFO ] pool-3-thread-2 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426580 2023-11-08T06:56:20,452 [INFO ] pool-3-thread-2 TS_METRICS - DiskAvailable.Gigabytes:67.2320785522461|#Level:Host|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426580 2023-11-08T06:56:20,452 [INFO ] pool-3-thread-2 TS_METRICS - DiskUsage.Gigabytes:112.75618362426758|#Level:Host|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426580 2023-11-08T06:56:20,452 [INFO ] pool-3-thread-2 TS_METRICS - DiskUtilization.Percent:62.6|#Level:Host|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426580 2023-11-08T06:56:20,453 [INFO ] pool-3-thread-2 TS_METRICS - GPUMemoryUtilization.Percent:0.0|#Level:Host,device_id:0|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426580 2023-11-08T06:56:20,453 [INFO ] pool-3-thread-2 TS_METRICS - GPUMemoryUsed.Megabytes:0|#Level:Host,device_id:0|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426580 2023-11-08T06:56:20,453 [INFO ] pool-3-thread-2 TS_METRICS - GPUUtilization.Percent:0|#Level:Host,device_id:0|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426580 2023-11-08T06:56:20,453 [INFO ] pool-3-thread-2 TS_METRICS - MemoryAvailable.Megabytes:13114.09765625|#Level:Host|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426580 2023-11-08T06:56:20,454 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUsed.Megabytes:2248.6640625|#Level:Host|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426580 2023-11-08T06:56:20,454 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUtilization.Percent:16.4|#Level:Host|#hostname:ip-10-194-163-225.eu-central-1.compute.internal,timestamp:1699426580 2023-11-08T06:56:32,432 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/ec2-user/anaconda3/envs/pytorch_p310/bin/python, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml] 2023-11-08T06:56:33,873 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-11-08T06:56:33,879 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Successfully loaded /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml. 2023-11-08T06:56:33,879 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - [PID]22253 2023-11-08T06:56:33,880 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STOPPED -> WORKER_STARTED 2023-11-08T06:56:33,880 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000 2023-11-08T06:56:33,880 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Torch worker started. 2023-11-08T06:56:33,881 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Python runtime: 3.10.11 2023-11-08T06:56:33,882 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req. to backend at: 1699426593882 2023-11-08T06:56:33,882 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-11-08T06:56:33,883 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - model_name: yolo_tiny, batchSize: 1 2023-11-08T06:56:34,758 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Backend worker process died. 2023-11-08T06:56:34,758 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-11-08T06:56:34,758 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 221, in 2023-11-08T06:56:34,758 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED 2023-11-08T06:56:34,758 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - worker.run_server() 2023-11-08T06:56:34,759 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 189, in run_server 2023-11-08T06:56:34,759 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket) 2023-11-08T06:56:34,759 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 154, in handle_connection 2023-11-08T06:56:34,759 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg) 2023-11-08T06:56:34,759 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED 2023-11-08T06:56:34,759 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 118, in load_model 2023-11-08T06:56:34,759 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service = model_loader.load( 2023-11-08T06:56:34,759 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_loader.py", line 135, in load 2023-11-08T06:56:34,759 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - initialize_fn(service.context) 2023-11-08T06:56:34,760 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/tmp/models/e0675cae7c07449daa30bcd78ecd3a99/handler.py", line 21, in initialize 2023-11-08T06:56:34,760 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.model = YolosForObjectDetection.from_pretrained(model_dir) 2023-11-08T06:56:34,761 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2864, in from_pretrained 2023-11-08T06:56:34,761 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - raise EnvironmentError( 2023-11-08T06:56:34,759 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died. java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432) ~[?:?] at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:191) [model-server.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:829) [?:?] 2023-11-08T06:56:34,761 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: yolo_tiny, error: Worker died. 2023-11-08T06:56:34,761 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STARTED -> WORKER_STOPPED 2023-11-08T06:56:34,761 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:56:34,761 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:56:34,762 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 34 seconds. 2023-11-08T06:56:34,761 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - OSError: Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory /tmp/models/e0675cae7c07449daa30bcd78ecd3a99. 2023-11-08T06:56:34,762 [INFO ] W-9000-yolo_tiny_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:56:34,776 [INFO ] W-9000-yolo_tiny_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:57:08,762 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/ec2-user/anaconda3/envs/pytorch_p310/bin/python, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml] 2023-11-08T06:57:10,149 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-11-08T06:57:10,154 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Successfully loaded /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml. 2023-11-08T06:57:10,154 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - [PID]22584 2023-11-08T06:57:10,154 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Torch worker started. 2023-11-08T06:57:10,154 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Python runtime: 3.10.11 2023-11-08T06:57:10,154 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STOPPED -> WORKER_STARTED 2023-11-08T06:57:10,155 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000 2023-11-08T06:57:10,156 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req. to backend at: 1699426630156 2023-11-08T06:57:10,156 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-11-08T06:57:10,156 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - model_name: yolo_tiny, batchSize: 1 2023-11-08T06:57:10,987 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Backend worker process died. 2023-11-08T06:57:10,987 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-11-08T06:57:10,987 [INFO ] epollEventLoopGroup-5-2 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED 2023-11-08T06:57:10,987 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 221, in 2023-11-08T06:57:10,987 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - worker.run_server() 2023-11-08T06:57:10,988 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 189, in run_server 2023-11-08T06:57:10,988 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket) 2023-11-08T06:57:10,988 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 154, in handle_connection 2023-11-08T06:57:10,988 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg) 2023-11-08T06:57:10,987 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED 2023-11-08T06:57:10,988 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 118, in load_model 2023-11-08T06:57:10,988 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service = model_loader.load( 2023-11-08T06:57:10,988 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_loader.py", line 135, in load 2023-11-08T06:57:10,988 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - initialize_fn(service.context) 2023-11-08T06:57:10,988 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/tmp/models/e0675cae7c07449daa30bcd78ecd3a99/handler.py", line 21, in initialize 2023-11-08T06:57:10,988 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died. java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432) ~[?:?] at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:191) [model-server.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:829) [?:?] 2023-11-08T06:57:10,988 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.model = YolosForObjectDetection.from_pretrained(model_dir) 2023-11-08T06:57:10,988 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: yolo_tiny, error: Worker died. 2023-11-08T06:57:10,989 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STARTED -> WORKER_STOPPED 2023-11-08T06:57:10,989 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:57:10,989 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:57:10,989 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 55 seconds. 2023-11-08T06:57:10,989 [INFO ] W-9000-yolo_tiny_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:57:11,003 [INFO ] W-9000-yolo_tiny_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:58:05,990 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/ec2-user/anaconda3/envs/pytorch_p310/bin/python, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml] 2023-11-08T06:58:07,410 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-11-08T06:58:07,415 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Successfully loaded /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml. 2023-11-08T06:58:07,415 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - [PID]23096 2023-11-08T06:58:07,415 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STOPPED -> WORKER_STARTED 2023-11-08T06:58:07,415 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000 2023-11-08T06:58:07,416 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Torch worker started. 2023-11-08T06:58:07,419 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Python runtime: 3.10.11 2023-11-08T06:58:07,420 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-11-08T06:58:07,420 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req. to backend at: 1699426687420 2023-11-08T06:58:07,431 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - model_name: yolo_tiny, batchSize: 1 2023-11-08T06:58:08,197 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Backend worker process died. 2023-11-08T06:58:08,197 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-11-08T06:58:08,197 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 221, in 2023-11-08T06:58:08,197 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - worker.run_server() 2023-11-08T06:58:08,197 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 189, in run_server 2023-11-08T06:58:08,197 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket) 2023-11-08T06:58:08,198 [INFO ] epollEventLoopGroup-5-3 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED 2023-11-08T06:58:08,198 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 154, in handle_connection 2023-11-08T06:58:08,198 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg) 2023-11-08T06:58:08,198 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 118, in load_model 2023-11-08T06:58:08,198 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED 2023-11-08T06:58:08,198 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service = model_loader.load( 2023-11-08T06:58:08,198 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_loader.py", line 135, in load 2023-11-08T06:58:08,199 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - initialize_fn(service.context) 2023-11-08T06:58:08,199 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/tmp/models/e0675cae7c07449daa30bcd78ecd3a99/handler.py", line 21, in initialize 2023-11-08T06:58:08,199 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.model = YolosForObjectDetection.from_pretrained(model_dir) 2023-11-08T06:58:08,198 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died. java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432) ~[?:?] at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:191) [model-server.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:829) [?:?] 2023-11-08T06:58:08,199 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: yolo_tiny, error: Worker died. 2023-11-08T06:58:08,203 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STARTED -> WORKER_STOPPED 2023-11-08T06:58:08,203 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:58:08,203 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:58:08,203 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 89 seconds. 2023-11-08T06:58:08,199 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2864, in from_pretrained 2023-11-08T06:58:08,204 [INFO ] W-9000-yolo_tiny_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:58:08,218 [INFO ] W-9000-yolo_tiny_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:59:37,204 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/ec2-user/anaconda3/envs/pytorch_p310/bin/python, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml] 2023-11-08T06:59:38,625 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-11-08T06:59:38,629 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Successfully loaded /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml. 2023-11-08T06:59:38,629 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - [PID]23984 2023-11-08T06:59:38,630 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Torch worker started. 2023-11-08T06:59:38,630 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Python runtime: 3.10.11 2023-11-08T06:59:38,630 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STOPPED -> WORKER_STARTED 2023-11-08T06:59:38,630 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000 2023-11-08T06:59:38,632 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req. to backend at: 1699426778632 2023-11-08T06:59:38,639 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-11-08T06:59:38,639 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - model_name: yolo_tiny, batchSize: 1 2023-11-08T06:59:39,404 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Backend worker process died. 2023-11-08T06:59:39,404 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-11-08T06:59:39,404 [INFO ] epollEventLoopGroup-5-4 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED 2023-11-08T06:59:39,404 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 221, in 2023-11-08T06:59:39,404 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - worker.run_server() 2023-11-08T06:59:39,404 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 189, in run_server 2023-11-08T06:59:39,404 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket) 2023-11-08T06:59:39,404 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 154, in handle_connection 2023-11-08T06:59:39,404 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED 2023-11-08T06:59:39,405 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg) 2023-11-08T06:59:39,405 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 118, in load_model 2023-11-08T06:59:39,405 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service = model_loader.load( 2023-11-08T06:59:39,405 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_loader.py", line 135, in load 2023-11-08T06:59:39,405 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died. java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432) ~[?:?] at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:191) [model-server.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:829) [?:?] 2023-11-08T06:59:39,406 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - initialize_fn(service.context) 2023-11-08T06:59:39,406 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/tmp/models/e0675cae7c07449daa30bcd78ecd3a99/handler.py", line 21, in initialize 2023-11-08T06:59:39,407 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.model = YolosForObjectDetection.from_pretrained(model_dir) 2023-11-08T06:59:39,407 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2864, in from_pretrained 2023-11-08T06:59:39,407 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - raise EnvironmentError( 2023-11-08T06:59:39,407 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - OSError: Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory /tmp/models/e0675cae7c07449daa30bcd78ecd3a99. 2023-11-08T06:59:39,406 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: yolo_tiny, error: Worker died. 2023-11-08T06:59:39,407 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STARTED -> WORKER_STOPPED 2023-11-08T06:59:39,407 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stderr 2023-11-08T06:59:39,408 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:59:39,408 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 144 seconds. 2023-11-08T06:59:39,421 [INFO ] W-9000-yolo_tiny_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stdout 2023-11-08T06:59:39,421 [INFO ] W-9000-yolo_tiny_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stderr 2023-11-08T07:02:03,409 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/ec2-user/anaconda3/envs/pytorch_p310/bin/python, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml] 2023-11-08T07:02:04,874 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-11-08T07:02:04,880 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Successfully loaded /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/configs/metrics.yaml. 2023-11-08T07:02:04,880 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - [PID]25366 2023-11-08T07:02:04,880 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Torch worker started. 2023-11-08T07:02:04,880 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Python runtime: 3.10.11 2023-11-08T07:02:04,881 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STOPPED -> WORKER_STARTED 2023-11-08T07:02:04,881 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000 2023-11-08T07:02:04,882 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-11-08T07:02:04,882 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req. to backend at: 1699426924882 2023-11-08T07:02:04,883 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - model_name: yolo_tiny, batchSize: 1 2023-11-08T07:02:05,652 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Backend worker process died. 2023-11-08T07:02:05,652 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-11-08T07:02:05,653 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 221, in 2023-11-08T07:02:05,653 [INFO ] epollEventLoopGroup-5-5 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED 2023-11-08T07:02:05,653 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED 2023-11-08T07:02:05,653 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - worker.run_server() 2023-11-08T07:02:05,653 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 189, in run_server 2023-11-08T07:02:05,654 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket) 2023-11-08T07:02:05,654 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/ts/model_service_worker.py", line 154, in handle_connection 2023-11-08T07:02:05,653 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died. java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432) ~[?:?] at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:191) [model-server.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:829) [?:?] 2023-11-08T07:02:05,654 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: yolo_tiny, error: Worker died. 2023-11-08T07:02:05,654 [DEBUG] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-yolo_tiny_1.0 State change WORKER_STARTED -> WORKER_STOPPED 2023-11-08T07:02:05,655 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stderr 2023-11-08T07:02:05,655 [WARN ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-yolo_tiny_1.0-stdout 2023-11-08T07:02:05,655 [INFO ] W-9000-yolo_tiny_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 233 seconds. 2023-11-08T07:02:05,654 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg) 2023-11-08T07:02:05,655 [INFO ] W-9000-yolo_tiny_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stdout 2023-11-08T07:02:05,675 [INFO ] W-9000-yolo_tiny_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-yolo_tiny_1.0-stderr

lxning commented 9 months ago

@naveenjr The log shows "user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2864, in from_pretrained 2023-11-08T06:55:22,325 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - raise EnvironmentError( 2023-11-08T06:55:22,326 [INFO ] W-9000-yolo_tiny_1.0-stdout MODEL_LOG - OSError: Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory /tmp/models/e0675cae7c07449daa30bcd78ecd3a99."

it seems that you didn't provided all the model files in model artifacts.