Open navaneethakarli opened 10 months ago
agunapal
commented
10 months ago Can you please share a link to the example/model you are trying and what does your handler look like. You could try to first run the handler standalone to make sure it works
navaneethakarli
commented
10 months ago import torch import torchaudio from transformers import WhisperProcessor, WhisperForConditionalGeneration import io
class SpeechToTextHandler(object): def init(self): self._context = None self.initialized = False self.model = None self.processor = None self.device = None
self.expected_sampling_rate = 16_000
def initialize(self, context):
"""Initialize properties and load model"""
self._context = context
self.initialized = True
properties = context.system_properties
# See https://pytorch.org/serve/custom_service.html#handling-model-execution-on-multiple-gpus
self.device = torch.device("cuda:" + str(properties.get("gpu_id")) if torch.cuda.is_available() else "cpu")
model_dir = properties.get("model_dir")
self.processor = WhisperProcessor.from_pretrained(model_dir)
self.model = WhisperForConditionalGeneration.from_pretrained(model_dir)
self.forced_decoder_ids = self.processor.get_decoder_prompt_ids(language="sinhala", task="transcribe")
def handle(self, data, context):
"""Transform input to tensor, resample, run model, and return transcribed text."""
input = data[0].get("data")
if input is None:
input = data[0].get("body")
input = input.decode('utf-8')
# Load the audio using torchaudio and resample if needed
waveform, sample_rate = torchaudio.load(io.BytesIO(input), format="wav")
# Ensure the sampling rate is the same as the trained model
if sample_rate != self.expected_sampling_rate:
resampler = torchaudio.transforms.Resample(orig_freq=sample_rate, new_freq=self.expected_sampling_rate)
waveform = resampler(waveform)
print("Input Shape Before Preprocessing:", waveform.shape) # Debugging line
# Process the waveform using the processor
input_features = self.processor(waveform.numpy(), sampling_rate=self.expected_sampling_rate, return_tensors="pt").input_features
predicted_ids = self.model.generate(input_features, forced_decoder_ids=self.forced_decoder_ids)
transcription = self.processor.batch_decode(predicted_ids, skip_special_tokens=True)
return [transcription]_service = SpeechToTextHandler()
def handle(data, context): """ Entry point for SpeechToTextHandler handler """ try: if not _service.initialized: _service.initialize(context)
if data is None:
return None
return _service.handle(data, context)
except Exception as e:
raise Exception("Unable to process input data. " + str(e))i am using this code for asr whisper and also i want text_to_speech of microsoft/speecht5_tts i am not getting any reference.please give me suggestions
navaneethakarli
commented
10 months ago i am susing fine-tune models
agunapal
commented
9 months ago Hi @navaneethakarli You can debug your handler code using this example.
Please let me know if this helps
navaneethakarli
commented
9 months ago is it possible using of serialized file as .bin file i am using .bin file for all models . i am getting this error for my models 2023-10-03T12:54:01,537 [DEBUG] W-9000-spp_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
š Describe the bug
I am trying to surve ASR whisper model in torchserve but i am facing worker issue
Error logs
main org.pytorch.serve.servingsdk.impl.PluginsManager - Loading snapshot serializer plugin... 2023-09-08T11:47:27,120 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: torchserve_stt/model_store/speechtotext.mar 2023-09-08T11:47:44,718 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model speechtotext 2023-09-08T11:47:44,718 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model speechtotext 2023-09-08T11:47:44,718 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model speechtotext loaded. 2023-09-08T11:47:44,719 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: speechtotext, count: 1 2023-09-08T11:47:44,727 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml] 2023-09-08T11:47:44,729 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: EpollServerSocketChannel. 2023-09-08T11:47:44,836 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://127.0.0.1:8080 2023-09-08T11:47:44,836 [INFO ] main org.pytorch.serve.ModelServer - Initialize Management server with: EpollServerSocketChannel. 2023-09-08T11:47:44,837 [INFO ] main org.pytorch.serve.ModelServer - Management API bind to: http://127.0.0.1:8081 2023-09-08T11:47:44,837 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: EpollServerSocketChannel. 2023-09-08T11:47:44,838 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://127.0.0.1:8082 Model server started. 2023-09-08T11:47:45,044 [WARN ] pool-3-thread-1 org.pytorch.serve.metrics.MetricCollector - worker pid is not available yet. 2023-09-08T11:47:45,631 [INFO ] pool-3-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:black,timestamp:1694153865 2023-09-08T11:47:45,633 [INFO ] pool-3-thread-1 TS_METRICS - DiskAvailable.Gigabytes:64.30083847045898|#Level:Host|#hostname:black,timestamp:1694153865 2023-09-08T11:47:45,633 [INFO ] pool-3-thread-1 TS_METRICS - DiskUsage.Gigabytes:1803.2123374938965|#Level:Host|#hostname:black,timestamp:1694153865 2023-09-08T11:47:45,633 [INFO ] pool-3-thread-1 TS_METRICS - DiskUtilization.Percent:96.6|#Level:Host|#hostname:black,timestamp:1694153865 2023-09-08T11:47:45,634 [INFO ] pool-3-thread-1 TS_METRICS - GPUMemoryUtilization.Percent:0.02629618267081562|#Level:Host,DeviceId:0|#hostname:black,timestamp:1694153865 2023-09-08T11:47:45,634 [INFO ] pool-3-thread-1 TS_METRICS - GPUMemoryUsed.Megabytes:12.0|#Level:Host,DeviceId:0|#hostname:black,timestamp:1694153865 2023-09-08T11:47:45,634 [INFO ] pool-3-thread-1 TS_METRICS - GPUUtilization.Percent:0.0|#Level:Host,DeviceId:0|#hostname:black,timestamp:1694153865 2023-09-08T11:47:45,634 [INFO ] pool-3-thread-1 TS_METRICS - MemoryAvailable.Megabytes:53097.89453125|#Level:Host|#hostname:black,timestamp:1694153865 2023-09-08T11:47:45,634 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUsed.Megabytes:10246.71875|#Level:Host|#hostname:black,timestamp:1694153865 2023-09-08T11:47:45,635 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUtilization.Percent:17.4|#Level:Host|#hostname:black,timestamp:1694153865 2023-09-08T11:47:46,079 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127192 2023-09-08T11:47:46,084 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000 2023-09-08T11:47:46,087 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml. 2023-09-08T11:47:46,087 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127192 2023-09-08T11:47:46,087 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started. 2023-09-08T11:47:46,088 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10 2023-09-08T11:47:46,088 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change null -> WORKER_STARTED 2023-09-08T11:47:46,093 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000 2023-09-08T11:47:46,100 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000. 2023-09-08T11:47:46,103 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1694153866103 2023-09-08T11:47:46,135 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - model_name: speechtotext, batchSize: 1 2023-09-08T11:47:46,682 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Backend worker process died. 2023-09-08T11:47:46,683 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Traceback (most recent call last): 2023-09-08T11:47:46,683 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py", line 253, in
2023-09-08T11:47:46,683 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - worker.run_server()
2023-09-08T11:47:46,683 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py", line 221, in run_server
2023-09-08T11:47:46,683 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket)
2023-09-08T11:47:46,684 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py", line 184, in handle_connection
2023-09-08T11:47:46,684 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg)
2023-09-08T11:47:46,684 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py", line 131, in load_model
2023-09-08T11:47:46,684 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - service = model_loader.load(
2023-09-08T11:47:46,684 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_loader.py", line 135, in load
2023-09-08T11:47:46,685 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - initialize_fn(service.context)
2023-09-08T11:47:46,685 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/tmp/models/47afe52012fe4966ae64cb2af806c171/handler.py", line 85, in initialize
2023-09-08T11:47:46,685 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - self.processor = WhisperProcessor.from_pretrained(model_dir)
2023-09-08T11:47:46,685 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/transformers/processing_utils.py", line 215, in from_pretrained
2023-09-08T11:47:46,685 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - args = cls._get_arguments_from_pretrained(pretrained_model_name_or_path, kwargs)
2023-09-08T11:47:46,685 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/transformers/processing_utils.py", line 259, in _get_arguments_from_pretrained
2023-09-08T11:47:46,686 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - args.append(attribute_class.from_pretrained(pretrained_model_name_or_path, kwargs))
2023-09-08T11:47:46,686 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1841, in from_pretrained
2023-09-08T11:47:46,686 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - return cls._from_pretrained(
2023-09-08T11:47:46,686 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:47:46,686 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2004, in _from_pretrained
2023-09-08T11:47:46,686 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - tokenizer = cls(*init_inputs, **init_kwargs)
2023-09-08T11:47:46,686 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/transformers/models/whisper/tokenization_whisper.py", line 294, in init
2023-09-08T11:47:46,686 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:47:46,687 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - with open(merges_file, encoding="utf-8") as merges_handle:
2023-09-08T11:47:46,687 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - TypeError: expected str, bytes or os.PathLike object, not NoneType
2023-09-08T11:47:46,687 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432) ~[?:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:213) [model-server.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:47:46,698 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: speechtotext, error: Worker died.
2023-09-08T11:47:46,698 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:47:46,698 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery start timestamp: 1694153866698
2023-09-08T11:47:46,698 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:46,699 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:46,699 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 1 seconds.
2023-09-08T11:47:46,713 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:46,713 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:47,700 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:47:48,978 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127281
2023-09-08T11:47:48,979 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:47:48,987 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:47:48,987 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127281
2023-09-08T11:47:48,987 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:47:48,987 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:47:48,987 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:47:48,988 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:47:48,989 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:47:48,989 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:47:48,990 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:47:48,990 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:47:48,991 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:47:48,991 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:47:48,991 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:48,991 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:48,991 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 1 seconds.
2023-09-08T11:47:49,003 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:49,003 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:49,992 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:47:51,274 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127305
2023-09-08T11:47:51,275 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:47:51,283 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:47:51,283 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127305
2023-09-08T11:47:51,283 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:47:51,283 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:47:51,283 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:47:51,284 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:47:51,285 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:47:51,285 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:47:51,285 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:47:51,286 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:47:51,286 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:47:51,286 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:47:51,286 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:51,286 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:51,287 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 2 seconds.
2023-09-08T11:47:51,300 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:51,300 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:53,287 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:47:54,567 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127341
2023-09-08T11:47:54,567 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:47:54,576 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:47:54,576 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127341
2023-09-08T11:47:54,576 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:47:54,576 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:47:54,576 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:47:54,576 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:47:54,578 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:47:54,578 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:47:54,578 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:47:54,578 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:47:54,579 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:47:54,579 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:47:54,580 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:54,580 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:54,580 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 3 seconds.
2023-09-08T11:47:54,593 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:54,593 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:57,581 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:47:58,848 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127382
2023-09-08T11:47:58,849 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:47:58,857 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:47:58,857 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127382
2023-09-08T11:47:58,857 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:47:58,857 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:47:58,857 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:47:58,858 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:47:58,859 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:47:58,859 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:47:58,861 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:47:58,861 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:47:58,862 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:47:58,862 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:47:58,862 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:58,862 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:58,862 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 5 seconds.
2023-09-08T11:47:58,873 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:58,873 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:48:03,863 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:48:05,216 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127418
2023-09-08T11:48:05,216 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:48:05,224 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:48:05,225 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127418
2023-09-08T11:48:05,225 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:48:05,225 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:48:05,225 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:48:05,225 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:48:05,227 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:48:05,227 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:48:05,227 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:48:05,227 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:48:05,228 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:48:05,228 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:48:05,228 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:48:05,228 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:48:05,229 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 8 seconds.
2023-09-08T11:48:05,240 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:48:05,240 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:48:13,229 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:48:14,608 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127471
2023-09-08T11:48:14,609 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:48:14,617 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:48:14,617 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127471
2023-09-08T11:48:14,617 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:48:14,617 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:48:14,617 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:48:14,617 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:48:14,619 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:48:14,619 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:48:14,619 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:48:14,619 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:48:14,620 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:48:14,620 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:48:14,620 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:48:14,620 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:48:14,620 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 13 seconds.
2023-09-08T11:48:14,631 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:48:14,631 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:48:27,621 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:48:28,901 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127510
2023-09-08T11:48:28,901 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:48:28,909 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:48:28,910 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127510
2023-09-08T11:48:28,910 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:48:28,910 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:48:28,910 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:48:28,910 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:48:28,913 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:48:28,913 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:48:28,913 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:48:28,913 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:48:28,913 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:48:28,913 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:48:28,914 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:48:28,914 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:48:28,914 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 21 seconds.
2023-09-08T11:48:28,924 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:48:28,924 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:48:45,629 [INFO ] pool-3-thread-2 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,630 [INFO ] pool-3-thread-2 TS_METRICS - DiskAvailable.Gigabytes:64.30076217651367|#Level:Host|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,630 [INFO ] pool-3-thread-2 TS_METRICS - DiskUsage.Gigabytes:1803.2124137878418|#Level:Host|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,630 [INFO ] pool-3-thread-2 TS_METRICS - DiskUtilization.Percent:96.6|#Level:Host|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,630 [INFO ] pool-3-thread-2 TS_METRICS - GPUMemoryUtilization.Percent:0.02629618267081562|#Level:Host,DeviceId:0|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,631 [INFO ] pool-3-thread-2 TS_METRICS - GPUMemoryUsed.Megabytes:12.0|#Level:Host,DeviceId:0|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,631 [INFO ] pool-3-thread-2 TS_METRICS - GPUUtilization.Percent:0.0|#Level:Host,DeviceId:0|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,631 [INFO ] pool-3-thread-2 TS_METRICS - MemoryAvailable.Megabytes:53042.8046875|#Level:Host|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,631 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUsed.Megabytes:10301.76953125|#Level:Host|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,631 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUtilization.Percent:17.5|#Level:Host|#hostname:black,timestamp:1694153925
2023-09-08T11:48:49,914 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:48:51,243 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127607
2023-09-08T11:48:51,243 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:48:51,252 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:48:51,252 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127607
2023-09-08T11:48:51,252 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:48:51,252 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:48:51,252 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:48:51,253 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:48:51,256 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:48:51,256 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:48:51,256 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:48:51,256 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:48:51,256 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:48:51,257 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:48:51,257 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:48:51,257 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:48:51,257 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 34 seconds.
2023-09-08T11:48:51,270 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:48:51,270 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:49:25,258 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:49:26,828 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127660
2023-09-08T11:49:26,829 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:49:26,837 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:49:26,838 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127660
2023-09-08T11:49:26,838 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:49:26,838 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:49:26,838 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:49:26,838 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:49:26,841 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:49:26,841 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:49:26,841 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:49:26,841 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:49:26,842 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:49:26,842 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:49:26,842 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:49:26,842 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:49:26,842 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 55 seconds.
2023-09-08T11:49:26,853 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:49:26,853 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:50:21,844 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLi
Installation instructions
i am not using Docker i am using server
Model Packaing
!/bin/bash
set -euo pipefail
mkdir -p model_store
Extra files add all files necessary for processor
torch-model-archiver --model-name speechtotext --version 1.0 --serialized-file model/pytorch_model.bin --handler ./handler.py --extra-files "model/config.json,model/special_tokens_map.json,model/tokenizer_config.json,model/vocab.json,model/preprocessor_config.json" -f mv speechtotext.mar model_store
config.properties
No response
Versions
absl-py==1.4.0 accelerate==0.22.0 aiohttp==3.8.5 aiosignal==1.3.1 aniso8601==9.0.1 ansi2html==1.8.0 arrow==1.2.3 asttokens==2.2.1 async-timeout==4.0.2 attrs==23.1.0 backcall==0.2.0 blinker==1.6.2 certifi==2023.7.22 cffi==1.15.1 charset-normalizer==3.2.0 click==8.1.6 cmake==3.27.0 colorama==0.4.6 coloredlogs==15.0.1 contourpy==1.1.0 cycler==0.11.0 datasets==2.14.2 decorator==5.1.1 dill==0.3.7 enum-compat==0.0.3 evaluate==0.4.0 exceptiongroup==1.1.3 executing==1.2.0 ffmpeg==1.4 ffmpeg-python==0.2.0 filelock==3.12.2 Flask==2.3.2 Flask-RESTful==0.3.10 flatbuffers==23.5.26 fonttools==4.42.1 frozenlist==1.4.0 fsspec==2023.6.0 future==0.18.3 huggingface-hub==0.16.4 humanfriendly==10.0 idna==3.4 importlib-metadata==6.8.0 importlib-resources==6.0.1 iniconfig==2.0.0 install==1.3.5 ipython==8.12.2 itsdangerous==2.1.2 jedi==0.19.0 Jinja2==3.1.2 joblib==1.3.1 kiwisolver==1.4.5 lit==16.0.6 lxml==4.9.3 MarkupSafe==2.1.3 matplotlib==3.7.2 matplotlib-inline==0.1.6 mpmath==1.3.0 multidict==6.0.4 multiprocess==0.70.15 networkx==3.1 nltk==3.8.1 numpy==1.24.4 nvgpu==0.9.0 nvidia-cublas-cu11==11.10.3.66 nvidia-cuda-cupti-cu11==11.7.101 nvidia-cuda-nvrtc-cu11==11.7.99 nvidia-cuda-runtime-cu11==11.7.99 nvidia-cudnn-cu11==8.5.0.96 nvidia-cufft-cu11==10.9.0.58 nvidia-curand-cu11==10.2.10.91 nvidia-cusolver-cu11==11.4.0.1 nvidia-cusparse-cu11==11.7.4.91 nvidia-nccl-cu11==2.14.3 nvidia-nvtx-cu11==11.7.91 onnx==1.14.0 onnxruntime==1.15.1 optimum==1.10.1 packaging==23.1 pandas==2.0.3 parso==0.8.3 pexpect==4.8.0 pickleshare==0.7.5 Pillow==10.0.0 pluggy==1.2.0 portalocker==2.7.0 prompt-toolkit==3.0.39 protobuf==4.23.4 psutil==5.9.5 ptyprocess==0.7.0 pure-eval==0.2.2 pyarrow==12.0.1 pycparser==2.21 Pygments==2.16.1 pynvml==11.4.1 pyparsing==3.0.9 pyreadline3==3.4.1 pytest==7.4.0 python-dateutil==2.8.2 pytz==2023.3 PyYAML==6.0.1 regex==2023.6.3 requests==2.31.0 responses==0.18.0 rouge-score==0.1.2 sacrebleu==2.3.1 sacremoses==0.0.53 safetensors==0.3.1 scikit-learn==1.3.0 scipy==1.10.1 sentencepiece==0.1.99 six==1.16.0 sklearn==0.0.post7 soundfile==0.12.1 stack-data==0.6.2 sympy==1.12 tabulate==0.9.0 termcolor==2.3.0 threadpoolctl==3.2.0 tokenizers==0.13.3 tomli==2.0.1 torch==2.0.0 torch-model-archiver==0.8.1 torchaudio==2.0.1 torchdata==0.6.1 torchserve==0.8.1 torchtext==0.15.2 torchvision==0.15.1 tqdm==4.65.0 traitlets==5.9.0 transformers==4.31.0 triton==2.0.0 typing-extensions==4.7.1 tzdata==2023.3 urllib3==2.0.4 wcwidth==0.2.6 Werkzeug==2.3.6 whisper==1.1.10 xxhash==3.3.0 yarl==1.9.2 zipp==3.16.2
Repro instructions
!/bin/bash
set -euo pipefail
mkdir -p model_store
Extra files add all files necessary for processor
torch-model-archiver --model-name speechtotext --version 1.0 --serialized-file model/pytorch_model.bin --handler ./handler.py --extra-files "model/config.json,model/special_tokens_map.json,model/tokenizer_config.json,model/vocab.json,model/preprocessor_config.json" -f mv speechtotext.mar model_store
Possible Solution
please give me any solution for this