search

weaviate / t2v-transformers-models

This is the repo for the container that holds the models for the text2vec-transformers module
BSD 3-Clause "New" or "Revised" License
40 stars 27 forks source link

High (100%) CPU Utilization for Transformer Container. ThreadPoolExecutor not using max_workers defined. #81

Open pchunduru10 opened 7 months ago

pchunduru10 commented 7 months ago

Hi I am trying to run the transformers in kubernetes pod on my instance and its using up all the CPU cores available and throttling other pods in the cluster eventually crashing the sytem.

I tried to create a sed patch in in the dokcer-entrypoint file for the transformers to override ThreadPoolExecutor max workers which is set to. None (by default) to a value that user defines (with an ENV variable)

This doesnt seem to work as well, When I /bin/bash into the container the fix seems to work as below, but its not perofrming as expected and still using all the cores on the instance it was deployed.


class Vectorizer:
    executor: ThreadPoolExecutor

    def __init__(self, model_path: str, cuda_support: bool, cuda_core: str, cuda_per_process_memory_fraction: float,
                 model_type: str, architecture: str, direct_tokenize: bool, onnx_runtime: bool,
                 use_sentence_transformer_vectorizer: bool, model_name: str):
        self.executor = ThreadPoolExecutor(max_workers=2)

Appreciate any help regarding this issue.