towhee-io / examples

Analyze the unstructured data with Towhee, such as reverse image search, reverse video search, audio classification, question and answer systems, molecular search, etc.
Apache License 2.0
436 stars 111 forks source link

Memory leakage during image feature extraction(OOM) #258

Closed youwenwang2024 closed 2 months ago

youwenwang2024 commented 2 months ago
def pipline(img):
    p_search = (
        pipe.input('img')
        .map('img', 'vec', ops.image_embedding.timm('lambda_resnet50ts'))
        .output('vec')
    )
    res =  pipline(img).get()
    del p_search
    return res

if __name__ =="__main__":
    from glob import glob
    path = 'mypath'
    inputFiles =  glob(path+"/*.*")
    print(len(inputFiles))
    for idx in range(len(inputFiles)):
        input_file_path  = inputFiles[idx]
        pipline(input_file_path)

There are a large number of images in the folder. When calling the feature extraction interface in a loop, the memory gradually increases, ultimately leading to OOM. What is the reason for this

Environment