Closed pribadihcr closed 6 months ago
Hi, I got out of memory/memory leak after after some calls. Here the script
model = AutoModel.from_pretrained('openbmb/MiniCPM-V', trust_remote_code=True, cache_dir="./cache", torch_dtype=torch.bfloat16) model = model.to(device='cuda', dtype=torch.bfloat16) tokenizer = AutoTokenizer.from_pretrained('openbmb/MiniCPM-V', trust_remote_code=True, cache_dir="./cache") files = ['a.jpg', 'b.jpg', 'c.jpg', ...] for image_path in files: msgs = [{'role': 'user', 'content': prompt}] image = Image.open(image_path).convert('RGB') res, context, _ = model.chat( image=image, msgs=msgs, context=None, tokenizer=tokenizer, sampling=False, temperature=0.0 ) print(res)
look like it depend on the prompt
Hi, I got out of memory/memory leak after after some calls. Here the script