Open snow-wind-001 opened 1 month ago
Why are the results different after two consecutive runs? It was right the first time. The second time I ran it by saving features_galleries, the result was ridiculously wrong. @churnikov @chang48 @dapladoc @alexmelekhin @leoromanovich
@snow-wind-001 Hi! Could you please reformat the code so it's easier to read?
I reformat the code@AlekseySh
Hey, @snow-wind-001
Once again. The problem is that ii_closest
has changed when you run the script 2nd time?
The script looks okay for me:
inference_on_images
Do you have the problem when you don't cache features? It's hard to debug from my place, but you need to find the exact moment when you have differences between two runs. Are models the same? Are datasets the same and not shuffled after reloading? Are features the same?
PS. You don't need to reimplement ViTExtractor to be able to load weights from the disk. You can just provide path to your weights as weights
parameter. In other words, weights
may be either special name of the weights storing in cloud OR path you your weights.
@AlekseySh I have another question. Is this code valid for is_query and is_gallery in the csv file during the training process?
Hey, @snow-wind-001
Once again. The problem is that
ii_closest
has changed when you run the script 2nd time?The script looks okay for me:
- there is no randomness in transforms
- model is set to eval mode inside
inference_on_images
Do you have the problem when you don't cache features? It's hard to debug from my place, but you need to find the exact moment when you have differences between two runs. Are models the same? Are datasets the same and not shuffled after reloading? Are features the same?
PS. You don't need to reimplement ViTExtractor to be able to load weights from the disk. You can just provide path to your weights as
weights
parameter. In other words,weights
may be either special name of the weights storing in cloud OR path you your weights.
import torch
import pandas as pd
import random
from pathlib import Path
from torch.utils.data import DataLoader
from typing import Union, Optional
from pprint import pprint
from oml.const import PATHS_COLUMN
from oml.datasets.base import DatasetQueryGallery
from oml.inference.flat import inference_on_dataframe, inference_on_images
from oml.inference.pairs import pairwise_inference_on_images
from oml.models import ConcatSiamese, ViTExtractor
from oml.registry.transforms import get_transforms_for_pretrained
from oml.retrieval.postprocessors.pairwise import PairwiseImagesPostprocessor
from oml.utils.misc_torch import pairwise_dist
from oml.utils.io import download_checkpoint_one_of
from NumpyImageDataset import *
from pathlib import Path
from PIL import Image # 正确导入 Image
import os
def print_dataset_paths(dataset):
for i in range(len(dataset)):
data = dataset[i]
print(f"Index {i}: Path {data['path']}")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
checkpoints_path = './best.ckpt'
model = ViTExtractor(weights=checkpoints_path, arch='vits16', normalise_features =True)
transform, _ = get_transforms_for_pretrained("vits16_dino")
from oml.transforms.images.torchvision import get_normalisation_resize_torch
transform = get_normalisation_resize_torch(im_size=224, mean=[0.40590053,0.40100682,0.35496407], std=[0.20830545,0.18484329,0.19726294])
#读取不同文件夹的图片,组成query和gallery
image_is_query_path = Path('/home/snowolf/dataset/bottle_test/is_query/1')
image_is_gallery_path = Path('/home/snowolf/dataset/bottle_test/is_gallery/1')
query_path = []
gallery_path = []
for img_file in image_is_query_path.glob('*.jpg'): # 假设图像文件是.jpg格式
query_path.append(img_file)
for img_file in image_is_gallery_path.glob('*.jpg'): # 假设图像文件是.jpg格式
gallery_path.append(img_file)
print("Image paths from folder 1:")
print(query_path)
print("\nImage paths from folder 2:")
print(gallery_path)
num_workers = 4
batch_size = 16
verbose = True
output_tensor = pairwise_inference_on_images(
model=model,
paths1=query_path,
paths2=gallery_path,
transform=transform,
num_workers=num_workers,
batch_size=batch_size,
verbose=verbose
)
print(output_tensor)
ii_closest = torch.argmin(output_tensor, dim=1)
print(ii_closest)
### I tried your method but reality lacks predict method.
Traceback (most recent call last):
File "/home/snowolf/git_code/open-metric-learning/pipelines/postprocessing/pairwise_postprocessing/test_v3.py", line 94, in
I try to use OML to perform image recognition on objects such as bottles. I input the test image and the standard image of a known category into OML, and ultimately hope to achieve good recognition results. I hope you can give us a demo to facilitate our implementation of this application. @AlekseySh
wait a second, why do you use pairwise_inference_on_images
? it's a special function for Siamese like models similar mostly used for postprocessing (re-ranking). It's a wrong function, the inference_on_images
was correct.
What do you mean by recognition results? What exactly do you do?
We have encountered such a need to identify beverage bottles. Originally there were 172 types of bottles, which may be expanded to more than 600 types in the future. Currently we use vit to implement a bottle classification model. Although the effect is good, the training cost for subsequent addition of new categories is very high, so we hope to use the OML project to implement this function. My plan is to set the views of all bottles in four directions as gallery, and through training, let the network judge the distance between the input image and the gallery image to achieve classification, and in the future, a small amount of training will be required to change the gallery at the same time. Data sets to achieve algorithm scalability. I saw this example of PyTorch Metric Learning with our Pipelines just follow the standard tutorial of adding custom loss. But there is no val code for sorting the input image output by correlation with the gallery image. I don’t know if my idea is correct, and can I provide an idea for calling the OML algorithm. @AlekseySh
@snow-wind-001 As far as I understood your task -- yes, it's a metric learning problem suitable for OML.
There is validation, you probably missed an example. Take a look at the examples section in readme or in the docs. (They are hidden under spoilers, so you need to click on them). You can use validation from there.
If you saw this validation but found it not convenient to analyze your outputs you can try reworked validation that will be published soon with OML 3.0. If you don' want to wait, just jump into docs
branch (you can use git clone and specify the branch) and run this example: https://github.com/OML-Team/open-metric-learning/tree/docs?tab=readme-ov-file#examples. With RetrievalResults
you will have more sense of the retrieval process during validation.
hey @snow-wind-001
We've just released OML 3.0 where we made work with retrieved items more transparent and simple. Take a look at the examples: https://github.com/OML-Team/open-metric-learning?tab=readme-ov-file#examples. I hope your problem will be automatically solved.