Not getting good results

[imvansh25]

Greetings, I am attempting to involve your proposed model in my undertaking. I have utilized the accompanying cfg to prepare my model.

from vos.detection.default_trainer_gmm import DefaultTrainer from detectron2.config import CfgNode, get_cfg from vos.detection.core.setup import add_probabilistic_config

cfg = get_cfg()#CfgNode() add_probabilistic_config(cfg)

cfg_file_name = "vos/detection/configs/VOC-Detection/faster-rcnn/vos.yaml" cfg.merge_from_file(cfg_file_name) cfg.SOLVER.BASE_LR = 0.0001 cfg.SOLVER.IMS_PER_BATCH=6 cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 64 cfg.MODEL.ROI_HEADS.POSITIVE_FRACTION = 0.25 cfg.MODEL.RPN.BATCH_SIZE_PER_IMAGE = 128 cfg.SOLVER.MAX_ITER = 20000 cfg.SOLVER.STEPS = (8000,12000) cfg.VOS.STARTING_ITER = 10000 cfg.SOLVER.CHECKPOINT_PERIOD = 8000 cfg.MODEL.ROI_HEADS.NUM_CLASSES = len(MetadataCatalog.get('my_data').thing_classes)

Some way or another, I am not obtain great outcomes. Would you be able to diagram why it isn't functioning admirably

Dataset info

Training Graphs:

Please let me know if you need more info, to help me.

[d12306]

hi, @imvansh25 , thanks for trying vos out! It seems like you using a different dataset for training. Unfortunately, we can not provide any practical advice in model tunning other than the ones in the paper. If you are having trouble with the OOD detection, please consider tunning the mentioned hyperparameters in the paper for sampling and virtual outlier synthesis.

[imvansh25]

Hi @d12306 , I have a few inquiries in regards to VOS

1. For what reason does it gives higher Probability when contrasted with Normal Detectron?
2. Why learning rate has been kept so high(0.02)?
3. How can we optimize energy threshold value?

[d12306]

hi, @imvansh25 ,

1. vos uses synthesize virtual outliers to regularize the model using the designed generalized energy score, which is defintely better than just training on the id samples in normal detectron2.
2. we follow the default setting of training a faster-rcnn in detectron2.
3. not sure which threshold value you mean, normally we determine the threshold when the true positive rate of ID samples is at 95% when calculating the fpr95.

[imvansh25]

hi @d12306 , Thank you for responding. Vos appears to be intriguing to me. I was looking for a way to create a UMAP visualization of Custom Dataset feature embeddings (Fig3 in paper). I can't find a script for this in the repo. Could you please include that as well?

[d12306]

visualizing the feature embeddings is not quite related to the main code, I will post it here. @imvansh25

[d12306]

import umap
from sklearn.datasets import fetch_openml
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np

sns.set(context="paper", style="white")

data = np.load('data.npy')#.item()

data_dict = {}
for sub in data:
    label = int(sub[-1])
    if label in list(data_dict.keys()):
        data_dict[label] = np.concatenate((data_dict[label], sub[:-1].reshape(-1, 256)), 0)
    else:
        data_dict[label] = sub[:-1].reshape(-1,256)
# breakpoint()
data = data_dict

for i in range(20):
    if i == 0:
        data_preprocess = data[i]
    else:
        data_preprocess = np.concatenate((data_preprocess, data[i]), 0)

data_preprocess = np.array(data_preprocess).reshape(-1, 256)
targets = []
for i in range(20):
    targets.append(i * np.ones(len(data_dict[i])))
targets = np.array(targets).reshape(-1)

reducer = umap.UMAP(random_state=42,n_neighbors=15, min_dist=0.2, n_components=2, metric='euclidean')
embedding = reducer.fit_transform(data_preprocess)

fig, ax = plt.subplots(figsize=(12, 12))
# color = mnist.target.astype(int)
def get_cmap(n, name='hsv'):
    '''Returns a function that maps each index in 0, 1, ..., n-1 to a distinct
    RGB color; the keyword argument name must be a standard mpl colormap name.'''
    return plt.cm.get_cmap(name, n)

classes = [str(hhh) for hhh in range(20)]
# color = targets.astype(int)#[index for index in range(20)]#
color = get_cmap(20)
# color = plt.cm.coolwarm(np.linspace(0.1,0.9,11))
selected = np.random.choice(20, 10, replace=False)
index = 0
sum = 0
for i in range(0, 20):
    plt.scatter(embedding[:, 0][sum: sum + len(data_dict[i])],
                embedding[:, 1][sum: sum + len(data_dict[i])],
                c=color(i),
                label=index, cmap="Spectral", s=1)
    sum += len(data_dict[i])
    index += 1

plt.legend(fontsize=20)
ax.legend(loc='lower left',markerscale=9)#, bbox_to_anchor=(1, 0.5)

plt.setp(ax, xticks=[], yticks=[])

plt.savefig('./fig.jpg', dpi=250)
# plt.show()