yunlongyu / EPGN

26 stars 6 forks source link

Do you have validation set to select the best model? #3

Closed liubai01 closed 4 years ago

liubai01 commented 4 years ago

Are the reported results in "Episode-based Prototype Generating Network for Zero-Shot Learning"

liubai01 commented 4 years ago

BTW, I download your CUB feature, it sounds like there is a overlap of unseen test set and training set.

It may potentially be led by inconsistent labeling of matlab index(start from 1) and python index(strat from 0) from the original dataset.

>>> import scipy.io as sio
>>> data = sio.loadmat("data.mat")
>>> data.keys()
dict_keys(['__header__', '__version__', '__globals__', 'att_train', 'attribute', 'seen_pro', 'test_seen_fea', 'test_unseen_fea', 'train_fea', 'train_pro', 'unseen_pro'])
>>> label = sio.loadmat("label.mat")
>>> label.keys()
dict_keys(['__header__', '__version__', '__globals__', 'test_seen_idex', 'test_unseen_idex', 'train_idx', 'train_label_new'])
>>> np.unique(label['train_idx'])
array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,
        14,  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,
        27,  28,  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,
        40,  41,  42,  43,  44,  45,  46,  47,  48,  49,  50,  51,  52,
        53,  54,  55,  56,  57,  58,  59,  60,  61,  62,  63,  64,  65,
        66,  67,  68,  69,  70,  71,  72,  73,  74,  75,  76,  77,  78,
        79,  80,  81,  82,  83,  84,  85,  86,  87,  88,  89,  90,  91,
        92,  93,  94,  95,  96,  97,  98,  99, 100, 101, 102, 103, 104,
       105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117,
       118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130,
       131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143,
       144, 145, 146, 147, 148, 149, 150], dtype=uint8)
>>> np.unique(label['test_unseen_idex'])
array([  7,  19,  21,  29,  34,  36,  50,  56,  62,  68,  69,  72,  79,
        80,  87,  88,  91,  95,  98, 100, 104, 108, 116, 120, 122, 124,
       125, 129, 139, 141, 142, 150, 152, 157, 159, 160, 166, 167, 171,
       174, 176, 179, 182, 185, 187, 189, 191, 192, 193, 195], dtype=uint8)
>>> np.unique(label['test_seen_idex'])
array([  1,   2,   3,   4,   5,   6,   8,   9,  10,  11,  12,  13,  14,
        15,  16,  17,  18,  20,  22,  23,  24,  25,  26,  27,  28,  30,
        31,  32,  33,  35,  37,  38,  39,  40,  41,  42,  43,  44,  45,
        46,  47,  48,  49,  51,  52,  53,  54,  55,  57,  58,  59,  60,
        61,  63,  64,  65,  66,  67,  70,  71,  73,  74,  75,  76,  77,
        78,  81,  82,  83,  84,  85,  86,  89,  90,  92,  93,  94,  96,
        97,  99, 101, 102, 103, 105, 106, 107, 109, 110, 111, 112, 113,
       114, 115, 117, 118, 119, 121, 123, 126, 127, 128, 130, 131, 132,
       133, 134, 135, 136, 137, 138, 140, 143, 144, 145, 146, 147, 148,
       149, 151, 153, 154, 155, 156, 158, 161, 162, 163, 164, 165, 168,
       169, 170, 172, 173, 175, 177, 178, 180, 181, 183, 184, 186, 188,
       190, 194, 196, 197, 198, 199, 200], dtype=uint8)