search

Chenxingyu1990 / A-Boundary-Based-Out-of-Distribution-Classifier-for-Generalized-Zero-Shot-Learning

MIT License
30 stars 3 forks source link

Standard split of benchmarks #7

Closed Bang2hen1iu closed 1 year ago

Bang2hen1iu commented 1 year ago

@Chenxingyu1990 Sorry for bordering... I've already read #1, trying to reproduce the results by myself but got the same result as WilliamYi96. Looks like the provided classifier weights of F-CLSWGAN are trained based on the Standard Split (SS), which means that I run the code with Proposed Split (PS) data and SS classifier. But I didn't find out proper split files in link2. Would you please also upload the corresponding splits file too? I will be grateful if you do so!

Bang2hen1iu commented 1 year ago

OK！ Finally, I completely understand the so-called previous version means the original Proposed Split (PS), not the SS. Following this would help reproduce the result.

I think I've found the true reason. The reproducing issue is caused by the Datasets!!! Link 1 has two kinds of dataset splits. The first one called split-V2 is different from the previous one. Currently, most previous GZSL algorithms use the previous split (Link 2), including f-CLSGWAN, CADA-VAE, COSMO, et al., and ours.

Link 1: https://www.mpi-inf.mpg.de/departments/computer-vision-and-machine-learning/research/zero-shot-learning/zero-shot-learning-the-good-the-bad-and-the-ugly

Link 2: The previous dataset splits link: wget http://chechiklab.biu.ac.il/~yuvval/COSMO/data.zip

@ZiyangW2000 's results have no problem on split-V2. I also get similar results.

The following link describes the details. https://drive.google.com/file/d/1p9gtkuHCCCyjkyezSarCw-1siCSXUykH/view

Interestingly, I visit some previous git repos. I find the reproducing issues also may be caused by the dataset problem.

https://github.com/akku1506/Feature-Generating-Networks-for-ZSL/issues stevehuanghe/GDAN#3