search

EGiunchiglia / C-HMCNN

Code for paper: "Coherent Hierarchical Multi-Label Classification Networks"
GNU General Public License v3.0
85 stars 21 forks source link

ran on cellcycle_FUN and didn't get the result in paper #1

Closed luome closed 3 years ago

luome commented 3 years ago

Hi,

I ran the code on cellcycle_FUN with seed 0 and get an AUPRC of 0.17 max, which was much smaller than the value in your paper. I tried the code in python 3 and pytorch 1.6 and the code worked just fine, although there's some deprecated warnings. I am wandering why there is such a difference? Maybe the version difference ?

EGiunchiglia commented 3 years ago

Hi,

Thank you for your interest in our work!

In order to faithfully replicate the results you should first create an environment using the yml file in the following way:

conda env create -f c-hmcnn_env.yml

and then activate it in the following way:

conda activate py27.

As a double check, I tried to run the code again by creating the environment as described above (from scratch) and I got the following AUPRC for cellcycle_FUN:

Seed AUPRC
9 0.25546415850363147
7 0.25515710548385745
3 0.25472701099135214
4 0.2554883421712122
6 0.25533237842729817
8 0.2555740225767902
2 0.25435716879340153
5 0.25516504368163523
1 0.25511300954822325
0 0.2575526651299051

Note that in this way the code does not produce any warning.

Just out of curiosity, could you please share the yml of the environment you are using right now so that I can check what is going wrong?

luome commented 3 years ago

I tried to use conda create before but it would cause error because of the network issue in China, so I used my own env. I will try using proxy to create the complete conda environment.

There is a lot redundant packages so I list some packages which may affect the result. Thanks for replying me.

networkx 2.5 pypi_0 pypi numpy 1.19.2 py38h54aff64_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main numpy-base 1.19.2 py38hfa32c7d_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main pytorch 1.7.0 py3.8_cuda11.0.221_cudnn8.0.3_0 pytorch tensorflow 2.4.0 keras 2.4.3 pypi_0 pypi keras-preprocessing 1.1.2 pypi_0 pypi scikit-learn 0.23.2 py38h0573a6f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main

luome commented 3 years ago

Hi,

I used the complete conda environment and it yielded the result of 0.257