Low performance

[mellinging]

Hello, I got a problem(low performance micro=0.14 macro=0.04) when running the SDNE code on blogCatalog Dataset . Set the layer as N-1000-100, alpha =0.2 ,beta =10, reg =1. Could you show your performance on this dataset? Thank you very much .

[suanrong]

Sorry, guy. There is a bug in checking-classification. I have fixed it. I achieve the performance at micro = 0.27 and macro = 0.13. I recommend to set bigger alpha.

[mellinging]

Thank you,When I run your new code on blogcatalog, i did not see any improvement after one epoch. Anyway, "micro = 0.27 and macro = 0.13 “ is lower than paper’s “micro=0.31 macro = 0.19 ”. Any problem in tuning parameter ?

On 14 May 2018, at 9:42 AM, xuanrong yao notifications@github.com wrote:

Sorry, guy. There is a bug in checking-classification. I have fixed it. I achieve the performance at micro = 0.27 and macro = 0.13. I recommend to set bigger alpha.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/suanrong/SDNE/issues/9#issuecomment-388674037, or mute the thread https://github.com/notifications/unsubscribe-auth/AVWV3PHFzMRdWN4sOwDwVyNdLFeiaPDtks5tyODpgaJpZM4T8cKA.

[mellinging]

And as turn down the test_ratio of check multi_lable, the micro and macro does not change normally. it seems like wrong ?

Thanks for your reply .

Epoch : 2 loss : 1690985235.750 2 classification micro_f1: 0.2491 macro_f1 : 0.1083 test_ratio = 0.9 2 classification micro_f1: 0.2693 macro_f1 : 0.1214 test_ratio = 0.8 2 classification micro_f1: 0.2630 macro_f1 : 0.1153 test_ratio = 0.7 2 classification micro_f1: 0.2751 macro_f1 : 0.1205 test_ratio = 0.6 2 classification micro_f1: 0.2783 macro_f1 : 0.1248 test_ratio = 0.5 2 classification micro_f1: 0.2766 macro_f1 : 0.1213 test_ratio = 0.4 2 classification micro_f1: 0.2841 macro_f1 : 0.1300 test_ratio = 0.3 2 classification micro_f1: 0.2824 macro_f1 : 0.1171 test_ratio = 0.2 2 classification micro_f1: 0.2749 macro_f1 : 0.1194 test_ratio = 0.1 Epoch : 4 loss : 1716842071.312 4 classification micro_f1: 0.2457 macro_f1 : 0.1133 4 classification micro_f1: 0.2586 macro_f1 : 0.1161 4 classification micro_f1: 0.2613 macro_f1 : 0.1166 4 classification micro_f1: 0.2672 macro_f1 : 0.1193 4 classification micro_f1: 0.2726 macro_f1 : 0.1216 4 classification micro_f1: 0.2826 macro_f1 : 0.1239 4 classification micro_f1: 0.2713 macro_f1 : 0.1193 4 classification micro_f1: 0.2789 macro_f1 : 0.1264 4 classification micro_f1: 0.2670 macro_f1 : 0.1132 Epoch : 6 loss : 1691031755.438 6 classification micro_f1: 0.2400 macro_f1 : 0.1074 6 classification micro_f1: 0.2550 macro_f1 : 0.1151 6 classification micro_f1: 0.2591 macro_f1 : 0.1106 6 classification micro_f1: 0.2609 macro_f1 : 0.1113 6 classification micro_f1: 0.2641 macro_f1 : 0.1160 6 classification micro_f1: 0.2682 macro_f1 : 0.1106 6 classification micro_f1: 0.2608 macro_f1 : 0.1152 6 classification micro_f1: 0.2797 macro_f1 : 0.1254 6 classification micro_f1: 0.2810 macro_f1 : 0.1137 Epoch : 8 loss : 1667297124.625 8 classification micro_f1: 0.2419 macro_f1 : 0.1066 8 classification micro_f1: 0.2579 macro_f1 : 0.1125 8 classification micro_f1: 0.2616 macro_f1 : 0.1119 8 classification micro_f1: 0.2643 macro_f1 : 0.1169 8 classification micro_f1: 0.2744 macro_f1 : 0.1170 8 classification micro_f1: 0.2690 macro_f1 : 0.1155 8 classification micro_f1: 0.2791 macro_f1 : 0.1179 8 classification micro_f1: 0.2717 macro_f1 : 0.1183 8 classification micro_f1: 0.2738 macro_f1 : 0.1135 Epoch : 10 loss : 1654633685.125 10 classification micro_f1: 0.2402 macro_f1 : 0.1066 10 classification micro_f1: 0.2523 macro_f1 : 0.1101 10 classification micro_f1: 0.2621 macro_f1 : 0.1151 10 classification micro_f1: 0.2667 macro_f1 : 0.1166 10 classification micro_f1: 0.2606 macro_f1 : 0.1139 10 classification micro_f1: 0.2701 macro_f1 : 0.1160 10 classification micro_f1: 0.2684 macro_f1 : 0.1184 10 classification micro_f1: 0.2665 macro_f1 : 0.1138 10 classification micro_f1: 0.2736 macro_f1 : 0.1089 Epoch : 12 loss : 1654192828.125 12 classification micro_f1: 0.2461 macro_f1 : 0.1030 12 classification micro_f1: 0.2558 macro_f1 : 0.1056 12 classification micro_f1: 0.2620 macro_f1 : 0.1137 12 classification micro_f1: 0.2734 macro_f1 : 0.1166 12 classification micro_f1: 0.2676 macro_f1 : 0.1160 12 classification micro_f1: 0.2678 macro_f1 : 0.1183 12 classification micro_f1: 0.2736 macro_f1 : 0.1152 12 classification micro_f1: 0.2666 macro_f1 : 0.1136 12 classification micro_f1: 0.2652 macro_f1 : 0.1258 Epoch : 14 loss : 1666404121.812 14 classification micro_f1: 0.2442 macro_f1 : 0.1004 14 classification micro_f1: 0.2529 macro_f1 : 0.1092 14 classification micro_f1: 0.2605 macro_f1 : 0.1092 14 classification micro_f1: 0.2699 macro_f1 : 0.1165 14 classification micro_f1: 0.2616 macro_f1 : 0.1151 14 classification micro_f1: 0.2745 macro_f1 : 0.1215 14 classification micro_f1: 0.2753 macro_f1 : 0.1214 14 classification micro_f1: 0.2640 macro_f1 : 0.1142 14 classification micro_f1: 0.2670 macro_f1 : 0.1085 Epoch : 16 loss : 1660764040.375 16 classification micro_f1: 0.2421 macro_f1 : 0.1061 16 classification micro_f1: 0.2615 macro_f1 : 0.1163 16 classification micro_f1: 0.2603 macro_f1 : 0.1140 16 classification micro_f1: 0.2634 macro_f1 : 0.1148 16 classification micro_f1: 0.2731 macro_f1 : 0.1153 16 classification micro_f1: 0.2730 macro_f1 : 0.1206 16 classification micro_f1: 0.2708 macro_f1 : 0.1155 16 classification micro_f1: 0.2753 macro_f1 : 0.1241 16 classification micro_f1: 0.2926 macro_f1 : 0.1262

On 14 May 2018, at 9:42 AM, xuanrong yao notifications@github.com wrote:

Closed #9 https://github.com/suanrong/SDNE/issues/9.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/suanrong/SDNE/issues/9#event-1623802811, or mute the thread https://github.com/notifications/unsubscribe-auth/AVWV3GUeImjWgZfNc8NcJLfwe6bOq6uIks5tyOD_gaJpZM4T8cKA.