Could you pls release the search code for ImageNet

[iamweiweishi]

Hi, thank you for your great contribution. Could you pls release the search code for ImageNet? I can hardly reproduce the results reported in the paper. regards

[yuhuixu1993]

Hi, @iamweiweishi , thanks for your interest. We will release the code on imagenet recently and the core part of the code has already been released as model_search_imagenet.py. Because of the original main code is running in the company with many specialized dependencies packet. I need more time to clear the code for public while I will try my best.

[carolinadp]

Hello, @yuhuixu1993 I am also looking for the ImageNet search code. Could you please update us on this? Thank you

[yuhuixu1993]

@iamweiweishi ,@carolinadp , we have uploaded the ImageNet search code.

[iamweiweishi]

Thank you. But I found some typo in the 'train_search_imagenet.py' from line 121 to line 130. The current_lr is not defined.

Shoud I modify the code like this:

And I found that the 'architect.step' is not included in 'train_search_imagenet.py', I am not sure if this is correct? Best wishes.

[yuhuixu1993]

Hi, @iamweiweishi , thanks for pointing out this type, you are correct. I have update the code. If there were other types please let me know. Thanks again! You mentioned three typos in the attached picture and what is the third one (the third red circle)?

[iamweiweishi]

@yuhuixu1993 Hi, I also swap the order of parameters 'optimizer_a' and 'criterion', in the function 'train'. The code in Line 137 could work now. Thank you again.

[yuhuixu1993]

@iamweiweishi, I also updated the code. Thanks for your help!

[iamweiweishi]

Hi, there I have tried several times to reproduce the results, but never succeed. The acc seldom surpassed 0.10. I also noticed that the arch_learning_rate was never used. Could you pls post some advice on this? Thanks.

The acc goes like this, 12/31 07:54:28 AM train 000 6.906188e+00 0.195312 0.585938 12/31 07:59:24 AM train 050 6.910337e+00 0.091912 0.547641 12/31 08:04:19 AM train 100 6.910375e+00 0.098623 0.547262 12/31 08:09:14 AM train 150 6.910520e+00 0.094423 0.544547 12/31 08:14:08 AM train 200 6.910999e+00 0.100086 0.558730 12/31 08:18:59 AM train 250 6.911148e+00 0.102291 0.555182

[yuhuixu1993]

@iamweiweishi, could you please offer the full log file? Thanks.

[iamweiweishi]

The logs:

2019-12-30 20:37:15,826 args = Namespace(arch_learning_rate=0.006, arch_weight_decay=0.001, batch_size=512, begin=35, cutout=False, cutout_length=16, drop_path_prob=0.3, epochs=50, grad_clip=5, init_channels=16, layers=8, learning_rate=0.5, learning_rate_min=0.0, model_path='saved_models', momentum=0.9, note='try', report_freq=50, save='search-log-20191230-203714', seed=2, tmp_data_dir='/home/s00444365/datasets', unrolled=False, weight_decay=0.0003, workers=32) 2019-12-30 20:37:53,315 param size = 0.956512MB 2019-12-30 20:37:54,209 epoch 0 lr 5.000000e-01 2019-12-30 20:37:55,624 genotype = Genotype(normal=[('sep_conv_5x5', 1), ('sep_conv_3x3', 0), ('dil_conv_5x5', 0), ('max_pool_3x3', 1), ('dil_conv_5x5', 0), ('max_pool_3x3', 3), ('dil_conv_5x5', 4), ('max_pool_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 1), ('sep_conv_5x5', 0), ('max_pool_3x3', 0), ('avg_pool_3x3', 2), ('sep_conv_3x3', 2), ('avg_pool_3x3', 1), ('sep_conv_5x5', 2), ('max_pool_3x3', 1)], reduce_concat=range(2, 6)) 2019-12-30 21:04:07,088 train 000 6.915554e+00 0.000000 0.390625 2019-12-30 21:28:35,755 train 050 6.912632e+00 0.103401 0.497855 2019-12-30 21:33:25,617 train 100 6.912803e+00 0.110226 0.525990 2019-12-30 21:38:14,259 train 150 6.912623e+00 0.103477 0.529025 2019-12-30 21:43:03,691 train 200 6.912730e+00 0.107859 0.550956 2019-12-30 21:53:15,624 train 250 6.912362e+00 0.114004 0.562210 2019-12-30 21:53:16,850 train_acc 0.114004 2019-12-30 21:53:17,205 epoch 1 lr 4.995067e-01 2019-12-30 21:53:17,208 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('skip_connect', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 3), ('dil_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('avg_pool_3x3', 0), ('sep_conv_3x3', 1), ('avg_pool_3x3', 2), ('dil_conv_3x3', 0), ('dil_conv_5x5', 4), ('sep_conv_5x5', 0)], reduce_concat=range(2, 6)) 2019-12-30 22:15:07,180 train 000 6.912509e+00 0.000000 0.000000 2019-12-30 22:20:10,796 train 050 6.911884e+00 0.103401 0.566789 2019-12-30 22:25:18,731 train 100 6.911668e+00 0.104425 0.535659 2019-12-30 22:30:19,724 train 150 6.911179e+00 0.107357 0.561362 2019-12-30 22:35:14,149 train 200 6.911370e+00 0.110774 0.562617 2019-12-30 22:40:02,210 train 250 6.911471e+00 0.103853 0.555963 2019-12-30 22:40:03,586 train_acc 0.103853 2019-12-30 22:40:03,971 epoch 2 lr 4.980287e-01 2019-12-30 22:40:03,974 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('dil_conv_5x5', 1), ('sep_conv_3x3', 0), ('avg_pool_3x3', 2), ('dil_conv_5x5', 0), ('dil_conv_5x5', 4), ('max_pool_3x3', 0)], reduce_concat=range(2, 6)) 2019-12-30 22:40:42,512 train 000 6.910872e+00 0.390625 0.585938 2019-12-30 22:45:33,480 train 050 6.911595e+00 0.149357 0.643382 2019-12-30 22:50:29,409 train 100 6.911848e+00 0.117961 0.574335 2019-12-30 22:55:22,001 train 150 6.911471e+00 0.117705 0.576883 2019-12-30 23:00:12,741 train 200 6.911465e+00 0.117576 0.583022 2019-12-30 23:05:01,275 train 250 6.911376e+00 0.113223 0.591882 2019-12-30 23:05:02,763 train_acc 0.113223 2019-12-30 23:05:03,084 epoch 3 lr 4.955718e-01 2019-12-30 23:05:03,086 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('skip_connect', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 2), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('avg_pool_3x3', 0), ('dil_conv_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-30 23:05:48,591 train 000 6.917373e+00 0.390625 0.976562 2019-12-30 23:10:40,070 train 050 6.912191e+00 0.099571 0.551471 2019-12-30 23:15:33,585 train 100 6.911378e+00 0.123762 0.587871 2019-12-30 23:20:28,940 train 150 6.911189e+00 0.113825 0.579470 2019-12-30 23:25:21,135 train 200 6.911420e+00 0.103972 0.556786 2019-12-30 23:30:10,627 train 250 6.911249e+00 0.104634 0.555182 2019-12-30 23:30:12,118 train_acc 0.104634 2019-12-30 23:30:12,447 epoch 4 lr 4.921458e-01 2019-12-30 23:30:12,450 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_5x5', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('dil_conv_5x5', 0), ('avg_pool_3x3', 2), ('dil_conv_5x5', 4), ('sep_conv_5x5', 0)], reduce_concat=range(2, 6)) 2019-12-30 23:30:55,789 train 000 6.902474e+00 0.000000 0.976562 2019-12-30 23:35:48,190 train 050 6.910361e+00 0.088082 0.547641 2019-12-30 23:40:40,423 train 100 6.910512e+00 0.098623 0.537593 2019-12-30 23:45:35,107 train 150 6.910753e+00 0.104770 0.514797 2019-12-30 23:50:27,613 train 200 6.910940e+00 0.105916 0.511116 2019-12-30 23:55:16,571 train 250 6.910860e+00 0.106195 0.536442 2019-12-30 23:55:17,866 train_acc 0.106195 2019-12-30 23:55:18,187 epoch 5 lr 4.877641e-01 2019-12-30 23:55:18,189 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('skip_connect', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_5x5', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('sep_conv_3x3', 0), ('avg_pool_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-30 23:56:00,794 train 000 6.908577e+00 0.195312 0.195312 2019-12-31 00:00:51,679 train 050 6.910772e+00 0.045956 0.562960 2019-12-31 00:05:44,815 train 100 6.911373e+00 0.085087 0.605275 2019-12-31 00:10:36,682 train 150 6.910863e+00 0.087955 0.587231 2019-12-31 00:15:28,693 train 200 6.910745e+00 0.089397 0.604400 2019-12-31 00:20:19,079 train 250 6.910781e+00 0.090578 0.590321 2019-12-31 00:20:20,461 train_acc 0.090578 2019-12-31 00:20:20,772 epoch 6 lr 4.824441e-01 2019-12-31 00:20:20,774 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_5x5', 0), ('avg_pool_3x3', 0), ('avg_pool_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-31 00:20:57,925 train 000 6.904783e+00 0.195312 0.195312 2019-12-31 00:25:49,618 train 050 6.912709e+00 0.114890 0.536152 2019-12-31 00:30:43,836 train 100 6.911638e+00 0.116027 0.562732 2019-12-31 00:35:36,242 train 150 6.911309e+00 0.116411 0.578177 2019-12-31 00:40:26,974 train 200 6.911143e+00 0.103972 0.562617 2019-12-31 00:45:17,071 train 250 6.910941e+00 0.101510 0.559087 2019-12-31 00:45:18,610 train_acc 0.101510 2019-12-31 00:45:19,021 epoch 7 lr 4.762068e-01 2019-12-31 00:45:19,024 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0), ('max_pool_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('sep_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-31 00:46:00,966 train 000 6.904390e+00 0.195312 0.781250 2019-12-31 00:50:55,475 train 050 6.909521e+00 0.130208 0.562960 2019-12-31 00:55:49,513 train 100 6.910160e+00 0.135365 0.572401 2019-12-31 01:00:45,110 train 150 6.910941e+00 0.125466 0.565242 2019-12-31 01:05:40,730 train 200 6.911049e+00 0.117576 0.583994 2019-12-31 01:10:30,990 train 250 6.911116e+00 0.114004 0.580170 2019-12-31 01:10:32,439 train_acc 0.114004 2019-12-31 01:10:32,822 epoch 8 lr 4.690767e-01 2019-12-31 01:10:32,824 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 2), ('dil_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('avg_pool_3x3', 1), ('avg_pool_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-31 01:11:14,794 train 000 6.913342e+00 0.000000 0.390625 2019-12-31 01:16:08,157 train 050 6.911868e+00 0.111060 0.520833 2019-12-31 01:21:04,022 train 100 6.911707e+00 0.116027 0.572401 2019-12-31 01:25:58,755 train 150 6.911452e+00 0.112531 0.563949 2019-12-31 01:30:53,288 train 200 6.911111e+00 0.113689 0.555815 2019-12-31 01:35:41,603 train 250 6.910989e+00 0.115565 0.554402 2019-12-31 01:35:42,874 train_acc 0.115565 2019-12-31 01:35:43,264 epoch 9 lr 4.610820e-01 2019-12-31 01:35:43,266 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_5x5', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('sep_conv_3x3', 2), ('dil_conv_3x3', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-31 01:36:20,137 train 000 6.916134e+00 0.000000 0.390625 2019-12-31 01:41:19,080 train 050 6.910622e+00 0.061275 0.517004 2019-12-31 01:46:16,756 train 100 6.910502e+00 0.083153 0.524056 2019-12-31 01:51:17,340 train 150 6.910645e+00 0.089249 0.519971 2019-12-31 01:56:13,444 train 200 6.910736e+00 0.091340 0.521805 2019-12-31 02:01:01,445 train 250 6.911053e+00 0.095263 0.538004 2019-12-31 02:01:02,883 train_acc 0.095263 2019-12-31 02:01:03,283 epoch 10 lr 4.522542e-01 2019-12-31 02:01:03,285 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('skip_connect', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('sep_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('sep_conv_5x5', 2), ('dil_conv_3x3', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-31 02:01:39,403 train 000 6.900641e+00 0.195312 0.976562 2019-12-31 02:06:38,031 train 050 6.908765e+00 0.111060 0.555300 2019-12-31 02:11:31,900 train 100 6.910181e+00 0.108292 0.529858 2019-12-31 02:16:27,253 train 150 6.910482e+00 0.093129 0.551014 2019-12-31 02:21:19,424 train 200 6.910984e+00 0.097170 0.541239 2019-12-31 02:26:09,641 train 250 6.910902e+00 0.092140 0.534100 2019-12-31 02:26:11,150 train_acc 0.092140 2019-12-31 02:26:11,467 epoch 11 lr 4.426283e-01 2019-12-31 02:26:11,469 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('skip_connect', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_5x5', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-31 02:26:48,175 train 000 6.903584e+00 0.000000 0.390625 2019-12-31 02:31:43,546 train 050 6.910579e+00 0.130208 0.585938 2019-12-31 02:36:39,948 train 100 6.911383e+00 0.112160 0.578202 2019-12-31 02:41:35,999 train 150 6.911111e+00 0.102183 0.570416 2019-12-31 02:46:29,763 train 200 6.911105e+00 0.104944 0.572334 2019-12-31 02:51:18,680 train 250 6.911020e+00 0.109319 0.561429 2019-12-31 02:51:19,927 train_acc 0.109319 2019-12-31 02:51:20,247 epoch 12 lr 4.322422e-01 2019-12-31 02:51:20,249 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('avg_pool_3x3', 1), ('sep_conv_5x5', 2), ('dil_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('sep_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('avg_pool_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-31 02:52:04,109 train 000 6.916040e+00 0.000000 0.585938 2019-12-31 02:57:01,260 train 050 6.911184e+00 0.134038 0.635723 2019-12-31 03:01:56,154 train 100 6.910871e+00 0.106358 0.607209 2019-12-31 03:06:51,014 train 150 6.911156e+00 0.104770 0.591111 2019-12-31 03:11:44,898 train 200 6.911006e+00 0.113689 0.592739 2019-12-31 03:16:34,552 train 250 6.911048e+00 0.114004 0.577046 2019-12-31 03:16:35,996 train_acc 0.114004 2019-12-31 03:16:36,411 epoch 13 lr 4.211368e-01 2019-12-31 03:16:36,413 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_5x5', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('max_pool_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 0), ('avg_pool_3x3', 1), ('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_5x5', 2), ('sep_conv_5x5', 0), ('avg_pool_3x3', 4)], reduce_concat=range(2, 6)) 2019-12-31 03:17:19,532 train 000 6.916420e+00 0.000000 0.195312 2019-12-31 03:22:13,422 train 050 6.910109e+00 0.099571 0.486366 2019-12-31 03:27:08,115 train 100 6.910328e+00 0.088954 0.506652 2019-12-31 03:32:04,616 train 150 6.910851e+00 0.106064 0.518678 2019-12-31 03:37:00,820 train 200 6.911063e+00 0.112718 0.531522 2019-12-31 03:41:50,908 train 250 6.911088e+00 0.105414 0.524729 2019-12-31 03:41:52,458 train_acc 0.105414 2019-12-31 03:41:52,776 epoch 14 lr 4.093560e-01 2019-12-31 03:41:52,779 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0), ('avg_pool_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-31 03:42:33,819 train 000 6.912027e+00 0.000000 0.585938 2019-12-31 03:47:28,531 train 050 6.911236e+00 0.137868 0.597426 2019-12-31 03:52:21,902 train 100 6.911177e+00 0.100557 0.562732 2019-12-31 03:57:14,868 train 150 6.911390e+00 0.115118 0.566536 2019-12-31 04:02:10,702 train 200 6.911063e+00 0.111746 0.564560 2019-12-31 04:06:59,930 train 250 6.911139e+00 0.105414 0.554402 2019-12-31 04:07:01,282 train_acc 0.105414 2019-12-31 04:07:01,642 epoch 15 lr 3.969463e-01 2019-12-31 04:07:01,644 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('skip_connect', 2), ('dil_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_5x5', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 0), ('avg_pool_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-31 04:07:41,033 train 000 6.907535e+00 0.390625 0.390625 2019-12-31 04:12:37,931 train 050 6.911418e+00 0.118719 0.589767 2019-12-31 04:17:33,264 train 100 6.910759e+00 0.092822 0.568533 2019-12-31 04:22:26,657 train 150 6.911244e+00 0.098303 0.556188 2019-12-31 04:27:22,261 train 200 6.911272e+00 0.089397 0.550956 2019-12-31 04:32:11,483 train 250 6.911101e+00 0.095263 0.561429 2019-12-31 04:32:13,004 train_acc 0.095263 2019-12-31 04:32:13,312 epoch 16 lr 3.839567e-01 2019-12-31 04:32:13,314 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_5x5', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('avg_pool_3x3', 0), ('avg_pool_3x3', 2), ('dil_conv_5x5', 4), ('sep_conv_5x5', 3)], reduce_concat=range(2, 6)) 2019-12-31 04:32:50,973 train 000 6.910200e+00 0.000000 0.390625 2019-12-31 04:37:45,113 train 050 6.911125e+00 0.118719 0.585938 2019-12-31 04:42:39,815 train 100 6.911381e+00 0.110226 0.537593 2019-12-31 04:47:35,330 train 150 6.911345e+00 0.112531 0.540666 2019-12-31 04:52:29,175 train 200 6.911152e+00 0.115633 0.535409 2019-12-31 04:57:17,484 train 250 6.911033e+00 0.115565 0.548155 2019-12-31 04:57:18,993 train_acc 0.115565 2019-12-31 04:57:19,319 epoch 17 lr 3.704384e-01 2019-12-31 04:57:19,321 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_5x5', 2), ('dil_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_5x5', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 0), ('avg_pool_3x3', 2), ('dil_conv_5x5', 4), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6)) 2019-12-31 04:58:04,018 train 000 6.912227e+00 0.000000 0.390625 2019-12-31 05:03:01,127 train 050 6.911225e+00 0.103401 0.628064 2019-12-31 05:07:53,678 train 100 6.911846e+00 0.092822 0.554997 2019-12-31 05:12:46,673 train 150 6.911492e+00 0.087955 0.525145 2019-12-31 05:17:38,750 train 200 6.911056e+00 0.088425 0.525692 2019-12-31 05:22:30,302 train 250 6.910874e+00 0.095263 0.527072 2019-12-31 05:22:31,557 train_acc 0.095263 2019-12-31 05:22:31,867 epoch 18 lr 3.564448e-01 2019-12-31 05:22:31,869 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_5x5', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('sep_conv_3x3', 1), ('avg_pool_3x3', 0), ('avg_pool_3x3', 0), ('avg_pool_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-31 05:23:16,157 train 000 6.913686e+00 0.585938 0.781250 2019-12-31 05:28:09,088 train 050 6.910643e+00 0.114890 0.555300 2019-12-31 05:33:04,475 train 100 6.910024e+00 0.094756 0.524056 2019-12-31 05:37:59,845 train 150 6.910359e+00 0.109944 0.548427 2019-12-31 05:42:53,748 train 200 6.910761e+00 0.106887 0.563588 2019-12-31 05:47:44,195 train 250 6.910785e+00 0.108538 0.556744 2019-12-31 05:47:45,677 train_acc 0.108538 2019-12-31 05:47:45,986 epoch 19 lr 3.420311e-01 2019-12-31 05:47:45,988 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('skip_connect', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 0), ('sep_conv_5x5', 2), ('sep_conv_3x3', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-31 05:48:26,255 train 000 6.913815e+00 0.000000 0.390625 2019-12-31 05:53:24,603 train 050 6.910912e+00 0.080423 0.532322 2019-12-31 05:58:18,362 train 100 6.910938e+00 0.114093 0.576269 2019-12-31 06:03:15,480 train 150 6.911035e+00 0.117705 0.582057 2019-12-31 06:08:09,036 train 200 6.910852e+00 0.118548 0.592739 2019-12-31 06:12:59,343 train 250 6.910917e+00 0.114004 0.578608 2019-12-31 06:13:00,873 train_acc 0.114004 2019-12-31 06:13:01,195 epoch 20 lr 3.272542e-01 2019-12-31 06:13:01,197 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_5x5', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 1), ('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('avg_pool_3x3', 0), ('avg_pool_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-31 06:13:39,429 train 000 6.912564e+00 0.195312 0.976562 2019-12-31 06:18:34,403 train 050 6.909996e+00 0.118719 0.505515 2019-12-31 06:23:28,270 train 100 6.910769e+00 0.102491 0.531791 2019-12-31 06:28:24,820 train 150 6.910868e+00 0.099596 0.548427 2019-12-31 06:33:20,010 train 200 6.911206e+00 0.097170 0.553871 2019-12-31 06:38:11,998 train 250 6.911031e+00 0.094483 0.544251 2019-12-31 06:38:13,448 train_acc 0.094483 2019-12-31 06:38:13,941 epoch 21 lr 3.121725e-01 2019-12-31 06:38:13,944 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('skip_connect', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_5x5', 2), ('skip_connect', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-31 06:38:53,487 train 000 6.916465e+00 0.000000 0.585938 2019-12-31 06:43:50,193 train 050 6.911452e+00 0.130208 0.559130 2019-12-31 06:48:43,430 train 100 6.910875e+00 0.121829 0.566600 2019-12-31 06:53:39,160 train 150 6.911049e+00 0.117705 0.560068 2019-12-31 06:58:32,210 train 200 6.911103e+00 0.111746 0.568447 2019-12-31 07:03:19,900 train 250 6.910911e+00 0.103853 0.570799 2019-12-31 07:03:21,413 train_acc 0.103853 2019-12-31 07:03:21,742 epoch 22 lr 2.968453e-01 2019-12-31 07:03:21,744 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('avg_pool_3x3', 2), ('sep_conv_3x3', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-31 07:03:58,330 train 000 6.918197e+00 0.000000 0.390625 2019-12-31 07:08:50,771 train 050 6.910912e+00 0.095741 0.570619 2019-12-31 07:13:46,871 train 100 6.911359e+00 0.090888 0.541460 2019-12-31 07:18:40,065 train 150 6.910955e+00 0.099596 0.578177 2019-12-31 07:23:32,419 train 200 6.911001e+00 0.100086 0.571362 2019-12-31 07:28:21,888 train 250 6.911139e+00 0.098387 0.560648 2019-12-31 07:28:23,402 train_acc 0.098387 2019-12-31 07:28:23,711 epoch 23 lr 2.813333e-01 2019-12-31 07:28:23,713 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_5x5', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 0), ('avg_pool_3x3', 2), ('sep_conv_3x3', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6)) 2019-12-31 07:29:07,046 train 000 6.917174e+00 0.000000 0.585938 2019-12-31 07:34:05,083 train 050 6.910776e+00 0.076593 0.536152 2019-12-31 07:39:01,895 train 100 6.911441e+00 0.083153 0.533725 2019-12-31 07:43:57,263 train 150 6.911437e+00 0.086662 0.540666 2019-12-31 07:48:53,471 train 200 6.911353e+00 0.088425 0.551928 2019-12-31 07:53:43,070 train 250 6.911177e+00 0.089797 0.552059 2019-12-31 07:53:44,538 train_acc 0.089797 2019-12-31 07:53:44,851 epoch 24 lr 2.656976e-01

Thank you. @yuhuixu1993

[yuhuixu1993]

@iamweiweishi ，hi，first you need to check the warm up code （the lr in the first five epochs）The log shows that you did not warm up https://github.com/yuhuixu1993/PC-DARTS/blob/0307f6e91c7bd58e16318377e95f77a811be9ae5/train_search_imagenet.py#L125

Second，the archtect learning rate didnot work until the 35th epoch

https://github.com/yuhuixu1993/PC-DARTS/blob/0307f6e91c7bd58e16318377e95f77a811be9ae5/train_search_imagenet.py#L168

Third， you need to check that the data is ramdomly sampled in each class of the1000 classes

[iamweiweishi]

I have also tried the warmup training. The results remained the same. The data are randomly sampled as you metioned.

I rerun the code again, it seems work well now although I still did not find what the problem is. Many thanks. ^^

[yuhuixu1993]

@iamweiweishi ，I will try it again myself，does the archtecture learning rate work after 35th epoch？

[yuhuixu1993]

@iamweiweishi , there indeed some problems in the the previous train_search_imagenet.py. I have update the code I have tested ok.