search

232525 / PureT

Implementation of 'End-to-End Transformer Based Model for Image Captioning' [AAAI 2022]
63 stars 12 forks source link

about backbone optimization #3

Closed wanboyang closed 2 years ago

wanboyang commented 2 years ago

Thanks for this amazing work. It seems the swin backbone is not being optimized in this work. I have a question about whether the model can perform better performance when the swin backbone is being trained end-to-end. Thanks again.

232525 commented 2 years ago

I am not sure because I did not do the experiment. It is easy to do this by modifying the code in https://github.com/232525/PureT/blob/main/models/pure_transformer.py#L50, maybe you can try it. But optimizing the backbone when training is time-consuming, so I suggest you only optimize the last few layers. Maybe it's enough to optimize only in SCST.

wanboyang commented 2 years ago

I am not sure because I did not do the experiment. It is easy to do this by modifying the code in https://github.com/232525/PureT/blob/main/models/pure_transformer.py#L50, maybe you can try it. But optimizing the backbone when training is time-consuming, so I suggest you only optimize the last few layers. Maybe it's enough to optimize only in SCST.

Thanks for replying. I am try to reproduce the performances of this work. the performance gained in XE training is shown as follows: [INFO: 2022-03-25 16:19:03,185] ######## Epoch (VAL)1 ######## [INFO: 2022-03-25 16:19:03,186] {'Bleu_1': 0.7298197334646436, 'Bleu_2': 0.5598084754130698, 'Bleu_3': 0.41686122052098756, 'Bleu_4': 0.3075288006376614, 'METEOR': 0.25183284357565483, 'ROUGE_L': 0.5364082632968674, 'CIDEr': 0.9874354841323122} [INFO: 2022-03-25 16:19:33,785] ######## Epoch (TEST)1 ######## [INFO: 2022-03-25 16:19:33,785] {'Bleu_1': 0.7278049640858596, 'Bleu_2': 0.5581841376501904, 'Bleu_3': 0.4165854529272677, 'Bleu_4': 0.3083541528657101, 'METEOR': 0.2530049140905188, 'ROUGE_L': 0.5378770300973704, 'CIDEr': 1.0009070560059365} [INFO: 2022-03-25 16:31:04,047] ######## Epoch (VAL)2 ######## [INFO: 2022-03-25 16:31:04,047] {'Bleu_1': 0.7651822697301816, 'Bleu_2': 0.60445187297167, 'Bleu_3': 0.46328362266392714, 'Bleu_4': 0.352366236098675, 'METEOR': 0.2736763486304883, 'ROUGE_L': 0.5659319351115341, 'CIDEr': 1.1266566854850768} [INFO: 2022-03-25 16:31:34,681] ######## Epoch (TEST)2 ######## [INFO: 2022-03-25 16:31:34,681] {'Bleu_1': 0.7615620906716226, 'Bleu_2': 0.5994168665866058, 'Bleu_3': 0.4575503550686494, 'Bleu_4': 0.34635391820802724, 'METEOR': 0.27140881783705706, 'ROUGE_L': 0.5618186369837561, 'CIDEr': 1.1302743082763234} [INFO: 2022-03-25 16:43:00,187] ######## Epoch (VAL)3 ######## [INFO: 2022-03-25 16:43:00,187] {'Bleu_1': 0.7638724817425642, 'Bleu_2': 0.6048651619059141, 'Bleu_3': 0.4676425103287978, 'Bleu_4': 0.3609007019938609, 'METEOR': 0.2790395228025375, 'ROUGE_L': 0.5661808603408964, 'CIDEr': 1.1455569387903557} [INFO: 2022-03-25 16:43:32,537] ######## Epoch (TEST)3 ######## [INFO: 2022-03-25 16:43:32,537] {'Bleu_1': 0.7611494497226113, 'Bleu_2': 0.6008574813778096, 'Bleu_3': 0.4641215640871456, 'Bleu_4': 0.3575734550936475, 'METEOR': 0.279229081780546, 'ROUGE_L': 0.5651109202238462, 'CIDEr': 1.1586279892363736} [INFO: 2022-03-25 16:55:02,705] ######## Epoch (VAL)4 ######## [INFO: 2022-03-25 16:55:02,705] {'Bleu_1': 0.7634881974591043, 'Bleu_2': 0.6045128667126691, 'Bleu_3': 0.46715489520559517, 'Bleu_4': 0.3596165291663262, 'METEOR': 0.2790157071440247, 'ROUGE_L': 0.565251102527745, 'CIDEr': 1.1545770264705828} [INFO: 2022-03-25 16:55:33,537] ######## Epoch (TEST)4 ######## [INFO: 2022-03-25 16:55:33,537] {'Bleu_1': 0.7611595629843703, 'Bleu_2': 0.6020151329164081, 'Bleu_3': 0.46556137091708577, 'Bleu_4': 0.3589986348050044, 'METEOR': 0.27933088953852747, 'ROUGE_L': 0.5646215017962027, 'CIDEr': 1.1635697341340177} [INFO: 2022-03-25 17:07:14,100] ######## Epoch (VAL)5 ######## [INFO: 2022-03-25 17:07:14,101] {'Bleu_1': 0.7708117488842583, 'Bleu_2': 0.6100297547289573, 'Bleu_3': 0.47225464938168615, 'Bleu_4': 0.3638810777424247, 'METEOR': 0.28171751139039897, 'ROUGE_L': 0.5700472170133182, 'CIDEr': 1.1706540412737927} [INFO: 2022-03-25 17:07:46,677] ######## Epoch (TEST)5 ######## [INFO: 2022-03-25 17:07:46,677] {'Bleu_1': 0.7692094017093853, 'Bleu_2': 0.6082708300173245, 'Bleu_3': 0.4706308407341337, 'Bleu_4': 0.36214398279537524, 'METEOR': 0.28265492753642063, 'ROUGE_L': 0.5708160353877729, 'CIDEr': 1.1876882985365016} [INFO: 2022-03-25 17:19:27,894] ######## Epoch (VAL)6 ######## [INFO: 2022-03-25 17:19:27,894] {'Bleu_1': 0.7752113437748265, 'Bleu_2': 0.6154646810865808, 'Bleu_3': 0.47679912512255285, 'Bleu_4': 0.3667357589941541, 'METEOR': 0.28326779752941195, 'ROUGE_L': 0.571812838366808, 'CIDEr': 1.1801001518818781} [INFO: 2022-03-25 17:19:59,459] ######## Epoch (TEST)6 ######## [INFO: 2022-03-25 17:19:59,459] {'Bleu_1': 0.773189850362988, 'Bleu_2': 0.6130208127766884, 'Bleu_3': 0.4731568641479194, 'Bleu_4': 0.36344151050761636, 'METEOR': 0.28305389324419483, 'ROUGE_L': 0.5711628012360231, 'CIDEr': 1.1941394557140739} [INFO: 2022-03-25 17:31:37,873] ######## Epoch (VAL)7 ######## [INFO: 2022-03-25 17:31:37,874] {'Bleu_1': 0.7733233820236535, 'Bleu_2': 0.6150534223019116, 'Bleu_3': 0.47798710765178004, 'Bleu_4': 0.37039312739754615, 'METEOR': 0.2852288447699481, 'ROUGE_L': 0.5715929351073654, 'CIDEr': 1.1908792017490206} [INFO: 2022-03-25 17:32:10,952] ######## Epoch (TEST)7 ######## [INFO: 2022-03-25 17:32:10,952] {'Bleu_1': 0.7754346150552203, 'Bleu_2': 0.614388413310137, 'Bleu_3': 0.47628621743223704, 'Bleu_4': 0.36751781007392686, 'METEOR': 0.2850058210157297, 'ROUGE_L': 0.5723354827606822, 'CIDEr': 1.2011028175316354} [INFO: 2022-03-25 17:43:51,442] ######## Epoch (VAL)8 ######## [INFO: 2022-03-25 17:43:51,442] {'Bleu_1': 0.7716909571294475, 'Bleu_2': 0.6147467912857207, 'Bleu_3': 0.4780569675900347, 'Bleu_4': 0.37017470143450665, 'METEOR': 0.2851865892843019, 'ROUGE_L': 0.5715145920033632, 'CIDEr': 1.1868539756282843} [INFO: 2022-03-25 17:44:23,136] ######## Epoch (TEST)8 ######## [INFO: 2022-03-25 17:44:23,137] {'Bleu_1': 0.7726406407263685, 'Bleu_2': 0.6126208479652554, 'Bleu_3': 0.47461904306364777, 'Bleu_4': 0.3655702288936115, 'METEOR': 0.284129202972283, 'ROUGE_L': 0.5698167767417722, 'CIDEr': 1.1922945302371477} [INFO: 2022-03-25 17:56:45,038] ######## Epoch (VAL)9 ######## [INFO: 2022-03-25 17:56:45,038] {'Bleu_1': 0.7811189331072457, 'Bleu_2': 0.6211331718378015, 'Bleu_3': 0.48060037280540197, 'Bleu_4': 0.37109273202652976, 'METEOR': 0.28704668129580335, 'ROUGE_L': 0.5747944867683352, 'CIDEr': 1.188724179327665} [INFO: 2022-03-25 17:57:19,898] ######## Epoch (TEST)9 ######## [INFO: 2022-03-25 17:57:19,898] {'Bleu_1': 0.775789564023327, 'Bleu_2': 0.6147668209604579, 'Bleu_3': 0.4746348300879723, 'Bleu_4': 0.36512617416505494, 'METEOR': 0.2852638507989185, 'ROUGE_L': 0.5718336841016628, 'CIDEr': 1.1998787402829523} [INFO: 2022-03-25 18:11:05,025] ######## Epoch (VAL)10 ######## [INFO: 2022-03-25 18:11:05,025] {'Bleu_1': 0.768226585401844, 'Bleu_2': 0.608391676892359, 'Bleu_3': 0.47087418841128853, 'Bleu_4': 0.3644691579067465, 'METEOR': 0.2853370628588019, 'ROUGE_L': 0.568258220615748, 'CIDEr': 1.1910997888272588} [INFO: 2022-03-25 18:11:47,025] ######## Epoch (TEST)10 ######## [INFO: 2022-03-25 18:11:47,025] {'Bleu_1': 0.7639393427388536, 'Bleu_2': 0.60245827551719, 'Bleu_3': 0.46458021149685635, 'Bleu_4': 0.3577118036842222, 'METEOR': 0.2839075947487805, 'ROUGE_L': 0.5675580163675967, 'CIDEr': 1.187310139840122} [INFO: 2022-03-25 18:26:11,401] ######## Epoch (VAL)11 ######## [INFO: 2022-03-25 18:26:11,401] {'Bleu_1': 0.7701997058760679, 'Bleu_2': 0.6103618189608139, 'Bleu_3': 0.47165546084330684, 'Bleu_4': 0.3638895500421402, 'METEOR': 0.28556188267605925, 'ROUGE_L': 0.5700621096033206, 'CIDEr': 1.180357806718078} [INFO: 2022-03-25 18:26:44,902] ######## Epoch (TEST)11 ######## [INFO: 2022-03-25 18:26:44,902] {'Bleu_1': 0.7668358425384748, 'Bleu_2': 0.6041706776732105, 'Bleu_3': 0.465801237879696, 'Bleu_4': 0.35853933569292856, 'METEOR': 0.2842091574971582, 'ROUGE_L': 0.5670475878192629, 'CIDEr': 1.1944596735820945} [INFO: 2022-03-25 18:38:56,461] ######## Epoch (VAL)12 ######## [INFO: 2022-03-25 18:38:56,461] {'Bleu_1': 0.7727686799567661, 'Bleu_2': 0.6122562546809791, 'Bleu_3': 0.47353152558247336, 'Bleu_4': 0.3647971009569169, 'METEOR': 0.28553535591754664, 'ROUGE_L': 0.5718032398078429, 'CIDEr': 1.1877379930265357} [INFO: 2022-03-25 18:39:29,014] ######## Epoch (TEST)12 ######## [INFO: 2022-03-25 18:39:29,014] {'Bleu_1': 0.7718810957509802, 'Bleu_2': 0.6106155103802475, 'Bleu_3': 0.4731045773092996, 'Bleu_4': 0.3655402884152335, 'METEOR': 0.2859594686483333, 'ROUGE_L': 0.5716946791685823, 'CIDEr': 1.2060311239127564} [INFO: 2022-03-25 18:51:32,510] ######## Epoch (VAL)13 ######## [INFO: 2022-03-25 18:51:32,510] {'Bleu_1': 0.7742105930591205, 'Bleu_2': 0.6123815108959906, 'Bleu_3': 0.47230147965668895, 'Bleu_4': 0.3628723545496971, 'METEOR': 0.2852318410907636, 'ROUGE_L': 0.5707796465179025, 'CIDEr': 1.19089151091835} [INFO: 2022-03-25 18:52:04,919] ######## Epoch (TEST)13 ######## [INFO: 2022-03-25 18:52:04,919] {'Bleu_1': 0.7708513647590136, 'Bleu_2': 0.6089846188131965, 'Bleu_3': 0.4697449546601519, 'Bleu_4': 0.36088186414423107, 'METEOR': 0.28653205878524785, 'ROUGE_L': 0.57061613751973, 'CIDEr': 1.2058925693564502} [INFO: 2022-03-25 19:03:51,212] ######## Epoch (VAL)14 ######## [INFO: 2022-03-25 19:03:51,213] {'Bleu_1': 0.7751880495815071, 'Bleu_2': 0.616990382241187, 'Bleu_3': 0.4803304924558586, 'Bleu_4': 0.3724606388441465, 'METEOR': 0.28792743357167694, 'ROUGE_L': 0.5741426903569211, 'CIDEr': 1.2090196732478555} [INFO: 2022-03-25 19:04:24,986] ######## Epoch (TEST)14 ######## [INFO: 2022-03-25 19:04:24,986] {'Bleu_1': 0.7709879947397283, 'Bleu_2': 0.6112609597472136, 'Bleu_3': 0.4740371821944923, 'Bleu_4': 0.3657260561070013, 'METEOR': 0.28616973238448656, 'ROUGE_L': 0.5718129661622097, 'CIDEr': 1.204804957985213} [INFO: 2022-03-25 19:16:53,541] ######## Epoch (VAL)15 ######## [INFO: 2022-03-25 19:16:53,541] {'Bleu_1': 0.7679316008928786, 'Bleu_2': 0.6057401577276523, 'Bleu_3': 0.4674167589575328, 'Bleu_4': 0.3601834399446217, 'METEOR': 0.285348756999946, 'ROUGE_L': 0.5685220972280842, 'CIDEr': 1.1758095779019135} [INFO: 2022-03-25 19:17:30,711] ######## Epoch (TEST)15 ######## [INFO: 2022-03-25 19:17:30,711] {'Bleu_1': 0.766011721549927, 'Bleu_2': 0.6039797642061746, 'Bleu_3': 0.4659948030443703, 'Bleu_4': 0.35852766360442767, 'METEOR': 0.2853768692098066, 'ROUGE_L': 0.5688851297479404, 'CIDEr': 1.1938618647074155} [INFO: 2022-03-25 19:30:05,365] ######## Epoch (VAL)16 ######## [INFO: 2022-03-25 19:30:05,365] {'Bleu_1': 0.7699051783949548, 'Bleu_2': 0.606834215417707, 'Bleu_3': 0.467105922078739, 'Bleu_4': 0.3589647264005351, 'METEOR': 0.2850236624606013, 'ROUGE_L': 0.5693791694598822, 'CIDEr': 1.1828855443460262} [INFO: 2022-03-25 19:30:40,015] ######## Epoch (TEST)16 ######## [INFO: 2022-03-25 19:30:40,015] {'Bleu_1': 0.7686013000399528, 'Bleu_2': 0.6059241941081452, 'Bleu_3': 0.46782157543665204, 'Bleu_4': 0.3614312582378431, 'METEOR': 0.286828689681855, 'ROUGE_L': 0.5708233824542729, 'CIDEr': 1.2032559208425309} [INFO: 2022-03-25 19:43:16,108] ######## Epoch (VAL)17 ######## [INFO: 2022-03-25 19:43:16,108] {'Bleu_1': 0.763260619977022, 'Bleu_2': 0.6012498937746904, 'Bleu_3': 0.4626721321953476, 'Bleu_4': 0.35614467881570716, 'METEOR': 0.28514513170072997, 'ROUGE_L': 0.567829845041875, 'CIDEr': 1.1793287721341845} [INFO: 2022-03-25 19:43:51,236] ######## Epoch (TEST)17 ######## [INFO: 2022-03-25 19:43:51,236] {'Bleu_1': 0.7632547248703637, 'Bleu_2': 0.6023073026887604, 'Bleu_3': 0.4645512837936857, 'Bleu_4': 0.35685763176664154, 'METEOR': 0.2866100364429016, 'ROUGE_L': 0.567992686970412, 'CIDEr': 1.1932564242487589}

Is it normal?

232525 commented 2 years ago

In general, it is normal (slightly insufficient? maybe). You can continue the SCST training to know more. The results of our final chosen XE model for SCST training is:

[INFO: 2021-08-23 01:52:14,715] ######## Epoch (VAL)16 ######## [INFO: 2021-08-23 01:52:14,715] {'Bleu_1': 0.7785170494529935, 'Bleu_2': 0.6202688540370216, 'Bleu_3': 0.48288052763278233, 'Bleu_4': 0.37480802263214913, 'METEOR': 0.2893485293726186, 'ROUGE_L': 0.577614615064298, 'CIDEr': 1.2182801264260739, 'SPICE': 0.2170227590036378} [INFO: 2021-08-23 01:58:48,847] ######## Epoch (TEST)16 ######## [INFO: 2021-08-23 01:58:48,848] {'Bleu_1': 0.7777327417014939, 'Bleu_2': 0.6167787156303476, 'Bleu_3': 0.4783774841959114, 'Bleu_4': 0.36858402548570396, 'METEOR': 0.2889732147622691, 'ROUGE_L': 0.576869742406256, 'CIDEr': 1.2240928659565975, 'SPICE': 0.21760507825932882}

wanboyang commented 2 years ago

the performance seems low than reported in this paper. It seems caused by the slightly lower performance of XE training stage.

the "XE" training stage [INFO: 2022-03-29 10:44:50,647] ######## Epoch (VAL)1 ######## [INFO: 2022-03-29 10:44:50,647] {'Bleu_1': 0.7247453882541082, 'Bleu_2': 0.5582907787103727, 'Bleu_3': 0.4171217747349581, 'Bleu_4': 0.3087926058787565, 'METEOR': 0.25195719634684, 'ROUGE_L': 0.5352246290724899, 'CIDEr': 0.9815993722047697} [INFO: 2022-03-29 10:45:22,093] ######## Epoch (TEST)1 ######## [INFO: 2022-03-29 10:45:22,093] {'Bleu_1': 0.7214040105270929, 'Bleu_2': 0.554745086907459, 'Bleu_3': 0.4143859591625026, 'Bleu_4': 0.3065262903750285, 'METEOR': 0.2504091667745919, 'ROUGE_L': 0.5341365895638841, 'CIDEr': 0.9877185160138512} [INFO: 2022-03-29 10:56:35,651] ######## Epoch (VAL)2 ######## [INFO: 2022-03-29 10:56:35,651] {'Bleu_1': 0.7601354744427863, 'Bleu_2': 0.5978345676144186, 'Bleu_3': 0.45620787440221805, 'Bleu_4': 0.34557934310073873, 'METEOR': 0.26855561117952714, 'ROUGE_L': 0.5601397691419213, 'CIDEr': 1.1109852003600482} [INFO: 2022-03-29 10:57:07,834] ######## Epoch (TEST)2 ######## [INFO: 2022-03-29 10:57:07,834] {'Bleu_1': 0.7604106078627956, 'Bleu_2': 0.5969506674555202, 'Bleu_3': 0.4549548810554764, 'Bleu_4': 0.34440691926062966, 'METEOR': 0.26855502925938385, 'ROUGE_L': 0.5581142903334949, 'CIDEr': 1.122726408081708} [INFO: 2022-03-29 11:08:27,735] ######## Epoch (VAL)3 ######## [INFO: 2022-03-29 11:08:27,736] {'Bleu_1': 0.7622450115284795, 'Bleu_2': 0.6010296293760446, 'Bleu_3': 0.46273598892458484, 'Bleu_4': 0.3551228497743721, 'METEOR': 0.27936211787471726, 'ROUGE_L': 0.5643210483452714, 'CIDEr': 1.1421898092922216} [INFO: 2022-03-29 11:09:00,454] ######## Epoch (TEST)3 ######## [INFO: 2022-03-29 11:09:00,454] {'Bleu_1': 0.7616674547246547, 'Bleu_2': 0.5995054943537415, 'Bleu_3': 0.462049436252612, 'Bleu_4': 0.3547988887080015, 'METEOR': 0.27941314855203575, 'ROUGE_L': 0.5648166489398633, 'CIDEr': 1.1557149097026342} [INFO: 2022-03-29 11:20:24,775] ######## Epoch (VAL)4 ######## [INFO: 2022-03-29 11:20:24,775] {'Bleu_1': 0.7671679098061569, 'Bleu_2': 0.6087307098813477, 'Bleu_3': 0.4705237143137454, 'Bleu_4': 0.36195834469974064, 'METEOR': 0.2806726532985546, 'ROUGE_L': 0.5674212721589084, 'CIDEr': 1.1624476127111862} [INFO: 2022-03-29 11:20:56,813] ######## Epoch (TEST)4 ######## [INFO: 2022-03-29 11:20:56,813] {'Bleu_1': 0.765633366591313, 'Bleu_2': 0.606963343607403, 'Bleu_3': 0.46986100924558655, 'Bleu_4': 0.36254857835822396, 'METEOR': 0.28143203364373676, 'ROUGE_L': 0.5675320604287266, 'CIDEr': 1.1784837893876052} [INFO: 2022-03-29 11:33:38,592] ######## Epoch (VAL)5 ######## [INFO: 2022-03-29 11:33:38,593] {'Bleu_1': 0.7664828278518971, 'Bleu_2': 0.6049905178021979, 'Bleu_3': 0.46667855168563016, 'Bleu_4': 0.359490272555771, 'METEOR': 0.27963862766818853, 'ROUGE_L': 0.568731589643677, 'CIDEr': 1.1717283109446948} [INFO: 2022-03-29 11:34:14,432] ######## Epoch (TEST)5 ######## [INFO: 2022-03-29 11:34:14,432] {'Bleu_1': 0.7699292941110356, 'Bleu_2': 0.6090253757159272, 'Bleu_3': 0.4696756727243853, 'Bleu_4': 0.36045503040682436, 'METEOR': 0.28156149433996147, 'ROUGE_L': 0.570558463474824, 'CIDEr': 1.1880097874172475} [INFO: 2022-03-29 11:47:00,225] ######## Epoch (VAL)6 ######## [INFO: 2022-03-29 11:47:00,225] {'Bleu_1': 0.7753301170322496, 'Bleu_2': 0.6143850323915848, 'Bleu_3': 0.4762056787932125, 'Bleu_4': 0.36767976064004215, 'METEOR': 0.28382440401538395, 'ROUGE_L': 0.5733715936483961, 'CIDEr': 1.1960498567040336} [INFO: 2022-03-29 11:47:35,936] ######## Epoch (TEST)6 ######## [INFO: 2022-03-29 11:47:35,937] {'Bleu_1': 0.7745111927163517, 'Bleu_2': 0.6141917931295976, 'Bleu_3': 0.47497490995407415, 'Bleu_4': 0.3657332299232694, 'METEOR': 0.2836271740136737, 'ROUGE_L': 0.572862598056184, 'CIDEr': 1.1966644771826074} [INFO: 2022-03-29 12:00:26,728] ######## Epoch (VAL)7 ######## [INFO: 2022-03-29 12:00:26,728] {'Bleu_1': 0.7713513170958244, 'Bleu_2': 0.6114715243467763, 'Bleu_3': 0.4731041118066275, 'Bleu_4': 0.3642001375529746, 'METEOR': 0.28402224913767515, 'ROUGE_L': 0.569534592008848, 'CIDEr': 1.185016872257861} [INFO: 2022-03-29 12:01:02,476] ######## Epoch (TEST)7 ######## [INFO: 2022-03-29 12:01:02,476] {'Bleu_1': 0.7715976080410372, 'Bleu_2': 0.6089409172699995, 'Bleu_3': 0.47015225020951995, 'Bleu_4': 0.3606154382082989, 'METEOR': 0.2860263509282357, 'ROUGE_L': 0.5706873362997136, 'CIDEr': 1.1976087547267389} [INFO: 2022-03-29 12:13:54,080] ######## Epoch (VAL)8 ######## [INFO: 2022-03-29 12:13:54,080] {'Bleu_1': 0.7740048094474344, 'Bleu_2': 0.6146434941382195, 'Bleu_3': 0.47791816824462086, 'Bleu_4': 0.370532503186726, 'METEOR': 0.2841014964855279, 'ROUGE_L': 0.571861238184942, 'CIDEr': 1.1839848186921555} [INFO: 2022-03-29 12:14:29,836] ######## Epoch (TEST)8 ######## [INFO: 2022-03-29 12:14:29,836] {'Bleu_1': 0.7677292388360443, 'Bleu_2': 0.6059704016379568, 'Bleu_3': 0.46809863381029154, 'Bleu_4': 0.36067439820538516, 'METEOR': 0.28429960115554015, 'ROUGE_L': 0.5694726487383694, 'CIDEr': 1.1890439831854371} [INFO: 2022-03-29 12:27:31,005] ######## Epoch (VAL)9 ######## [INFO: 2022-03-29 12:27:31,005] {'Bleu_1': 0.7773227953289147, 'Bleu_2': 0.6154716037653418, 'Bleu_3': 0.47501516747318046, 'Bleu_4': 0.3656308970448423, 'METEOR': 0.28555123558501877, 'ROUGE_L': 0.5717049667061956, 'CIDEr': 1.1868610387488188} [INFO: 2022-03-29 12:28:07,753] ######## Epoch (TEST)9 ######## [INFO: 2022-03-29 12:28:07,753] {'Bleu_1': 0.7743489036239452, 'Bleu_2': 0.6126574877870432, 'Bleu_3': 0.4731373143258618, 'Bleu_4': 0.36551702460650376, 'METEOR': 0.28443982707697296, 'ROUGE_L': 0.5708491879485584, 'CIDEr': 1.1985629804381288} [INFO: 2022-03-29 12:41:05,978] ######## Epoch (VAL)10 ######## [INFO: 2022-03-29 12:41:05,978] {'Bleu_1': 0.763945865933585, 'Bleu_2': 0.6018842171972376, 'Bleu_3': 0.46389448301138037, 'Bleu_4': 0.3571541796943293, 'METEOR': 0.2841622950225652, 'ROUGE_L': 0.5666408907107308, 'CIDEr': 1.1731996583337618} [INFO: 2022-03-29 12:41:43,054] ######## Epoch (TEST)10 ######## [INFO: 2022-03-29 12:41:43,054] {'Bleu_1': 0.7656905538839521, 'Bleu_2': 0.6033915611657295, 'Bleu_3': 0.46576102515094386, 'Bleu_4': 0.35902493091334997, 'METEOR': 0.2855045205289948, 'ROUGE_L': 0.5687654551719161, 'CIDEr': 1.1939741561415325} [INFO: 2022-03-29 12:54:39,096] ######## Epoch (VAL)11 ######## [INFO: 2022-03-29 12:54:39,096] {'Bleu_1': 0.7726667305277712, 'Bleu_2': 0.6131939725063422, 'Bleu_3': 0.47431945475808146, 'Bleu_4': 0.3655800008641907, 'METEOR': 0.28602692841286925, 'ROUGE_L': 0.57163397588099, 'CIDEr': 1.1865458300169058} [INFO: 2022-03-29 12:55:16,314] ######## Epoch (TEST)11 ######## [INFO: 2022-03-29 12:55:16,314] {'Bleu_1': 0.7724798495108608, 'Bleu_2': 0.6128102566828008, 'Bleu_3': 0.47485707894865903, 'Bleu_4': 0.3681820400643816, 'METEOR': 0.28724490171722467, 'ROUGE_L': 0.5736466437519466, 'CIDEr': 1.213111492502688} [INFO: 2022-03-29 13:08:16,288] ######## Epoch (VAL)12 ######## [INFO: 2022-03-29 13:08:16,288] {'Bleu_1': 0.7745840284015404, 'Bleu_2': 0.6139253819195992, 'Bleu_3': 0.4733422561358237, 'Bleu_4': 0.364294993365998, 'METEOR': 0.28458404689519123, 'ROUGE_L': 0.5701163638091513, 'CIDEr': 1.1903891442078107} [INFO: 2022-03-29 13:08:52,582] ######## Epoch (TEST)12 ######## [INFO: 2022-03-29 13:08:52,582] {'Bleu_1': 0.7716772896436505, 'Bleu_2': 0.610944131058377, 'Bleu_3': 0.47293318063948087, 'Bleu_4': 0.3658009640315867, 'METEOR': 0.2852360897403014, 'ROUGE_L': 0.5722345141499463, 'CIDEr': 1.209822868390816} [INFO: 2022-03-29 13:21:49,342] ######## Epoch (VAL)13 ######## [INFO: 2022-03-29 13:21:49,342] {'Bleu_1': 0.7740001694627866, 'Bleu_2': 0.6135151664911088, 'Bleu_3': 0.4740241504497968, 'Bleu_4': 0.3654746629722301, 'METEOR': 0.28567108370269795, 'ROUGE_L': 0.5722556294244451, 'CIDEr': 1.2014406847314845} [INFO: 2022-03-29 13:22:26,147] ######## Epoch (TEST)13 ######## [INFO: 2022-03-29 13:22:26,147] {'Bleu_1': 0.7744159894761246, 'Bleu_2': 0.6130123274215257, 'Bleu_3': 0.47356646789992396, 'Bleu_4': 0.3651064136597463, 'METEOR': 0.2872446904194274, 'ROUGE_L': 0.5736210320287891, 'CIDEr': 1.211168856470636} [INFO: 2022-03-29 13:35:25,857] ######## Epoch (VAL)14 ######## [INFO: 2022-03-29 13:35:25,857] {'Bleu_1': 0.7741962973220399, 'Bleu_2': 0.6130031844755682, 'Bleu_3': 0.4751180296223401, 'Bleu_4': 0.36588542140650754, 'METEOR': 0.28486774036796897, 'ROUGE_L': 0.5709635946543294, 'CIDEr': 1.191756577440829} [INFO: 2022-03-29 13:36:02,336] ######## Epoch (TEST)14 ######## [INFO: 2022-03-29 13:36:02,336] {'Bleu_1': 0.7755341629490325, 'Bleu_2': 0.6133314662039032, 'Bleu_3': 0.4749512950979759, 'Bleu_4': 0.36685050375493095, 'METEOR': 0.28743794554823776, 'ROUGE_L': 0.5738791718786966, 'CIDEr': 1.2128899135937532} [INFO: 2022-03-29 13:49:02,905] ######## Epoch (VAL)15 ######## [INFO: 2022-03-29 13:49:02,905] {'Bleu_1': 0.773966546355954, 'Bleu_2': 0.6123723367680349, 'Bleu_3': 0.4740558009168097, 'Bleu_4': 0.3657409714967515, 'METEOR': 0.2836633712461706, 'ROUGE_L': 0.5692278926269986, 'CIDEr': 1.1875353995904874} [INFO: 2022-03-29 13:49:40,415] ######## Epoch (TEST)15 ######## [INFO: 2022-03-29 13:49:40,415] {'Bleu_1': 0.7738520399586943, 'Bleu_2': 0.6120260384724112, 'Bleu_3': 0.4742701180288352, 'Bleu_4': 0.36699550161910455, 'METEOR': 0.2869913573014403, 'ROUGE_L': 0.5739792046397433, 'CIDEr': 1.2076548036111754} [INFO: 2022-03-29 14:02:41,477] ######## Epoch (VAL)16 ######## [INFO: 2022-03-29 14:02:41,477] {'Bleu_1': 0.7726400573512815, 'Bleu_2': 0.6119425978955048, 'Bleu_3': 0.4741803164368109, 'Bleu_4': 0.3673408161537655, 'METEOR': 0.28579364875126445, 'ROUGE_L': 0.572251056580891, 'CIDEr': 1.1929763577229122} [INFO: 2022-03-29 14:03:17,713] ######## Epoch (TEST)16 ######## [INFO: 2022-03-29 14:03:17,713] {'Bleu_1': 0.7708908512297252, 'Bleu_2': 0.6082464007124387, 'Bleu_3': 0.469652070358073, 'Bleu_4': 0.363254192089196, 'METEOR': 0.287594621298833, 'ROUGE_L': 0.5738956190405587, 'CIDEr': 1.2075282514677856} [INFO: 2022-03-29 14:16:23,521] ######## Epoch (VAL)17 ######## [INFO: 2022-03-29 14:16:23,521] {'Bleu_1': 0.7602889299512692, 'Bleu_2': 0.597935624364819, 'Bleu_3': 0.45996920099006844, 'Bleu_4': 0.3541917647001614, 'METEOR': 0.2836560184005871, 'ROUGE_L': 0.5650859615176635, 'CIDEr': 1.178156718712489} [INFO: 2022-03-29 14:16:58,731] ######## Epoch (TEST)17 ######## [INFO: 2022-03-29 14:16:58,731] {'Bleu_1': 0.7639985680291288, 'Bleu_2': 0.6016746303101609, 'Bleu_3': 0.46381241601676954, 'Bleu_4': 0.35745849182995987, 'METEOR': 0.2868364404952903, 'ROUGE_L': 0.5688119953905157, 'CIDEr': 1.2011311933952598} [INFO: 2022-03-29 14:29:59,842] ######## Epoch (VAL)18 ######## [INFO: 2022-03-29 14:29:59,842] {'Bleu_1': 0.7699919481289832, 'Bleu_2': 0.6063644251286365, 'Bleu_3': 0.46644302177300745, 'Bleu_4': 0.3586016526993709, 'METEOR': 0.28366865087963855, 'ROUGE_L': 0.5680837937756673, 'CIDEr': 1.1743736399547309} [INFO: 2022-03-29 14:30:36,505] ######## Epoch (TEST)18 ######## [INFO: 2022-03-29 14:30:36,505] {'Bleu_1': 0.7689104198537998, 'Bleu_2': 0.6041268278318207, 'Bleu_3': 0.4645945553235201, 'Bleu_4': 0.3568177613679576, 'METEOR': 0.2846742375333224, 'ROUGE_L': 0.5685875396518242, 'CIDEr': 1.1860734415051133} [INFO: 2022-03-29 14:43:56,074] ######## Epoch (VAL)19 ######## [INFO: 2022-03-29 14:43:56,074] {'Bleu_1': 0.7686829001827685, 'Bleu_2': 0.6067054508143839, 'Bleu_3': 0.46834665704829054, 'Bleu_4': 0.3612851996801912, 'METEOR': 0.2858398652774395, 'ROUGE_L': 0.5696498139325041, 'CIDEr': 1.1889615012851054} [INFO: 2022-03-29 14:44:55,670] ######## Epoch (TEST)19 ######## [INFO: 2022-03-29 14:44:55,670] {'Bleu_1': 0.7715938601862051, 'Bleu_2': 0.6092307375006375, 'Bleu_3': 0.47084790873124766, 'Bleu_4': 0.3627449857475135, 'METEOR': 0.2873154920284343, 'ROUGE_L': 0.5717021614826803, 'CIDEr': 1.2084778965161251} [INFO: 2022-03-29 15:04:28,571] ######## Epoch (VAL)20 ######## [INFO: 2022-03-29 15:04:28,572] {'Bleu_1': 0.7690197162728212, 'Bleu_2': 0.6075277197192123, 'Bleu_3': 0.46900968281817323, 'Bleu_4': 0.36061156800937433, 'METEOR': 0.2831634581075956, 'ROUGE_L': 0.5688776322067183, 'CIDEr': 1.1826135175540233} [INFO: 2022-03-29 15:05:25,992] ######## Epoch (TEST)20 ######## [INFO: 2022-03-29 15:05:25,992] {'Bleu_1': 0.771400392142475, 'Bleu_2': 0.6088945364420485, 'Bleu_3': 0.4696924093639634, 'Bleu_4': 0.36107869511252216, 'METEOR': 0.2858391098194909, 'ROUGE_L': 0.5703132396705682, 'CIDEr': 1.2006354763252507}

the SCST training stage [INFO: 2022-03-29 15:46:20,579] ######## Epoch (VAL)1 ######## [INFO: 2022-03-29 15:46:20,579] {'Bleu_1': 0.8023229681453293, 'Bleu_2': 0.6538677618007245, 'Bleu_3': 0.5109389378256096, 'Bleu_4': 0.3929069862640259, 'METEOR': 0.2893270811414356, 'ROUGE_L': 0.5894414090968995, 'CIDEr': 1.2809835124130498} [INFO: 2022-03-29 15:46:56,666] ######## Epoch (TEST)1 ######## [INFO: 2022-03-29 15:46:56,666] {'Bleu_1': 0.8040745973311654, 'Bleu_2': 0.6542689730559745, 'Bleu_3': 0.5134734312096199, 'Bleu_4': 0.39747899090053407, 'METEOR': 0.29015083365662925, 'ROUGE_L': 0.5926350608850924, 'CIDEr': 1.3015290887894506} [INFO: 2022-03-29 16:39:55,084] ######## Epoch (VAL)2 ######## [INFO: 2022-03-29 16:39:55,084] {'Bleu_1': 0.807802116597396, 'Bleu_2': 0.6612531547737988, 'Bleu_3': 0.5189127251919696, 'Bleu_4': 0.4005408320049586, 'METEOR': 0.2920054371861384, 'ROUGE_L': 0.5934643190392582, 'CIDEr': 1.3022198056707566} [INFO: 2022-03-29 16:40:30,932] ######## Epoch (TEST)2 ######## [INFO: 2022-03-29 16:40:30,932] {'Bleu_1': 0.8105146334021953, 'Bleu_2': 0.6612176764265516, 'Bleu_3': 0.5195571908527227, 'Bleu_4': 0.40246441823570045, 'METEOR': 0.29362012757158124, 'ROUGE_L': 0.5953554565516329, 'CIDEr': 1.3249394591037922} [INFO: 2022-03-29 17:54:15,478] ######## Epoch (VAL)3 ######## [INFO: 2022-03-29 17:54:15,478] {'Bleu_1': 0.8132288209761621, 'Bleu_2': 0.6652976622389033, 'Bleu_3': 0.521583038632466, 'Bleu_4': 0.4015117394045482, 'METEOR': 0.29386121620253297, 'ROUGE_L': 0.5938889405928431, 'CIDEr': 1.3194183400114443} [INFO: 2022-03-29 17:54:52,028] ######## Epoch (TEST)3 ######## [INFO: 2022-03-29 17:54:52,028] {'Bleu_1': 0.814190442204172, 'Bleu_2': 0.6643498890295545, 'Bleu_3': 0.521666344025538, 'Bleu_4': 0.40291279240699573, 'METEOR': 0.29521624754720044, 'ROUGE_L': 0.5966199041835981, 'CIDEr': 1.3375304831419803} [INFO: 2022-03-29 18:48:03,973] ######## Epoch (VAL)4 ######## [INFO: 2022-03-29 18:48:03,973] {'Bleu_1': 0.8137705088121979, 'Bleu_2': 0.6650102454308096, 'Bleu_3': 0.5209482717485655, 'Bleu_4': 0.4002258786130471, 'METEOR': 0.29417725654619975, 'ROUGE_L': 0.5929967286060597, 'CIDEr': 1.3213718042396188} [INFO: 2022-03-29 18:48:43,197] ######## Epoch (TEST)4 ######## [INFO: 2022-03-29 18:48:43,197] {'Bleu_1': 0.8162850469100345, 'Bleu_2': 0.6658931634298348, 'Bleu_3': 0.5220012751071282, 'Bleu_4': 0.40196735348928725, 'METEOR': 0.29625084513253985, 'ROUGE_L': 0.5954784336515855, 'CIDEr': 1.3409779299121918} [INFO: 2022-03-29 19:42:53,954] ######## Epoch (VAL)5 ######## [INFO: 2022-03-29 19:42:53,954] {'Bleu_1': 0.8148788533639488, 'Bleu_2': 0.6659214354388382, 'Bleu_3': 0.5212855949872696, 'Bleu_4': 0.4000659173514916, 'METEOR': 0.29480431485733677, 'ROUGE_L': 0.593557395063459, 'CIDEr': 1.326605464785419} [INFO: 2022-03-29 19:43:30,498] ######## Epoch (TEST)5 ######## [INFO: 2022-03-29 19:43:30,498] {'Bleu_1': 0.8170343667754226, 'Bleu_2': 0.6667556290977343, 'Bleu_3': 0.5227895754199599, 'Bleu_4': 0.40283467133004247, 'METEOR': 0.2967624239774498, 'ROUGE_L': 0.5972571177797447, 'CIDEr': 1.3467802412091703} [INFO: 2022-03-29 20:57:43,759] ######## Epoch (VAL)6 ######## [INFO: 2022-03-29 20:57:43,759] {'Bleu_1': 0.8171154378822924, 'Bleu_2': 0.6688878758184152, 'Bleu_3': 0.5238439902346944, 'Bleu_4': 0.4024057692525387, 'METEOR': 0.29541693716082984, 'ROUGE_L': 0.5953077028547167, 'CIDEr': 1.3315834551471368} [INFO: 2022-03-29 20:59:41,618] ######## Epoch (TEST)6 ######## [INFO: 2022-03-29 20:59:41,618] {'Bleu_1': 0.8166786464304127, 'Bleu_2': 0.6669375476056317, 'Bleu_3': 0.5232534525223929, 'Bleu_4': 0.4037112954668423, 'METEOR': 0.29678363788012124, 'ROUGE_L': 0.5965622616342436, 'CIDEr': 1.348188854601615} [INFO: 2022-03-29 22:31:53,259] ######## Epoch (VAL)7 ######## [INFO: 2022-03-29 22:31:53,259] {'Bleu_1': 0.8168225400348716, 'Bleu_2': 0.6680355784683867, 'Bleu_3': 0.5225205520062309, 'Bleu_4': 0.40123559810282466, 'METEOR': 0.2954085089150957, 'ROUGE_L': 0.5943555197042999, 'CIDEr': 1.333531420134005} [INFO: 2022-03-29 22:33:44,794] ######## Epoch (TEST)7 ######## [INFO: 2022-03-29 22:33:44,795] {'Bleu_1': 0.81848365450235, 'Bleu_2': 0.6684378942092322, 'Bleu_3': 0.5241347447707196, 'Bleu_4': 0.40380419185627564, 'METEOR': 0.29755178973716595, 'ROUGE_L': 0.5974985275377165, 'CIDEr': 1.351503128091282} [INFO: 2022-03-30 00:06:17,908] ######## Epoch (VAL)8 ######## [INFO: 2022-03-30 00:06:17,908] {'Bleu_1': 0.8183234718156953, 'Bleu_2': 0.6679576488571847, 'Bleu_3': 0.5218397724991815, 'Bleu_4': 0.40018481286283863, 'METEOR': 0.2960194842518364, 'ROUGE_L': 0.5937740229781564, 'CIDEr': 1.336000750404882} [INFO: 2022-03-30 00:08:17,506] ######## Epoch (TEST)8 ######## [INFO: 2022-03-30 00:08:17,506] {'Bleu_1': 0.8182183070325366, 'Bleu_2': 0.665377549164539, 'Bleu_3': 0.5193406973385524, 'Bleu_4': 0.39836675938665383, 'METEOR': 0.2976353116821502, 'ROUGE_L': 0.5969657084016576, 'CIDEr': 1.346231767358073} [INFO: 2022-03-30 01:40:10,215] ######## Epoch (VAL)9 ######## [INFO: 2022-03-30 01:40:10,216] {'Bleu_1': 0.8189871020682808, 'Bleu_2': 0.6684066930031677, 'Bleu_3': 0.5213670136219687, 'Bleu_4': 0.39894719617795454, 'METEOR': 0.2964342855485348, 'ROUGE_L': 0.593985583060034, 'CIDEr': 1.3353251565493425} [INFO: 2022-03-30 01:42:08,167] ######## Epoch (TEST)9 ######## [INFO: 2022-03-30 01:42:08,167] {'Bleu_1': 0.8190407744826393, 'Bleu_2': 0.6675435555375263, 'Bleu_3': 0.5230681872251349, 'Bleu_4': 0.40329032292904926, 'METEOR': 0.2987374871730973, 'ROUGE_L': 0.5972701933647744, 'CIDEr': 1.3568451145233202} [INFO: 2022-03-30 03:14:08,747] ######## Epoch (VAL)10 ######## [INFO: 2022-03-30 03:14:08,747] {'Bleu_1': 0.8178647241827122, 'Bleu_2': 0.6675768683616053, 'Bleu_3': 0.5209418350057958, 'Bleu_4': 0.3993941396591977, 'METEOR': 0.2956244771594206, 'ROUGE_L': 0.5929482265384971, 'CIDEr': 1.3322626479120445} [INFO: 2022-03-30 03:16:00,703] ######## Epoch (TEST)10 ######## [INFO: 2022-03-30 03:16:00,703] {'Bleu_1': 0.8198959187429703, 'Bleu_2': 0.6695120289911569, 'Bleu_3': 0.5255829549233132, 'Bleu_4': 0.40588818528602777, 'METEOR': 0.2986307938125697, 'ROUGE_L': 0.5982783541111346, 'CIDEr': 1.3619799900806668} [INFO: 2022-03-30 04:48:29,711] ######## Epoch (VAL)11 ######## [INFO: 2022-03-30 04:48:29,711] {'Bleu_1': 0.8190456390091522, 'Bleu_2': 0.6684549896117878, 'Bleu_3': 0.5214356573190176, 'Bleu_4': 0.39875039851570415, 'METEOR': 0.29548897826844367, 'ROUGE_L': 0.5937552819222658, 'CIDEr': 1.3327269655806908} [INFO: 2022-03-30 04:50:21,567] ######## Epoch (TEST)11 ######## [INFO: 2022-03-30 04:50:21,567] {'Bleu_1': 0.8193541556378864, 'Bleu_2': 0.6685382420599979, 'Bleu_3': 0.5238353918371624, 'Bleu_4': 0.40391213941526927, 'METEOR': 0.29808848011244354, 'ROUGE_L': 0.597321902832341, 'CIDEr': 1.357349629713385} [INFO: 2022-03-30 06:17:33,925] ######## Epoch (VAL)12 ######## [INFO: 2022-03-30 06:17:33,926] {'Bleu_1': 0.820409537560314, 'Bleu_2': 0.6701444164289361, 'Bleu_3': 0.5228786165134204, 'Bleu_4': 0.40020246665928494, 'METEOR': 0.2967893629821304, 'ROUGE_L': 0.594431979508805, 'CIDEr': 1.336742002108423} [INFO: 2022-03-30 06:19:29,509] ######## Epoch (TEST)12 ######## [INFO: 2022-03-30 06:19:29,509] {'Bleu_1': 0.818665652917679, 'Bleu_2': 0.6670754096968996, 'Bleu_3': 0.5225351007062287, 'Bleu_4': 0.40267175471915406, 'METEOR': 0.2983277700015166, 'ROUGE_L': 0.596861376105647, 'CIDEr': 1.3586142586863852} [INFO: 2022-03-30 07:46:24,431] ######## Epoch (VAL)13 ######## [INFO: 2022-03-30 07:46:24,431] {'Bleu_1': 0.8206326156954181, 'Bleu_2': 0.66961786640267, 'Bleu_3': 0.5224082180876286, 'Bleu_4': 0.3992828966803524, 'METEOR': 0.2971454321099192, 'ROUGE_L': 0.5948546500258374, 'CIDEr': 1.3382156297072947} [INFO: 2022-03-30 07:48:17,455] ######## Epoch (TEST)13 ######## [INFO: 2022-03-30 07:48:17,456] {'Bleu_1': 0.817214476478859, 'Bleu_2': 0.6665576020188411, 'Bleu_3': 0.5229951384800761, 'Bleu_4': 0.4035643476901544, 'METEOR': 0.2987752950619622, 'ROUGE_L': 0.5975871438074702, 'CIDEr': 1.3610024082156047} [INFO: 2022-03-30 09:14:55,045] ######## Epoch (VAL)14 ######## [INFO: 2022-03-30 09:14:55,045] {'Bleu_1': 0.8202197431409178, 'Bleu_2': 0.6700347078548996, 'Bleu_3': 0.5239563734987214, 'Bleu_4': 0.4012298575658356, 'METEOR': 0.296866385925953, 'ROUGE_L': 0.5946901702097699, 'CIDEr': 1.3405513065401873} [INFO: 2022-03-30 09:16:49,050] ######## Epoch (TEST)14 ######## [INFO: 2022-03-30 09:16:49,050] {'Bleu_1': 0.8198445552082281, 'Bleu_2': 0.6692867303767357, 'Bleu_3': 0.5253649879518693, 'Bleu_4': 0.4053733809658871, 'METEOR': 0.2989125081925638, 'ROUGE_L': 0.5984111063822556, 'CIDEr': 1.3635819992248037} [INFO: 2022-03-30 10:29:33,702] ######## Epoch (VAL)15 ######## [INFO: 2022-03-30 10:29:33,702] {'Bleu_1': 0.8191942230324367, 'Bleu_2': 0.6684954814133222, 'Bleu_3': 0.5219897977425628, 'Bleu_4': 0.3996953851299458, 'METEOR': 0.29684716666502203, 'ROUGE_L': 0.59484809362447, 'CIDEr': 1.3383260700887052} [INFO: 2022-03-30 10:30:06,210] ######## Epoch (TEST)15 ######## [INFO: 2022-03-30 10:30:06,210] {'Bleu_1': 0.8200380067567395, 'Bleu_2': 0.6687500149072699, 'Bleu_3': 0.5241433606236191, 'Bleu_4': 0.40390468347112063, 'METEOR': 0.29859580650674705, 'ROUGE_L': 0.5978353717614242, 'CIDEr': 1.3563864566047017} [INFO: 2022-03-30 11:15:10,202] ######## Epoch (VAL)16 ######## [INFO: 2022-03-30 11:15:10,203] {'Bleu_1': 0.8193272043459922, 'Bleu_2': 0.6691220184824213, 'Bleu_3': 0.5227848033630601, 'Bleu_4': 0.39993875682029656, 'METEOR': 0.2985399199273802, 'ROUGE_L': 0.5954565155162561, 'CIDEr': 1.3447460624558618} [INFO: 2022-03-30 11:15:43,154] ######## Epoch (TEST)16 ######## [INFO: 2022-03-30 11:15:43,154] {'Bleu_1': 0.8175777331995817, 'Bleu_2': 0.6666666201021076, 'Bleu_3': 0.5227292915317484, 'Bleu_4': 0.4027602702708449, 'METEOR': 0.299779212913522, 'ROUGE_L': 0.5974699167508084, 'CIDEr': 1.3609937302270536} [INFO: 2022-03-30 12:27:44,764] ######## Epoch (VAL)17 ######## [INFO: 2022-03-30 12:27:44,765] {'Bleu_1': 0.8177225273573264, 'Bleu_2': 0.6679124699553066, 'Bleu_3': 0.5217263590227181, 'Bleu_4': 0.39942149792827436, 'METEOR': 0.2975560306004898, 'ROUGE_L': 0.5943295539224605, 'CIDEr': 1.3395164296119602} [INFO: 2022-03-30 12:28:55,038] ######## Epoch (TEST)17 ######## [INFO: 2022-03-30 12:28:55,038] {'Bleu_1': 0.8178844056706481, 'Bleu_2': 0.6677321206015862, 'Bleu_3': 0.5240379623031027, 'Bleu_4': 0.40417954669106226, 'METEOR': 0.3000616138505837, 'ROUGE_L': 0.5985323164800282, 'CIDEr': 1.3612018315938632} [INFO: 2022-03-30 13:50:42,657] ######## Epoch (VAL)18 ######## [INFO: 2022-03-30 13:50:42,658] {'Bleu_1': 0.818690296631456, 'Bleu_2': 0.6691448128671497, 'Bleu_3': 0.5230941927906647, 'Bleu_4': 0.4007835840256963, 'METEOR': 0.2974750396034569, 'ROUGE_L': 0.5954485075401104, 'CIDEr': 1.342510283483538} [INFO: 2022-03-30 13:51:42,316] ######## Epoch (TEST)18 ######## [INFO: 2022-03-30 13:51:42,317] {'Bleu_1': 0.8190930486677142, 'Bleu_2': 0.6688342687955581, 'Bleu_3': 0.5248591617320482, 'Bleu_4': 0.40471812287203474, 'METEOR': 0.2997572772227253, 'ROUGE_L': 0.5984813017625699, 'CIDEr': 1.362652996652835} [INFO: 2022-03-30 15:12:59,216] ######## Epoch (VAL)19 ######## [INFO: 2022-03-30 15:12:59,216] {'Bleu_1': 0.8200648263812004, 'Bleu_2': 0.6707902523393878, 'Bleu_3': 0.5247614736769226, 'Bleu_4': 0.4026228112303762, 'METEOR': 0.2971018663003848, 'ROUGE_L': 0.5954518668119173, 'CIDEr': 1.3441300080210044} [INFO: 2022-03-30 15:14:07,517] ######## Epoch (TEST)19 ######## [INFO: 2022-03-30 15:14:07,517] {'Bleu_1': 0.8198241891455705, 'Bleu_2': 0.6696028284737725, 'Bleu_3': 0.5254520940040104, 'Bleu_4': 0.4053481666440005, 'METEOR': 0.29963302654078067, 'ROUGE_L': 0.5993031611334895, 'CIDEr': 1.3627018328201381} [INFO: 2022-03-30 16:35:35,310] ######## Epoch (VAL)20 ######## [INFO: 2022-03-30 16:35:35,311] {'Bleu_1': 0.8193052772210918, 'Bleu_2': 0.669169514417431, 'Bleu_3': 0.5226071665752947, 'Bleu_4': 0.3999406412967215, 'METEOR': 0.2982247524733388, 'ROUGE_L': 0.5958666322844872, 'CIDEr': 1.3432778624385209} [INFO: 2022-03-30 16:36:44,502] ######## Epoch (TEST)20 ######## [INFO: 2022-03-30 16:36:44,502] {'Bleu_1': 0.8174589889521088, 'Bleu_2': 0.6671308358736229, 'Bleu_3': 0.5226353861062081, 'Bleu_4': 0.4020007353888741, 'METEOR': 0.29931242815179837, 'ROUGE_L': 0.5986235363713338, 'CIDEr': 1.3595423280013987} [INFO: 2022-03-30 17:58:35,444] ######## Epoch (VAL)21 ######## [INFO: 2022-03-30 17:58:35,444] {'Bleu_1': 0.8186762805818194, 'Bleu_2': 0.6680034893255214, 'Bleu_3': 0.5220553932928597, 'Bleu_4': 0.3999833485874379, 'METEOR': 0.29847440899986755, 'ROUGE_L': 0.594715134255696, 'CIDEr': 1.3412624147879142} [INFO: 2022-03-30 17:59:44,046] ######## Epoch (TEST)21 ######## [INFO: 2022-03-30 17:59:44,047] {'Bleu_1': 0.8151698270472846, 'Bleu_2': 0.6634465823632005, 'Bleu_3': 0.51934542466881, 'Bleu_4': 0.39986538500848723, 'METEOR': 0.2995963552052798, 'ROUGE_L': 0.5977662065273381, 'CIDEr': 1.3585713571138696} [INFO: 2022-03-30 18:58:00,891] ######## Epoch (VAL)22 ######## [INFO: 2022-03-30 18:58:00,891] {'Bleu_1': 0.8214533162568851, 'Bleu_2': 0.6704817059806788, 'Bleu_3': 0.5239242605481028, 'Bleu_4': 0.40182701213428196, 'METEOR': 0.29785383298102325, 'ROUGE_L': 0.5962225664634064, 'CIDEr': 1.3442210502634602} [INFO: 2022-03-30 18:58:33,986] ######## Epoch (TEST)22 ######## [INFO: 2022-03-30 18:58:33,986] {'Bleu_1': 0.81785098237198, 'Bleu_2': 0.6668626175429108, 'Bleu_3': 0.5227334505123895, 'Bleu_4': 0.4034593766406667, 'METEOR': 0.29909436775777176, 'ROUGE_L': 0.5974498123149987, 'CIDEr': 1.3597678817125016}

wanboyang commented 2 years ago

I achieve the performance by using RTX3090 GPU. Could you provide the GPU used in this paper?

232525 commented 2 years ago

The performance of SCST training is really a little low, even not achieving 1.37. It is unnormal based on my experience of many experiments. I used Tesla V100 for our experiments.

wanboyang commented 2 years ago

Insteading of using the extracted features by using the swin backbone， I retrain the whole network by using raw images. the performance is normal now. It seems using extracted features can accelerate training stage but hurts the performance. [INFO: 2022-03-31 10:55:19,421] ######## Epoch (VAL)19 ######## [INFO: 2022-03-31 10:55:19,422] {'Bleu_1': 0.7746328605497517, 'Bleu_2': 0.6145045759930599, 'Bleu_3': 0.4766668567180544, 'Bleu_4': 0.3692766075383698, 'METEOR': 0.2881090826171161, 'ROUGE_L': 0.5768037509153917, 'CIDEr': 1.2130405690886492} [INFO: 2022-03-31 10:57:03,625] ######## Epoch (TEST)19 ######## [INFO: 2022-03-31 10:57:03,625] {'Bleu_1': 0.7762067591096777, 'Bleu_2': 0.6145491353114909, 'Bleu_3': 0.475394273385697, 'Bleu_4': 0.3674453187534967, 'METEOR': 0.28980076723532106, 'ROUGE_L': 0.5754550562123268, 'CIDEr': 1.2205390420092503}