search

ruizhecao96 / CMGAN

Conformer-based Metric GAN for speech enhancement
MIT License
309 stars 60 forks source link

Can not reproduce the results #37

Closed coreeey closed 1 year ago

coreeey commented 1 year ago

Sorry to bother you, when I used the original parameters in Github for training, and trained for 50 epochs, the pesq only reached 2.2. I found that the generator loss would increase around 20 epochs, so I used 20 epoches for testing, with a pesq of 2.5. May I ask if this model is easily affected by random seeds or similar factors? Has PESQ not risen after 50 rounds? Here is my part of my log file

INFO:root:GPU: 0, Epoch 19, Step 500, loss: 0.13138285279273987, disc_loss: 0.0011457887012511492 INFO:root:GPU: 0, Epoch 19, Step 1000, loss: 0.13329894840717316, disc_loss: 0.002238199347630143 INFO:root:GPU: 0, Epoch 19, Step 1500, loss: 0.1375545859336853, disc_loss: 0.0013448027893900871 INFO:root:GPU: 0, Epoch 19, Step 2000, loss: 0.10363011807203293, disc_loss: 0.00407591974362731 INFO:root:GPU: 0, Epoch 19, Step 2500, loss: 0.12573812901973724, disc_loss: 0.0016270074993371964 INFO:root:GPU: 0, Generator loss: 0.10857707276506331, Discriminator loss: 0.006840812125004764 INFO:root:GPU: 0, Epoch 20, Step 500, loss: 0.13214150071144104, disc_loss: 0.0050138202495872974 INFO:root:GPU: 0, Epoch 20, Step 1000, loss: 0.1271681934595108, disc_loss: 0.001531225978396833 INFO:root:GPU: 0, Epoch 20, Step 1500, loss: 0.14278484880924225, disc_loss: 0.0020016769412904978 INFO:root:GPU: 0, Epoch 20, Step 2000, loss: 0.10626572370529175, disc_loss: 0.001245336257852614 INFO:root:GPU: 0, Epoch 20, Step 2500, loss: 0.12693673372268677, disc_loss: 0.0011474542552605271 INFO:root:GPU: 0, Generator loss: 0.11096440830711023, Discriminator loss: 0.009940159190909539 INFO:root:GPU: 0, Epoch 21, Step 500, loss: 0.13440656661987305, disc_loss: 0.002420067088678479 INFO:root:GPU: 0, Epoch 21, Step 1000, loss: 0.13075625896453857, disc_loss: 0.003778402227908373 INFO:root:GPU: 0, Epoch 21, Step 1500, loss: 0.1465650051832199, disc_loss: 0.000708427163772285 INFO:root:GPU: 0, Epoch 21, Step 2000, loss: 0.10176242142915726, disc_loss: 0.001363859511911869 INFO:root:GPU: 0, Epoch 21, Step 2500, loss: 0.12090417742729187, disc_loss: 0.003150659380480647 INFO:root:GPU: 0, Generator loss: 0.1136595553275451, Discriminator loss: 0.005748485746082524 INFO:root:GPU: 0, Epoch 22, Step 500, loss: 0.1308746635913849, disc_loss: 0.0012463699094951153 INFO:root:GPU: 0, Epoch 22, Step 1000, loss: 0.1357329934835434, disc_loss: 0.0028033568523824215 INFO:root:GPU: 0, Epoch 22, Step 1500, loss: 0.1422165483236313, disc_loss: 0.011835111305117607 INFO:root:GPU: 0, Epoch 22, Step 2000, loss: 0.10482912510633469, disc_loss: 0.002375335432589054 INFO:root:GPU: 0, Epoch 22, Step 2500, loss: 0.1230633407831192, disc_loss: 0.0025530874263495207 . . . INFO:root:GPU: 0, Generator loss: 0.10818793404015523, Discriminator loss: 0.0060944516351333486 INFO:root:GPU: 0, Epoch 50, Step 500, loss: 0.13102197647094727, disc_loss: 0.001470182673074305 INFO:root:GPU: 0, Epoch 50, Step 1000, loss: 0.12735071778297424, disc_loss: 0.00043258434743620455 INFO:root:GPU: 0, Epoch 50, Step 1500, loss: 0.13080495595932007, disc_loss: 0.0006283668917603791 INFO:root:GPU: 0, Epoch 50, Step 2000, loss: 0.09814462810754776, disc_loss: 0.0006183524965308607 INFO:root:GPU: 0, Epoch 50, Step 2500, loss: 0.11451182514429092, disc_loss: 0.0008931112242862582 INFO:root:GPU: 0, Generator loss: 0.1056402796535816, Discriminator loss: 0.004768050249429471 INFO:root:GPU: 0, Epoch 51, Step 500, loss: 0.1282331794500351, disc_loss: 0.0003159200132358819 INFO:root:GPU: 0, Epoch 51, Step 1000, loss: 0.13263651728630066, disc_loss: 0.0025379392318427563 INFO:root:GPU: 0, Epoch 51, Step 1500, loss: 0.1349661946296692, disc_loss: 0.0008474360220134258 INFO:root:GPU: 0, Epoch 51, Step 2000, loss: 0.10511360317468643, disc_loss: 0.0023576354142278433 INFO:root:GPU: 0, Epoch 51, Step 2500, loss: 0.1188545823097229, disc_loss: 0.011680787429213524 INFO:root:GPU: 0, Generator loss: 0.1116596677349609, Discriminator loss: 0.003238930973998984 INFO:root:GPU: 0, Epoch 52, Step 500, loss: 0.12579278647899628, disc_loss: 0.0030293799936771393 INFO:root:GPU: 0, Epoch 52, Step 1000, loss: 0.13355058431625366, disc_loss: 0.0017012724420055747 INFO:root:GPU: 0, Epoch 52, Step 1500, loss: 0.13323292136192322, disc_loss: 0.0010849591344594955 INFO:root:GPU: 0, Epoch 52, Step 2000, loss: 0.09986048191785812, disc_loss: 0.002493425039574504 INFO:root:GPU: 0, Epoch 52, Step 2500, loss: 0.11676986515522003, disc_loss: 0.005572836380451918 INFO:root:GPU: 0, Generator loss: 0.10965107189654147, Discriminator loss: 0.00365501402201808 INFO:root:GPU: 0, Epoch 53, Step 500, loss: 0.12720242142677307, disc_loss: 0.020244047045707703 INFO:root:GPU: 0, Epoch 53, Step 1000, loss: 0.12291329354047775, disc_loss: 0.0010470837587490678 INFO:root:GPU: 0, Epoch 53, Step 1500, loss: 0.1304602324962616, disc_loss: 0.0023244067560881376 INFO:root:GPU: 0, Epoch 53, Step 2000, loss: 0.10364727675914764, disc_loss: 0.0025291203055530787 INFO:root:GPU: 0, Epoch 53, Step 2500, loss: 0.11734053492546082, disc_loss: 0.0014599505811929703 INFO:root:GPU: 0, Generator loss: 0.10770693768575354, Discriminator loss: 0.004855742649931361 INFO:root:GPU: 0, Epoch 54, Step 500, loss: 0.12715910375118256, disc_loss: 0.0006953651900403202 INFO:root:GPU: 0, Epoch 54, Step 1000, loss: 0.1277923434972763, disc_loss: 0.0006543396739289165 INFO:root:GPU: 0, Epoch 54, Step 1500, loss: 0.12873995304107666, disc_loss: 0.001449460512958467 INFO:root:GPU: 0, Epoch 54, Step 2000, loss: 0.10310039669275284, disc_loss: 0.0012442576698958874

coreeey commented 1 year ago

Hi, I was wondering if you were able to reproduce the results and how? I had a similar PESQ of 2.2. I set training for 55 epochs and evaluated the model saved at the last epoch. I didn't modify the code other than changing the return/input of the STFT and iSTFT.

I'm sorry, I haven't reproduced the results of the article here either. I guess the torch version may have caused some differences in feature extraction, but due to time constraints, I'm not prepared to continue trying

coreeey commented 1 year ago

Hi, I was wondering if you were able to reproduce the results and how? I had a similar PESQ of 2.2. I set training for 55 epochs and evaluated the model saved at the last epoch. I didn't modify the code other than changing the return/input of the STFT and iSTFT.

I'm sorry, I haven't reproduced the results of the article here either. I guess the torch version may have caused some differences in feature extraction, but due to time constraints, I'm not prepared to continue trying

Thanks for the reply. I did get up to ~3.0 PESQ but couldn't get to 3.41. I wonder what others used in the other threads about reproducing the results, they seemed to be able to get ~3.2... Anyway I may try another repo

If it reaches 3.0, the network may converge normally. How many epochs did you get this result? Can you share your loss curve in tensorboard? I'm still quite curious about this

coreeey commented 1 year ago

So I ended up looking at this again and I got it to 3.28 PESQ, similar to this thread.

I used Sherif's version of this repository, here. I didn't do a deep compare between the repos but the biggest thing I noticed is the training dataloader was shuffled in that version whereas it isn't here. I didn't try this repo with shuffling on but I'm assuming that's the major difference.

Anyways here are my hyperparameters: epochs=55, batch_size=4, log_interval=500, decay_epoch=12, init_lr=0.0005, cut_len=32000 Loss weights: 0.3, 0.7, 1, 0.01

I evaluated at 3 different epochs (31, 52, 54) and the best result came from the very last epoch, even though the generator loss was not the lowest. So it's possible the results could have improved if it was trained longer.

Here is the training log:

INFO:root:Epoch 0, Step 500, loss: 0.312759131193161, disc_loss: 0.014180301688611507
INFO:root:Epoch 0, Step 1000, loss: 0.3000572919845581, disc_loss: 0.03669717535376549
INFO:root:Epoch 0, Step 1500, loss: 0.3671327233314514, disc_loss: 0.02478821389377117
INFO:root:Epoch 0, Step 2000, loss: 0.1893772929906845, disc_loss: 0.015164515003561974
INFO:root:Epoch 0, Step 2500, loss: 0.2994377017021179, disc_loss: 0.017490889877080917
INFO:root:Generator loss: 0.1954199135375833, Discriminator loss: 0.010855219110188717
INFO:root:Epoch 1, Step 500, loss: 0.3214994966983795, disc_loss: 0.0008018735097721219
INFO:root:Epoch 1, Step 1000, loss: 0.25855952501296997, disc_loss: 0.007563697639852762
INFO:root:Epoch 1, Step 1500, loss: 0.3672272861003876, disc_loss: 0.008135653100907803
INFO:root:Epoch 1, Step 2000, loss: 0.2542274296283722, disc_loss: 0.006575008388608694
INFO:root:Epoch 1, Step 2500, loss: 0.3384028375148773, disc_loss: 0.03157956898212433
INFO:root:Generator loss: 0.1993667078249663, Discriminator loss: 0.007349369336878632
INFO:root:Epoch 2, Step 500, loss: 0.2520577609539032, disc_loss: 0.011542283929884434
INFO:root:Epoch 2, Step 1000, loss: 0.283456027507782, disc_loss: 0.005646033212542534
INFO:root:Epoch 2, Step 1500, loss: 0.24863296747207642, disc_loss: 0.006484236102551222
INFO:root:Epoch 2, Step 2000, loss: 0.2130897492170334, disc_loss: 0.0032469583675265312
INFO:root:Epoch 2, Step 2500, loss: 0.24691542983055115, disc_loss: 0.023507077246904373
INFO:root:Generator loss: 0.18021206854327212, Discriminator loss: 0.007001019695839275
INFO:root:Epoch 3, Step 500, loss: 0.36660319566726685, disc_loss: 0.0022028894163668156
INFO:root:Epoch 3, Step 1000, loss: 0.2732304632663727, disc_loss: 0.0022656084038317204
INFO:root:Epoch 3, Step 1500, loss: 0.2375778704881668, disc_loss: 0.0063988384790718555
INFO:root:Epoch 3, Step 2000, loss: 0.2714841067790985, disc_loss: 0.007243407424539328
INFO:root:Epoch 3, Step 2500, loss: 0.20046576857566833, disc_loss: 0.0051153902895748615
INFO:root:Generator loss: 0.17687235407314253, Discriminator loss: 0.005908454136410954
INFO:root:Epoch 4, Step 500, loss: 0.26929739117622375, disc_loss: 0.002962844679132104
INFO:root:Epoch 4, Step 1000, loss: 0.18813900649547577, disc_loss: 0.0035468311980366707
INFO:root:Epoch 4, Step 1500, loss: 0.34768155217170715, disc_loss: 0.002427500206977129
INFO:root:Epoch 4, Step 2000, loss: 0.2553648054599762, disc_loss: 0.010068511590361595
INFO:root:Epoch 4, Step 2500, loss: 0.23005756735801697, disc_loss: 0.0028513793367892504
INFO:root:Generator loss: 0.16993942122580935, Discriminator loss: 0.0087586595099425
INFO:root:Epoch 5, Step 500, loss: 0.30520328879356384, disc_loss: 0.00796982366591692
INFO:root:Epoch 5, Step 1000, loss: 0.26169344782829285, disc_loss: 0.0033064233139157295
INFO:root:Epoch 5, Step 1500, loss: 0.2837878167629242, disc_loss: 0.0033368244767189026
INFO:root:Epoch 5, Step 2000, loss: 0.3022649884223938, disc_loss: 0.0015388906467705965
INFO:root:Epoch 5, Step 2500, loss: 0.19645152986049652, disc_loss: 0.0026794704608619213
INFO:root:Generator loss: 0.17689822230813573, Discriminator loss: 0.01272232923427121
INFO:root:Epoch 6, Step 500, loss: 0.29179835319519043, disc_loss: 0.0010662907734513283
INFO:root:Epoch 6, Step 1000, loss: 0.2581096589565277, disc_loss: 0.004199117422103882
INFO:root:Epoch 6, Step 1500, loss: 0.297472208738327, disc_loss: 0.0029484110418707132
INFO:root:Epoch 6, Step 2000, loss: 0.14254534244537354, disc_loss: 0.0005551559734158218
INFO:root:Epoch 6, Step 2500, loss: 0.23787181079387665, disc_loss: 0.001854499219916761
INFO:root:Generator loss: 0.1655087476071802, Discriminator loss: 0.005884178486614244
INFO:root:Epoch 7, Step 500, loss: 0.2807033061981201, disc_loss: 0.003636919194832444
INFO:root:Epoch 7, Step 1000, loss: 0.2307279258966446, disc_loss: 0.0006609580595977604
INFO:root:Epoch 7, Step 1500, loss: 0.17860105633735657, disc_loss: 0.0018876992398872972
INFO:root:Epoch 7, Step 2000, loss: 0.17049115896224976, disc_loss: 0.003694066544994712
INFO:root:Epoch 7, Step 2500, loss: 0.19690778851509094, disc_loss: 0.004949377384036779
INFO:root:Generator loss: 0.16698780728196636, Discriminator loss: 0.007069916313765452
INFO:root:Epoch 8, Step 500, loss: 0.23593047261238098, disc_loss: 0.007670776918530464
INFO:root:Epoch 8, Step 1000, loss: 0.2838055491447449, disc_loss: 0.002970515750348568
INFO:root:Epoch 8, Step 1500, loss: 0.14858031272888184, disc_loss: 0.00419649388641119
INFO:root:Epoch 8, Step 2000, loss: 0.18173342943191528, disc_loss: 0.006143204402178526
INFO:root:Epoch 8, Step 2500, loss: 0.27629354596138, disc_loss: 0.005271576810628176
INFO:root:Generator loss: 0.16036879802936488, Discriminator loss: 0.0040861366057284715
INFO:root:Epoch 9, Step 500, loss: 0.2189137190580368, disc_loss: 0.0057810521684587
INFO:root:Epoch 9, Step 1000, loss: 0.21471352875232697, disc_loss: 0.004761099815368652
INFO:root:Epoch 9, Step 1500, loss: 0.1557544767856598, disc_loss: 0.00562161672860384
INFO:root:Epoch 9, Step 2000, loss: 0.24498240649700165, disc_loss: 0.0012258212082087994
INFO:root:Epoch 9, Step 2500, loss: 0.23638907074928284, disc_loss: 0.003574251662939787
INFO:root:Generator loss: 0.1707929576050888, Discriminator loss: 0.010396833450887216
INFO:root:Epoch 10, Step 500, loss: 0.24539637565612793, disc_loss: 0.0010290932841598988
INFO:root:Epoch 10, Step 1000, loss: 0.29992005228996277, disc_loss: 0.01089942455291748
INFO:root:Epoch 10, Step 1500, loss: 0.20314528048038483, disc_loss: 0.0011890940368175507
INFO:root:Epoch 10, Step 2000, loss: 0.214199498295784, disc_loss: 0.00865689292550087
INFO:root:Epoch 10, Step 2500, loss: 0.16544727981090546, disc_loss: 0.0019958531484007835
INFO:root:Generator loss: 0.15670098707947916, Discriminator loss: 0.006327984908159689
INFO:root:Epoch 11, Step 500, loss: 0.2719419598579407, disc_loss: 0.002507247729226947
INFO:root:Epoch 11, Step 1000, loss: 0.2289179414510727, disc_loss: 0.0014765533851459622
INFO:root:Epoch 11, Step 1500, loss: 0.15163052082061768, disc_loss: 0.0020043537952005863
INFO:root:Epoch 11, Step 2000, loss: 0.2619889974594116, disc_loss: 0.00040794862434268
INFO:root:Epoch 11, Step 2500, loss: 0.21947583556175232, disc_loss: 0.0033102056477218866
INFO:root:Generator loss: 0.159215243985352, Discriminator loss: 0.008283218649471432
INFO:root:Epoch 12, Step 500, loss: 0.25795814394950867, disc_loss: 0.002124498598277569
INFO:root:Epoch 12, Step 1000, loss: 0.21739135682582855, disc_loss: 0.0023645160254091024
INFO:root:Epoch 12, Step 1500, loss: 0.21675364673137665, disc_loss: 0.0009649371495470405
INFO:root:Epoch 12, Step 2000, loss: 0.25408288836479187, disc_loss: 0.00045945390593260527
INFO:root:Epoch 12, Step 2500, loss: 0.2382950633764267, disc_loss: 0.00102323479950428
INFO:root:Generator loss: 0.15589472546594815, Discriminator loss: 0.004769237699522298
INFO:root:Epoch 13, Step 500, loss: 0.12218505889177322, disc_loss: 0.00337812677025795
INFO:root:Epoch 13, Step 1000, loss: 0.21033677458763123, disc_loss: 0.0019488180987536907
INFO:root:Epoch 13, Step 1500, loss: 0.16327118873596191, disc_loss: 0.0028526477981358767
INFO:root:Epoch 13, Step 2000, loss: 0.16071835160255432, disc_loss: 0.002321660052984953
INFO:root:Epoch 13, Step 2500, loss: 0.16742675006389618, disc_loss: 0.007365376688539982
INFO:root:Generator loss: 0.15445987167722972, Discriminator loss: 0.00741506577935635
INFO:root:Epoch 14, Step 500, loss: 0.23686109483242035, disc_loss: 0.000746804173104465
INFO:root:Epoch 14, Step 1000, loss: 0.22421026229858398, disc_loss: 0.001401600195094943
INFO:root:Epoch 14, Step 1500, loss: 0.24702338874340057, disc_loss: 0.0036804969422519207
INFO:root:Epoch 14, Step 2000, loss: 0.20147472620010376, disc_loss: 0.0014264158671721816
INFO:root:Epoch 14, Step 2500, loss: 0.21461667120456696, disc_loss: 0.0011242892360314727
INFO:root:Generator loss: 0.15769419533390444, Discriminator loss: 0.010084183753913978
INFO:root:Epoch 15, Step 500, loss: 0.18812422454357147, disc_loss: 0.00045774210593663156
INFO:root:Epoch 15, Step 1000, loss: 0.21720658242702484, disc_loss: 0.0015750795137137175
INFO:root:Epoch 15, Step 1500, loss: 0.18461938202381134, disc_loss: 0.00022904400248080492
INFO:root:Epoch 15, Step 2000, loss: 0.2020915150642395, disc_loss: 0.0012964744819328189
INFO:root:Epoch 15, Step 2500, loss: 0.24323663115501404, disc_loss: 0.0013249609619379044
INFO:root:Generator loss: 0.1528906009512619, Discriminator loss: 0.007542880900623503
INFO:root:Epoch 16, Step 500, loss: 0.2666374444961548, disc_loss: 0.0026336447335779667
INFO:root:Epoch 16, Step 1000, loss: 0.1554514616727829, disc_loss: 0.0012972827535122633
INFO:root:Epoch 16, Step 1500, loss: 0.14579275250434875, disc_loss: 0.0010206089355051517
INFO:root:Epoch 16, Step 2000, loss: 0.2060222327709198, disc_loss: 0.0013034377479925752
INFO:root:Epoch 16, Step 2500, loss: 0.1689905822277069, disc_loss: 0.0031409799121320248
INFO:root:Generator loss: 0.15393709026060057, Discriminator loss: 0.007572623159018618
INFO:root:Epoch 17, Step 500, loss: 0.2191568911075592, disc_loss: 0.0015926468186080456
INFO:root:Epoch 17, Step 1000, loss: 0.21130087971687317, disc_loss: 0.0025851901154965162
INFO:root:Epoch 17, Step 1500, loss: 0.13330042362213135, disc_loss: 0.0006709725712426007
INFO:root:Epoch 17, Step 2000, loss: 0.2143322229385376, disc_loss: 0.001584080164320767
INFO:root:Epoch 17, Step 2500, loss: 0.2092394232749939, disc_loss: 0.0036217852029949427
INFO:root:Generator loss: 0.1504001126272007, Discriminator loss: 0.00593943976636386
INFO:root:Epoch 18, Step 500, loss: 0.19369980692863464, disc_loss: 0.001540558529086411
INFO:root:Epoch 18, Step 1000, loss: 0.2042849361896515, disc_loss: 0.0012967906659469008
INFO:root:Epoch 18, Step 1500, loss: 0.296305388212204, disc_loss: 0.00024529595975764096
INFO:root:Epoch 18, Step 2000, loss: 0.19594815373420715, disc_loss: 0.0018008043989539146
INFO:root:Epoch 18, Step 2500, loss: 0.289691299200058, disc_loss: 0.0032066514249891043
INFO:root:Generator loss: 0.1606973894663806, Discriminator loss: 0.009119160932791645
INFO:root:Epoch 19, Step 500, loss: 0.16933570802211761, disc_loss: 0.0008373638265766203
INFO:root:Epoch 19, Step 1000, loss: 0.26960769295692444, disc_loss: 0.0008637731662020087
INFO:root:Epoch 19, Step 1500, loss: 0.1529017984867096, disc_loss: 0.0011711203260347247
INFO:root:Epoch 19, Step 2000, loss: 0.23268896341323853, disc_loss: 0.0012849016347900033
INFO:root:Epoch 19, Step 2500, loss: 0.2509724199771881, disc_loss: 0.0006878843414597213
INFO:root:Generator loss: 0.1548184705807746, Discriminator loss: 0.007865976493280647
INFO:root:Epoch 20, Step 500, loss: 0.23991236090660095, disc_loss: 0.0014005980920046568
INFO:root:Epoch 20, Step 1000, loss: 0.20016886293888092, disc_loss: 0.00015595517470501363
INFO:root:Epoch 20, Step 1500, loss: 0.10266516357660294, disc_loss: 0.006176771596074104
INFO:root:Epoch 20, Step 2000, loss: 0.1987983137369156, disc_loss: 0.003655677428469062
INFO:root:Epoch 20, Step 2500, loss: 0.16489088535308838, disc_loss: 0.01451909076422453
INFO:root:Generator loss: 0.15392981896412025, Discriminator loss: 0.009336405254817664
INFO:root:Epoch 21, Step 500, loss: 0.1588912457227707, disc_loss: 0.003106571501120925
INFO:root:Epoch 21, Step 1000, loss: 0.2122514247894287, disc_loss: 0.001715341117233038
INFO:root:Epoch 21, Step 1500, loss: 0.17192889750003815, disc_loss: 0.0022574549075216055
INFO:root:Epoch 21, Step 2000, loss: 0.20623578131198883, disc_loss: 0.0018650146666914225
INFO:root:Epoch 21, Step 2500, loss: 0.2032129466533661, disc_loss: 0.001579348579980433
INFO:root:Generator loss: 0.15073225383185646, Discriminator loss: 0.00716394909186508
INFO:root:Epoch 22, Step 500, loss: 0.21944525837898254, disc_loss: 0.0006659588543698192
INFO:root:Epoch 22, Step 1000, loss: 0.1950821578502655, disc_loss: 0.0035487571731209755
INFO:root:Epoch 22, Step 1500, loss: 0.13821372389793396, disc_loss: 0.0026758627500385046
INFO:root:Epoch 22, Step 2000, loss: 0.16071826219558716, disc_loss: 0.000786982593126595
INFO:root:Epoch 22, Step 2500, loss: 0.1513395607471466, disc_loss: 0.003028666600584984
INFO:root:Generator loss: 0.1656777167566193, Discriminator loss: 0.00689646471691605
INFO:root:Epoch 23, Step 500, loss: 0.17124043405056, disc_loss: 0.00040915425051935017
INFO:root:Epoch 23, Step 1000, loss: 0.23920418322086334, disc_loss: 0.0017987532773986459
INFO:root:Epoch 23, Step 1500, loss: 0.14390625059604645, disc_loss: 0.001338382251560688
INFO:root:Epoch 23, Step 2000, loss: 0.19782759249210358, disc_loss: 0.001112670754082501
INFO:root:Epoch 23, Step 2500, loss: 0.20089788734912872, disc_loss: 0.0020863921381533146
INFO:root:Generator loss: 0.15288031542474784, Discriminator loss: 0.005006089134554908
INFO:root:Epoch 24, Step 500, loss: 0.21623064577579498, disc_loss: 0.0004943881649523973
INFO:root:Epoch 24, Step 1000, loss: 0.1983931064605713, disc_loss: 0.0024461522698402405
INFO:root:Epoch 24, Step 1500, loss: 0.22622570395469666, disc_loss: 0.054060615599155426
INFO:root:Epoch 24, Step 2000, loss: 0.1344752013683319, disc_loss: 0.00027848611352965236
INFO:root:Epoch 24, Step 2500, loss: 0.22571316361427307, disc_loss: 0.06606058031320572
INFO:root:Generator loss: 0.1486151763464062, Discriminator loss: 0.0073336784491084914
INFO:root:Epoch 25, Step 500, loss: 0.1643536388874054, disc_loss: 0.002177179092541337
INFO:root:Epoch 25, Step 1000, loss: 0.1477476805448532, disc_loss: 0.003139293985441327
INFO:root:Epoch 25, Step 1500, loss: 0.23426830768585205, disc_loss: 0.0012699420331045985
INFO:root:Epoch 25, Step 2000, loss: 0.15443092584609985, disc_loss: 0.0009152680868282914
INFO:root:Epoch 25, Step 2500, loss: 0.14525850117206573, disc_loss: 0.0014707488007843494
INFO:root:Generator loss: 0.151314642063333, Discriminator loss: 0.006845492455804295
INFO:root:Epoch 26, Step 500, loss: 0.17136962711811066, disc_loss: 0.0028529104311019182
INFO:root:Epoch 26, Step 1000, loss: 0.19324925541877747, disc_loss: 0.0018980796448886395
INFO:root:Epoch 26, Step 1500, loss: 0.1829681247472763, disc_loss: 0.027104754000902176
INFO:root:Epoch 26, Step 2000, loss: 0.2756149172782898, disc_loss: 0.0010534025495871902
INFO:root:Epoch 26, Step 2500, loss: 0.22364293038845062, disc_loss: 0.000749379163607955
INFO:root:Generator loss: 0.14807583926950843, Discriminator loss: 0.010684788025703277
INFO:root:Epoch 27, Step 500, loss: 0.20551946759223938, disc_loss: 0.0022747379262000322
INFO:root:Epoch 27, Step 1000, loss: 0.2037193924188614, disc_loss: 0.0014990002382546663
INFO:root:Epoch 27, Step 1500, loss: 0.28234779834747314, disc_loss: 0.001453203265555203
INFO:root:Epoch 27, Step 2000, loss: 0.2405531257390976, disc_loss: 0.002639435464516282
INFO:root:Epoch 27, Step 2500, loss: 0.20126855373382568, disc_loss: 0.0028889309614896774
INFO:root:Generator loss: 0.15275396493453425, Discriminator loss: 0.00758786126028196
INFO:root:Epoch 28, Step 500, loss: 0.17823928594589233, disc_loss: 0.0006160064949654043
INFO:root:Epoch 28, Step 1000, loss: 0.18538382649421692, disc_loss: 0.001021132804453373
INFO:root:Epoch 28, Step 1500, loss: 0.18735086917877197, disc_loss: 0.00253280415199697
INFO:root:Epoch 28, Step 2000, loss: 0.1933680772781372, disc_loss: 0.0010457060998305678
INFO:root:Epoch 28, Step 2500, loss: 0.1937006562948227, disc_loss: 0.0017968968022614717
INFO:root:Generator loss: 0.15479988152685675, Discriminator loss: 0.008467569512857471
INFO:root:Epoch 29, Step 500, loss: 0.17594964802265167, disc_loss: 0.0018970801029354334
INFO:root:Epoch 29, Step 1000, loss: 0.16756503283977509, disc_loss: 0.001571777742356062
INFO:root:Epoch 29, Step 1500, loss: 0.20182061195373535, disc_loss: 0.0032018704805523157
INFO:root:Epoch 29, Step 2000, loss: 0.23015911877155304, disc_loss: 0.0011170469224452972
INFO:root:Epoch 29, Step 2500, loss: 0.21530109643936157, disc_loss: 0.0010174177587032318
INFO:root:Generator loss: 0.15145057978705295, Discriminator loss: 0.0073985461544863445
INFO:root:Epoch 30, Step 500, loss: 0.18501895666122437, disc_loss: 0.0007586692809127271
INFO:root:Epoch 30, Step 1000, loss: 0.16187520325183868, disc_loss: 0.004758385010063648
INFO:root:Epoch 30, Step 1500, loss: 0.17666107416152954, disc_loss: 0.0039044057484716177
INFO:root:Epoch 30, Step 2000, loss: 0.14470462501049042, disc_loss: 0.0010651007760316133
INFO:root:Epoch 30, Step 2500, loss: 0.13727925717830658, disc_loss: 0.0047175087966024876
INFO:root:Generator loss: 0.14848027671280417, Discriminator loss: 0.00486005955364146
INFO:root:Epoch 31, Step 500, loss: 0.1879275143146515, disc_loss: 0.001492742565460503
INFO:root:Epoch 31, Step 1000, loss: 0.13713416457176208, disc_loss: 0.0016783855389803648
INFO:root:Epoch 31, Step 1500, loss: 0.20209118723869324, disc_loss: 0.004142406862229109
INFO:root:Epoch 31, Step 2000, loss: 0.168260857462883, disc_loss: 0.00043510296382009983
INFO:root:Epoch 31, Step 2500, loss: 0.24361303448677063, disc_loss: 0.0011475462233647704
INFO:root:Generator loss: 0.1451130717341761, Discriminator loss: 0.006827290648372794
INFO:root:Epoch 32, Step 500, loss: 0.2178606390953064, disc_loss: 0.0003761161060538143
INFO:root:Epoch 32, Step 1000, loss: 0.1325763314962387, disc_loss: 0.0015204480150714517
INFO:root:Epoch 32, Step 1500, loss: 0.19770699739456177, disc_loss: 0.0013429338578134775
INFO:root:Epoch 32, Step 2000, loss: 0.15416398644447327, disc_loss: 0.001986399292945862
INFO:root:Epoch 32, Step 2500, loss: 0.1255073994398117, disc_loss: 0.04426322504878044
INFO:root:Generator loss: 0.1506036072607758, Discriminator loss: 0.008968745695384112
INFO:root:Epoch 33, Step 500, loss: 0.15152476727962494, disc_loss: 0.001831963425502181
INFO:root:Epoch 33, Step 1000, loss: 0.20520329475402832, disc_loss: 0.0021505120676010847
INFO:root:Epoch 33, Step 1500, loss: 0.15221679210662842, disc_loss: 0.0006668538553640246
INFO:root:Epoch 33, Step 2000, loss: 0.16760966181755066, disc_loss: 0.00034050486283376813
INFO:root:Epoch 33, Step 2500, loss: 0.13106703758239746, disc_loss: 0.0009691817685961723
INFO:root:Generator loss: 0.14916171046715337, Discriminator loss: 0.006695696125920999
INFO:root:Epoch 34, Step 500, loss: 0.17272314429283142, disc_loss: 0.000887170375790447
INFO:root:Epoch 34, Step 1000, loss: 0.2132679969072342, disc_loss: 0.0016812332905828953
INFO:root:Epoch 34, Step 1500, loss: 0.16384224593639374, disc_loss: 0.0019412646070122719
INFO:root:Epoch 34, Step 2000, loss: 0.2238304764032364, disc_loss: 0.0032721676398068666
INFO:root:Epoch 34, Step 2500, loss: 0.22253461182117462, disc_loss: 0.0025769562926143408
INFO:root:Generator loss: 0.151082797862083, Discriminator loss: 0.007573019889481727
INFO:root:Epoch 35, Step 500, loss: 0.22639697790145874, disc_loss: 0.0017826772527769208
INFO:root:Epoch 35, Step 1000, loss: 0.17547865211963654, disc_loss: 0.003049442544579506
INFO:root:Epoch 35, Step 1500, loss: 0.13126714527606964, disc_loss: 0.0017061545513570309
INFO:root:Epoch 35, Step 2000, loss: 0.24197342991828918, disc_loss: 0.0005070678307674825
INFO:root:Epoch 35, Step 2500, loss: 0.19099745154380798, disc_loss: 0.0006508170044980943
INFO:root:Generator loss: 0.15685435558117708, Discriminator loss: 0.008581994546439445
INFO:root:Epoch 36, Step 500, loss: 0.17108827829360962, disc_loss: 0.0014752569841220975
INFO:root:Epoch 36, Step 1000, loss: 0.2389165610074997, disc_loss: 0.0015733256004750729
INFO:root:Epoch 36, Step 1500, loss: 0.19396057724952698, disc_loss: 0.0002904093998949975
INFO:root:Epoch 36, Step 2000, loss: 0.16288608312606812, disc_loss: 0.0021133290138095617
INFO:root:Epoch 36, Step 2500, loss: 0.20927363634109497, disc_loss: 0.000661283906083554
INFO:root:Generator loss: 0.14912311179400647, Discriminator loss: 0.0067926698288432
INFO:root:Epoch 37, Step 500, loss: 0.18722005188465118, disc_loss: 0.002159852534532547
INFO:root:Epoch 37, Step 1000, loss: 0.15937204658985138, disc_loss: 0.0018713129684329033
INFO:root:Epoch 37, Step 1500, loss: 0.15392547845840454, disc_loss: 0.0020612888038158417
INFO:root:Epoch 37, Step 2000, loss: 0.14782842993736267, disc_loss: 0.0010478856274858117
INFO:root:Epoch 37, Step 2500, loss: 0.2138817310333252, disc_loss: 0.00011799987987615168
INFO:root:Generator loss: 0.15218570727167777, Discriminator loss: 0.008030950438694066
INFO:root:Epoch 38, Step 500, loss: 0.12195228040218353, disc_loss: 0.0006151677225716412
INFO:root:Epoch 38, Step 1000, loss: 0.1600419580936432, disc_loss: 0.0010397137375548482
INFO:root:Epoch 38, Step 1500, loss: 0.15388871729373932, disc_loss: 0.0020387968979775906
INFO:root:Epoch 38, Step 2000, loss: 0.14046302437782288, disc_loss: 0.0007483579684048891
INFO:root:Epoch 38, Step 2500, loss: 0.15302549302577972, disc_loss: 0.0016489940462633967
INFO:root:Generator loss: 0.14867327575689382, Discriminator loss: 0.006284861755048748
INFO:root:Epoch 39, Step 500, loss: 0.17320281267166138, disc_loss: 0.0036755544133484364
INFO:root:Epoch 39, Step 1000, loss: 0.15600933134555817, disc_loss: 0.0006042426102794707
INFO:root:Epoch 39, Step 1500, loss: 0.20386555790901184, disc_loss: 0.004750823602080345
INFO:root:Epoch 39, Step 2000, loss: 0.15265820920467377, disc_loss: 0.0014454181073233485
INFO:root:Epoch 39, Step 2500, loss: 0.19368837773799896, disc_loss: 0.002038124715909362
INFO:root:Generator loss: 0.14978806808301545, Discriminator loss: 0.008053443771185132
INFO:root:Epoch 40, Step 500, loss: 0.1634645313024521, disc_loss: 0.0008760954951867461
INFO:root:Epoch 40, Step 1000, loss: 0.24363578855991364, disc_loss: 0.0014732916606590152
INFO:root:Epoch 40, Step 1500, loss: 0.1921803057193756, disc_loss: 0.001860494608990848
INFO:root:Epoch 40, Step 2000, loss: 0.12478583306074142, disc_loss: 0.0015014746459200978
INFO:root:Epoch 40, Step 2500, loss: 0.20051966607570648, disc_loss: 0.0013981866650283337
INFO:root:Generator loss: 0.1495014434879266, Discriminator loss: 0.008858757580755373
INFO:root:Epoch 41, Step 500, loss: 0.1557866483926773, disc_loss: 0.0009908645879477262
INFO:root:Epoch 41, Step 1000, loss: 0.14158287644386292, disc_loss: 0.01940055377781391
INFO:root:Epoch 41, Step 1500, loss: 0.15362367033958435, disc_loss: 0.0011035962961614132
INFO:root:Epoch 41, Step 2000, loss: 0.18089531362056732, disc_loss: 0.0012486111372709274
INFO:root:Epoch 41, Step 2500, loss: 0.1619526594877243, disc_loss: 0.0009440151043236256
INFO:root:Generator loss: 0.1488416255677788, Discriminator loss: 0.0070254697599470865
INFO:root:Epoch 42, Step 500, loss: 0.19800350069999695, disc_loss: 0.001883466960862279
INFO:root:Epoch 42, Step 1000, loss: 0.22153475880622864, disc_loss: 0.0011596799595281482
INFO:root:Epoch 42, Step 1500, loss: 0.1665220558643341, disc_loss: 0.0014407207490876317
INFO:root:Epoch 42, Step 2000, loss: 0.13804322481155396, disc_loss: 0.0003912303363904357
INFO:root:Epoch 42, Step 2500, loss: 0.22652436792850494, disc_loss: 0.0013467944227159023
INFO:root:Generator loss: 0.15468112484199328, Discriminator loss: 0.007592531941854847
INFO:root:Epoch 43, Step 500, loss: 0.1732303947210312, disc_loss: 0.0006291429745033383
INFO:root:Epoch 43, Step 1000, loss: 0.22385995090007782, disc_loss: 0.09368821978569031
INFO:root:Epoch 43, Step 1500, loss: 0.19474774599075317, disc_loss: 0.0015064507024362683
INFO:root:Epoch 43, Step 2000, loss: 0.19128721952438354, disc_loss: 0.004481759387999773
INFO:root:Epoch 43, Step 2500, loss: 0.1631593257188797, disc_loss: 0.0011000009253621101
INFO:root:Generator loss: 0.14729728560569216, Discriminator loss: 0.009075007488435694
INFO:root:Epoch 44, Step 500, loss: 0.17715486884117126, disc_loss: 0.0018717246130108833
INFO:root:Epoch 44, Step 1000, loss: 0.25556033849716187, disc_loss: 0.000396895018639043
INFO:root:Epoch 44, Step 1500, loss: 0.21550242602825165, disc_loss: 0.0014451321912929416
INFO:root:Epoch 44, Step 2000, loss: 0.1984427273273468, disc_loss: 0.0024722693488001823
INFO:root:Epoch 44, Step 2500, loss: 0.21361075341701508, disc_loss: 0.000819200009573251
INFO:root:Generator loss: 0.14986323091445616, Discriminator loss: 0.006630657830224593
INFO:root:Epoch 45, Step 500, loss: 0.20685818791389465, disc_loss: 0.0010365750640630722
INFO:root:Epoch 45, Step 1000, loss: 0.2404719889163971, disc_loss: 0.0006383370491676033
INFO:root:Epoch 45, Step 1500, loss: 0.17369501292705536, disc_loss: 0.0033343026880174875
INFO:root:Epoch 45, Step 2000, loss: 0.17553487420082092, disc_loss: 0.0008124223095364869
INFO:root:Epoch 45, Step 2500, loss: 0.20491598546504974, disc_loss: 9.308904554927722e-05
INFO:root:Generator loss: 0.14902781203244497, Discriminator loss: 0.00557117249770528
INFO:root:Epoch 46, Step 500, loss: 0.18839885294437408, disc_loss: 0.001555745955556631
INFO:root:Epoch 46, Step 1000, loss: 0.2238050103187561, disc_loss: 0.0016249532345682383
INFO:root:Epoch 46, Step 1500, loss: 0.13696785271167755, disc_loss: 0.0016735948156565428
INFO:root:Epoch 46, Step 2000, loss: 0.1770334541797638, disc_loss: 0.0014958776300773025
INFO:root:Epoch 46, Step 2500, loss: 0.2255016267299652, disc_loss: 0.0013764165341854095
INFO:root:Generator loss: 0.15075078678941264, Discriminator loss: 0.007199312354343631
INFO:root:Epoch 47, Step 500, loss: 0.26174628734588623, disc_loss: 0.0010621807305142283
INFO:root:Epoch 47, Step 1000, loss: 0.16118288040161133, disc_loss: 0.002037838101387024
INFO:root:Epoch 47, Step 1500, loss: 0.21030934154987335, disc_loss: 0.00024075487453956157
INFO:root:Epoch 47, Step 2000, loss: 0.2071937471628189, disc_loss: 0.000964963692240417
INFO:root:Epoch 47, Step 2500, loss: 0.1436789333820343, disc_loss: 0.0022595354821532965
INFO:root:Generator loss: 0.1490943245951412, Discriminator loss: 0.00782963457995985
INFO:root:Epoch 48, Step 500, loss: 0.16596905887126923, disc_loss: 0.0016859389143064618
INFO:root:Epoch 48, Step 1000, loss: 0.20538239181041718, disc_loss: 0.000355234689777717
INFO:root:Epoch 48, Step 1500, loss: 0.12697157263755798, disc_loss: 0.00154679617844522
INFO:root:Epoch 48, Step 2000, loss: 0.15941494703292847, disc_loss: 0.00042753099114634097
INFO:root:Epoch 48, Step 2500, loss: 0.12591500580310822, disc_loss: 0.001462430227547884
INFO:root:Generator loss: 0.15104802408554022, Discriminator loss: 0.0062026559350219805
INFO:root:Epoch 49, Step 500, loss: 0.19275020062923431, disc_loss: 0.000377537973690778
INFO:root:Epoch 49, Step 1000, loss: 0.18330758810043335, disc_loss: 8.651253301650286e-05
INFO:root:Epoch 49, Step 1500, loss: 0.21846988797187805, disc_loss: 0.000389557215385139
INFO:root:Epoch 49, Step 2000, loss: 0.2317807674407959, disc_loss: 0.0027107701171189547
INFO:root:Epoch 49, Step 2500, loss: 0.2366754412651062, disc_loss: 0.00023985184088815004
INFO:root:Generator loss: 0.1506285196152127, Discriminator loss: 0.006962537650429775
INFO:root:Epoch 50, Step 500, loss: 0.18307413160800934, disc_loss: 0.0006114450516179204
INFO:root:Epoch 50, Step 1000, loss: 0.18070276081562042, disc_loss: 0.0014288920210674405
INFO:root:Epoch 50, Step 1500, loss: 0.16713868081569672, disc_loss: 0.0002362645900575444
INFO:root:Epoch 50, Step 2000, loss: 0.1450747698545456, disc_loss: 0.00239872420206666
INFO:root:Epoch 50, Step 2500, loss: 0.26365476846694946, disc_loss: 0.001221455866470933
INFO:root:Generator loss: 0.15042886561126384, Discriminator loss: 0.006950676874783189
INFO:root:Epoch 51, Step 500, loss: 0.2152542918920517, disc_loss: 0.0007097358466126025
INFO:root:Epoch 51, Step 1000, loss: 0.12946553528308868, disc_loss: 0.000846027338411659
INFO:root:Epoch 51, Step 1500, loss: 0.16722026467323303, disc_loss: 0.0010668804170563817
INFO:root:Epoch 51, Step 2000, loss: 0.2161879539489746, disc_loss: 0.0006570870755240321
INFO:root:Epoch 51, Step 2500, loss: 0.19870325922966003, disc_loss: 0.0002888255112338811
INFO:root:Generator loss: 0.15483897298719118, Discriminator loss: 0.010537156224512446
INFO:root:Epoch 52, Step 500, loss: 0.1790369302034378, disc_loss: 0.000788058852776885
INFO:root:Epoch 52, Step 1000, loss: 0.16181029379367828, disc_loss: 0.0019506269600242376
INFO:root:Epoch 52, Step 1500, loss: 0.1600004881620407, disc_loss: 0.0030104797333478928
INFO:root:Epoch 52, Step 2000, loss: 0.13153015077114105, disc_loss: 0.0012453808449208736
INFO:root:Epoch 52, Step 2500, loss: 0.17636753618717194, disc_loss: 0.0017272598342970014
INFO:root:Generator loss: 0.15071525269197028, Discriminator loss: 0.007104984784693213
INFO:root:Epoch 53, Step 500, loss: 0.16577422618865967, disc_loss: 0.0005849118460901082
INFO:root:Epoch 53, Step 1000, loss: 0.1668897420167923, disc_loss: 0.0014300784096121788
INFO:root:Epoch 53, Step 1500, loss: 0.1756283938884735, disc_loss: 0.0009977087611332536
INFO:root:Epoch 53, Step 2000, loss: 0.1373414546251297, disc_loss: 0.0019349803915247321
INFO:root:Epoch 53, Step 2500, loss: 0.15059304237365723, disc_loss: 0.00017773735453374684
INFO:root:Generator loss: 0.1531875601526603, Discriminator loss: 0.006457391921305793
INFO:root:Epoch 54, Step 500, loss: 0.16114716231822968, disc_loss: 0.005647638346999884
INFO:root:Epoch 54, Step 1000, loss: 0.13460399210453033, disc_loss: 0.000806011026725173
INFO:root:Epoch 54, Step 1500, loss: 0.19509609043598175, disc_loss: 0.0008489741594530642
INFO:root:Epoch 54, Step 2000, loss: 0.16986267268657684, disc_loss: 0.0007771639502607286
INFO:root:Epoch 54, Step 2500, loss: 0.24058441817760468, disc_loss: 0.002403619699180126
INFO:root:Generator loss: 0.15293034997958582, Discriminator loss: 0.007628766357564159

Although there are still some differences between this result with the paper provided, compared to many other methods, this result is already very good. Thank you for your detailed response. I think I may try this repo again in the next few days :)