Standard G & D Loss Values & Trends?

[James-G-Hill]

Hello, I've been training my first GANs, it's far from easy.

I wondered if there are typical values and trends to look for in the loss? I'm using Wasserstain-GP. The starting value of the D loss is dependent on the lambda value for the W-GP loss. If lambda is 0 then both losses start at 0; if I increase lambda then D loss starts at values higher than 0.

The general trend of the losses in my model: If lambda is greater than 0, then D loss very rapidly drops from the starting value down to 0 and stays there for some time. G loss moves above and below zero with seemingly little pattern.

So I wondered if there are typical values for the various losses and whether we should look for certain trends? And can these be added into the list of hacks? (section 10: track failures early).

[HamzahNizami]

Hey, any luck with this information?

[James-G-Hill]

@HamzahNizami It has been some time since I was working on this. You can look into the report I wrote about my project to see some graphs which show the loss for a couple of GANs variations I tried. There are only a small number of graphs but you should be able to make out what a trend that works might look like.

https://github.com/James-G-Hill/Final-Project-Intelligent-Technologies-Conditional-Audio-GANs/blob/master/Report.pdf

Check pages 18 to 22 and you'll see 4 graphs that show typical loss trends when my GANs were performing well. When they weren't performing well the loss would tend to accelerate to positive / negative infinity.

[HamzahNizami]

Thank you so much for the amazingly quick reply! Unfortunately, the link you've provided doesn't take me anywhere.

@HamzahNizami It has been some time since I was working on this. You can look into the report I wrote about my project to see some graphs which show the loss for a couple of GANs variations I tried. There are only a small number of graphs but you should be able to make out what a trend that works might look like.

https://github.com/James-G-Hill/Final-Project-Intelligent-Technologies-Conditional-Audio-GANs/blob/master/Report.pdf

Check pages 18 to 22 and you'll see 4 graphs that show typical loss trends when my GANs were performing well. When they weren't performing well the loss would tend to accelerate to positive / negative infinity.

[James-G-Hill]

Sorry, the link doesn't work for some reason but if you paste the link into your browser it will work.

Edit: I have fixed the link now so it should take you directly to the correct webpage when you click on it.

[HamzahNizami]

Thank you so much i appreicate the help