search

KreitnerL / mrs-gan

GAN network that generates realistic distributed synthetic MRS samples
Other
1 stars 0 forks source link

Remove weird artifacts in generated spectra #29

Open KreitnerL opened 3 years ago

KreitnerL commented 3 years ago

fake_A and rec_A spectra exhibit weird artifacts: image

These artifact spikes seems to occur regularly in a fix distance. Applying entropy loss does not help. They appear in the very first batch and remain during the entire training process.

KreitnerL commented 3 years ago

The artifacts are generated because of the network architecture: In the generator the number of datapoints stayed the same (354). Because of this, the network was not able to directly change a specific part but could only make changes to the entire spectrum.

Fix: Make datapoints accessible by dividing them into channels. E.g. Encode + Transformer + Decoder architecture

KreitnerL commented 3 years ago

Downsampling the spectrum at least once proved effective against artifacts.

Another problem that arises, however, is that if the style vector is too big, it will cancel out the ideal spectrum. It will essentially ignore the content vector.

The cycle consistency loss is not strong enough, because the network can learn to keep a large part of the vector the same and keep vital information only in a very small part. The loss is calculated by an average, therefore the changes will not be important