Loss of signal in song roundtrip -- try keeping in amplitude space

sbenthall / deeptune

Music style transfer

MIT License

2 stars 0 forks source link

Loss of signal in song roundtrip -- try keeping in amplitude space #14

Closed sbenthall closed 4 years ago

sbenthall commented 4 years ago

There is some loss of signal in the roundtrip of a song from mp3 -> wav -> numpy -> wav (-> mp3).

On possibility is that this is being lost in the conversion from amplitude to decibel spaces. Another is that it's being lost in the short term Fourier transform step.

Experiment with

[x] reducing the amount of preprocessing and seeing which steps are causing the information loss
[x] checking which "purer" forms are still exportable to numpy and feasible to import into the CNN

sbenthall commented 4 years ago

I believe that staying in the amplitude space will result in the input matrix values being complex numbers.

Tensorflow supports complex numbers as a data type:

https://www.tensorflow.org/api_docs/python/tf/dtypes/complex

sbenthall commented 4 years ago

For visualizing in the complex plane: https://stackoverflow.com/questions/17044052/mathplotlib-imshow-complex-2d-array