Problem with the output

[Brodvd]

@kroll-software I tried to install and run Audio Delossifier, run audio_predict with BestModel-128k-v2 but I don't understand the output, since the debug isn't have any error. I mistake anything?:

Thanks you!

[MarcoRavich]

@Brodvd Eilà, ma sei italiano...

[Brodvd]

Il mondo è piccolo!

[Brodvd]

@kroll-software In addition there are problems with the BestModel-128k-v3 . I'm running all with python 3.8.

[Brodvd]

Sorry, I didn't saw that I should modify the audio_predict to use an other model. Now the problem Is the output (the first image).

[Brodvd]

@kroll-software ?

[MarcoRavich]

@kroll-software ?

They're probably inactive since the last post on official blog is pretty old: http://www.kroll-software.ch/blog/

Anyway seems that Detlef Kroll user contributions are pretty recent:

• https://github.com/Natooz/MidiTok/issues/195
• https://github.com/AI-Guru/helibrunna/issues/2

Waiting for reply

[Brodvd]

Note: I have modify the file audio_functions about this bug: https://github.com/tensorflow/tensorflow/issues/33786 , from from keras.engine.input_layer import Input to from tensorflow.keras.layers import Input because before was error.

[kroll-software]

Note: I have modify the file audio_functions about this bug: tensorflow/tensorflow#33786 , from from keras.engine.input_layer import Input to from tensorflow.keras.layers import Input because before was error.

image.png (view on web)

Thank you, the tensorflow/keras interface is changing a lot, there is another problem with the TimeDistributed layer.

I'll soon publish an update which works with the latest tensorflow release and also adds some audio analysis functions plus easier training with automatic data preparation.

[Brodvd]

Thank you much!

[Brodvd]

@kroll-software So you think that the problem in the first image is caused from the update of tensorflow?

[kroll-software]

@kroll-software So you think that the problem in the first image is caused from the update of tensorflow?

I had to update my code several times with each new version of tensorflow and keras.

I did't publish because of low interest, which now has changed ;)

Thank you for your input and best regards,

Detlef

[MarcoRavich]

1st of all, thank for reply @kroll-software !

I'm a total noob about ML anyway - as requested in the other issue - it would be great to have a Jupiter Notebook for training (I've posted some fun code generated by driving ChatGPT with the code on this repository too: hope that inspire).

Anyway, since you're an experienced AI developer, it would be extremely interesting if you could join the issue where all started @ OpenVINO™ AI Plugins for Audacity IMHO.

My organization goal is to "delossify" many old AC3 camera audio streams through a - specifically trained (we do have many recent live music multicam shootings and multitrack+mixed or stereo audio recordings from main mixer too) neural network.

Last but not least, check out AI-based \ Audio Enhancing resources I've collected for the HyMPS project, they may inspire.

Thanks in advance.

[kroll-software]

My organization goal is to "delossify" many old AC3 camera audio streams

What is the bitrate and sampling rate of these files? That sounds like a good challenge for me ..

Best regards,

[MarcoRavich]

What is the bitrate and sampling rate of these files? That sounds like a good challenge for me .. Best regards,

2 channels @ 256 kbps / 48 kHz (CONFIRMED)

Anyway, as said, the challenge is to "remaster" them to sound as mutch as possible like a - at least - a main out mixer recording: in other words we would like to train a (WaveNet?) neural network with our recent live recordings (like this) to "recover" older ones (like this) both shooted with same cameras (Canon HF100s).

Of course we do have stereo tracks from each camera, so a pre-treatment with something like @shiranaziz's Audio Enhancement from Multiple Crowdsourced Recordings (and not only) is probably needed...

[hajdutetalb2]

@MarcoRavich I made a colab notebook at my fork: https://github.com/hajdutetalb2/AudioDelossifier/blob/master/audiodelossifier.ipynb

It's by no way finished, just a starting point. It runs but a few minutes are spent on installing the right dependencies, which is time lost on the 3 free hours of GPU time that Colab offers per day, would be better to update the code to use the already installed versions instead. Also i don't attempt to uninstall previous versions, hence why i forcefully overwrite. Audio files are taken from Google Drive and checkpoints are also saved there.

Fyi @kroll-software when inferring with the already given model, a few samples every few chunks in the resulting file were lost, not sure if a bug of the code or wrong dependency versions.

[hajdutetalb2]

What just happened? I don't think that's normal ._.

[Brodvd]

No @hajdutetalb2 , it is perfect. The epochs are the steps of the elaboration. When in your model the epochs are 100 it will stop.

[hajdutetalb2]

@Brodvd I know that, but I wouldn't expect the model to go from 49% to 100% accuracy in only two epochs, in fact I wouldn't expect it to reach 100% at all.

[Brodvd]

Ah ok. I don't know, maybe you can try with others train files .

[MarcoRavich]

@MarcoRavich I made a colab notebook at my fork: https://github.com/hajdutetalb2/AudioDelossifier/blob/master/audiodelossifier.ipynb

Cool, check this notebook code made using ChatGPT drived by me, it may help/inspire (even if it don't fully work).

I'll test yours ASAP, thanks.

@kroll-software it would be great to made test with the updated version to better understand how far it can go.

[MarcoRavich]

@MarcoRavich I made a colab notebook at my fork: https://github.com/hajdutetalb2/AudioDelossifier/blob/master/audiodelossifier.ipynb

Hi there, just tested it by prompting these commands:

!git clone https://github.com/hajdutetalb2/AudioDelossifier
%cd AudioDelossifier
!git pull -f
![ ! -d /training-data/models ] && cp -r models/ /training-data/models
!rm -r models
!ln -sf /training-data/models models

# Install Python3.9
!sudo apt-get update
!sudo apt-get install python3.9

# !python3.9 --version
!sudo apt install python3.9-distutils
!sudo apt install python3-pip

# Install Cuda 11.2 and dependencies

!python3.9 -m pip install https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-2.9.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
!python3.9 -m pip install numpy tensorflow-io==0.26.0 scipy==1.7.1 SoundFile pydub==0.25.1 librosa==0.9.1 scikit-learn==0.24.2
!python3.9 -m pip install ipython

!wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
!sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600
!wget https://developer.download.nvidia.com/compute/cuda/11.2.0/local_installers/cuda-repo-ubuntu2004-11-2-local_11.2.0-460.27.04-1_amd64.deb
!sudo dpkg --force-overwrite -i cuda-repo-ubuntu2004-11-2-local_11.2.0-460.27.04-1_amd64.deb
!sudo apt-key add /var/cuda-repo-ubuntu2004-11-2-local/7fa2af80.pub
!sudo apt-get update
!apt-get -o Dpkg::Options::="--force-overwrite" install cuda-11-2 # hacky

# !apt --fix-broken -o Dpkg::Options::="--force-overwrite"  install

!python3.9 audio_train.py

I've uploaded my training files (AUDIO_COMPRESSED and AUDIO_UNCOMPRESSED 32bit fp WAVs) and created models dir manually since your code don't ask to connect to my Drive.

Here's what i obtained:

Traceback (most recent call last):
  File "/content/AudioDelossifier/audio_train.py", line 194, in <module>
    model.fit(x_train, y_train, batch_size=cfg.BatchSize, epochs=cfg.Epochs,
  File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1420, in fit
    raise ValueError('Unexpected result of `train_function` '
ValueError: Unexpected result of `train_function` (Empty logs). Please use `Model.compile(..., run_eagerly=True)`, or `tf.config.run_functions_eagerly(True)` for more information of where went wrong, or file a issue/bug to `tf.keras`.

Let me know what I can do.

[Brodvd]

@MarcoRavichHo creato un notebook Colab sul mio fork: https://github.com/hajdutetalb2/AudioDelossifier/blob/master/audiodelossifier.ipynb

Ciao, l'ho appena testato digitando questi comandi:

!git clone https://github.com/hajdutetalb2/AudioDelossifier
%cd AudioDelossifier
!git pull -f
![ ! -d /training-data/models ] && cp -r models/ /training-data/models
!rm -r models
!ln -sf /training-data/models models

# Install Python3.9
!sudo apt-get update
!sudo apt-get install python3.9

# !python3.9 --version
!sudo apt install python3.9-distutils
!sudo apt install python3-pip

# Install Cuda 11.2 and dependencies

!python3.9 -m pip install https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-2.9.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
!python3.9 -m pip install numpy tensorflow-io==0.26.0 scipy==1.7.1 SoundFile pydub==0.25.1 librosa==0.9.1 scikit-learn==0.24.2
!python3.9 -m pip install ipython

!wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
!sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600
!wget https://developer.download.nvidia.com/compute/cuda/11.2.0/local_installers/cuda-repo-ubuntu2004-11-2-local_11.2.0-460.27.04-1_amd64.deb
!sudo dpkg --force-overwrite -i cuda-repo-ubuntu2004-11-2-local_11.2.0-460.27.04-1_amd64.deb
!sudo apt-key add /var/cuda-repo-ubuntu2004-11-2-local/7fa2af80.pub
!sudo apt-get update
!apt-get -o Dpkg::Options::="--force-overwrite" install cuda-11-2 # hacky

# !apt --fix-broken -o Dpkg::Options::="--force-overwrite"  install

!python3.9 audio_train.py

Ho caricato i miei file di formazione (AUDIO_COMPRESSED e AUDIO_UNCOMPRESSED 32bit fp WAV) e creato manualmente la directory dei modelli poiché il tuo codice non chiede di connettersi al mio Drive.

Ecco cosa ho ottenuto:

Traceback (most recent call last):
  File "/content/AudioDelossifier/audio_train.py", line 194, in <module>
    model.fit(x_train, y_train, batch_size=cfg.BatchSize, epochs=cfg.Epochs,
  File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1420, in fit
    raise ValueError('Unexpected result of `train_function` '
ValueError: Unexpected result of `train_function` (Empty logs). Please use `Model.compile(..., run_eagerly=True)`, or `tf.config.run_functions_eagerly(True)` for more information of where went wrong, or file a issue/bug to `tf.keras`.

Fammi sapere cosa posso fare.

@MarcoRavich have you correct the problem about keras engine? tensorflow/tensorflow#33786

[MarcoRavich]

@MarcoRavich have you correct the problem about keras engine? tensorflow/tensorflow#33786

Uhm, let me understand how to fix this (do I need to mod audio_train.py ?).

Anyway a good JN have to auto configure and fix any possible problems, IMHO...

[Brodvd]

@MarcoRavich if It is the same problem that I had you should modify like this:

Note: I have modify the file audio_functions about this bug: tensorflow/tensorflow#33786 , from from keras.engine.input_layer import Input to from tensorflow.keras.layers import Input because before was error.

But the problem is in audio_train , so maybe can be another, try :)

[Brodvd]

Hi @kroll-software approximately when will you update Audio Delossifier?

[MarcoRavich]

Hi @kroll-software approximately when will you update Audio Delossifier?

Hoping to avoid @kroll-software's privacy violation (he gives me these infos privately): At the moment all my computers are running on another project, but I will continue working on the Delossifier and release an update to it later, which will also make training easier.

Meanwhile I've updated the Enhancers page (more resources, better categorization) @ HyMPS project: enjoy !

[MarcoRavich]

Update. Since I just replied to @kroll-software's private mail, I wanna share some (maybe) interesting parts of with all to hopefully stimulate a more indepth technical discussion:

Having been recently fascinated by Neural Amp Modeler's capabilities (by this Leo Gibson's video on it particularry), I've become convinced that machine learning techniques can have really interesting applications in the audio "manipulation" field.

About ML technicalities: even if I'm a (90s) licentiated IT, I'm not that expert - keep in mind that I'm just a (maybe a bit advanced) USER not a developer - to evaluate the real effectiveness of different neural networks types, but I think I've understood (nice tutorial) that choosing a "right" loss function could play a crucial role.

Even if it's still in early stages, NAM - which uses a Wavenet type NN (cool article about here) and ESR loss function - does a quite good job for "mimicking" audio signals' acoustic characteristics, even if I don't know if it's the best approach. There's a very interesting "issue" by @KaisKermani on NAM repository about WaveNet vs LSTM and loss function (both quality and speed) performances that I strongly suggest you to check: [FEATURE] More input samples for LSTM architecture and Use of ESR+DC as loss fuction

Another thing I think I've realized is that providing a pre-configured Jupiter Notebook for learning helps ALL users (even those who don't have the skills and/or the needed hardware) to test the inferencing results quality and therefore provide useful feedback for possible improvement.

About inferencing: since Delossyfier relies on LTSM, have you evaluated @jatinchowdhury18's RTNeural as backend to boost local inferencing ?

Hope that inspires !

[MarcoRavich]

@kroll-software Bump. Any news ?