search

Orkis-Research / Quaternion-Convolutional-Neural-Networks-for-End-to-End-Automatic-Speech-Recognition

This is the code for the paper 'Quaternion Convolutional Neural Networks for End-to-End Automatic Speech Recognition'. It provides all the Keras classes to run a Quaternion Convolutional NN.
GNU General Public License v3.0
62 stars 16 forks source link

QuaternionDense layer only accept input of dimension 2 #3

Open healthyertech opened 3 years ago

healthyertech commented 3 years ago

Hi, I'm recentrly approaching to quaternion neural networks and I found this fantastic repository which I am exploiting to implement an academic project. I'm developing a convolutional neural network for a project about speech recognition. After having defined a real-valued neural model, I was trying to convert it into a quaternion model. However, I noticed that the QuaternionDense layer accept ony inputs having dimension 2, while in the fully connected block of my model I would need a dense layer accepting input at least of dimension 3. I tried to slightly modify in my local folder the definition of the quaternion layer, but it slill retuns me a ValueError stressing that the input shape does not mathc the required shape. Would it actually be possible to make the quaternion dense layer working with input of size 3? If so, how could I proceed? Kind regards

TParcollet commented 3 years ago

Hi and thanks for you interest.

This is a pretty old code :( If you are interested in E2E ASR with quaternions, please have a look at https://speechbrain.github.io instead :)

healthyertech commented 3 years ago

Ok, thanks a lot for the advice. I also looked at that repository, even though my interest in this one was also related to the fact that I wanted to integrate it into a tensorflow project of mine, extending it by using quaternion neural models For the tensorflow library, I also found the implementation of a quaternion batch normalization layer, and I would like to ask if something similar could be possibly available also for the quaternion layers library implemented in pytorch. Kind regards.

TParcollet commented 3 years ago

Yes, here: https://github.com/speechbrain/speechbrain/blob/develop/speechbrain/nnet/quaternion_networks/q_normalization.py

Note: This is not done by using the whitening approach that is WAY too much memory demanding. The reference is given in the doctoring of the class.

healthyertech commented 3 years ago

Thank you very much for the helpful and quick reply and for the useful advices. Kind regards