DataLoader class to load MFCC features and e2e-format FSTs from disk into a pair of features Tensor and FST and create minibatches.
We can follow the way Deepspeech does this i.e. to use an SCP-like file that has the filepaths so that the features and FSTs can be read at the time of creating the minibatch. This should be implemented such that it can return a single pair (features, fst) read from disk when used with python iterator.
This probably needs as input an SCP-like file that has all the paths for features as well as FSTs including additional auxiliary information such as the duration of the utterance, which will be required by the Sampler.
We assume that the SCP-like file is sorted by length so we can create minibatches of same chunk size. A Sampler will use this information to create minibatches. #3
DataLoader
class to load MFCC features and e2e-format FSTs from disk into a pair of features Tensor and FST and create minibatches.We can follow the way Deepspeech does this i.e. to use an SCP-like file that has the filepaths so that the features and FSTs can be read at the time of creating the minibatch. This should be implemented such that it can return a single pair (features, fst) read from disk when used with python iterator.