This pull request extends the ability of the code to deal with various sources of data. It introduces the abstractions of
Signals and
Machines
Each of these are objects that carry the relevant information necessary for incorporating them into the overall pipeline. Signals know the Machine they live on, their mds+ paths, code for being downloaded, preprocessing approaches, their dimensionality, etc. Machines know which Signals are defined on them, which mds+ server houses the data, etc.
This required changes in
downloading (use the signals)
preprocessing (use signals)
normalization (need one normalizer for each machine)
model building (build model based on which signals are being used. Add convolutional layers for 1D signals)
shot storage (store data as a dictionary indexed by the Signal)
As such, this adds the following functionality
multi machine downloading, training, testing
handling of several different types of signals, simple selection of which signal subset to use, without re-downloading or re-preprocessing
handling of different types of signals, such as different dimensionalities (1D data)
This pull request also fixes various smaller and unrelated bugs that were uncovered.
1D profiles are currently not supported with the Theano backend, as there is a bug when constructing slicing layers in the model builder. The implementation creates a Keras Lambda layer which performs slicing of the portion of the input data that corresponds to profile information such that it can be fed to the convolutional layers. This slicing only works with Tensorflow. If only 0D signals are used, Theano is supported.
This pull request extends the ability of the code to deal with various sources of data. It introduces the abstractions of
SignalsandMachinesEach of these are objects that carry the relevant information necessary for incorporating them into the overall pipeline.
Signals know theMachinethey live on, their mds+ paths, code for being downloaded, preprocessing approaches, their dimensionality, etc.Machines know whichSignals are defined on them, which mds+ server houses the data, etc.This required changes in
Signal)As such, this adds the following functionality
This pull request also fixes various smaller and unrelated bugs that were uncovered.
1D profiles are currently not supported with the Theano backend, as there is a bug when constructing slicing layers in the model
builder. The implementation creates a KerasLambdalayer which performs slicing of the portion of the input data that corresponds to profile information such that it can be fed to the convolutional layers. This slicing only works with Tensorflow. If only 0D signals are used, Theano is supported.