ubclaunchpad / minutes

:telescope: Speaker diarization via transfer learning
https://medium.com/ubc-launch-pad-software-engineering-blog/speaker-diarisation-using-transfer-learning-47ca1a1226f4
27 stars 5 forks source link

Customizing Base Models #125

Open chadlagore opened 6 years ago

chadlagore commented 6 years ago

Customizing a BaseModel is pretty simple at the moment. Something like:

from minutes import BaseModel

class MyBaseMinutesModel(BaseModel):
    def fit(self, **kwargs):
        X_train, X_test, y_train, y_test = self._generate_training_data()
        # Design and train your model, then assign it to self.model.

I don't think we need to make this anymore convenient for the user, but some documentation for this should exist in the README.

chadlagore commented 6 years ago

I'm going to flesh out this approach a bit better to show how you customize a base model, then version control it here in the repository for others to use. Below is untested code, you may need to tweak it, but its correct in spirit to how Minutes should behave.

import os

from keras.models import Sequential

from minutes.base import BaseModel
from minutes import Speaker

class MyBaseMinutesModel(BaseModel):
    """Subclass the BaseModel and override the fit method 
    (thats all you need to do).
    """

    def fit(self, **kwargs):
        X_train, X_test, y_train, y_test = self._generate_training_data()
        # Design and train your model, then assign it to self.model.
        self.model = Sequential([
              # ...
        ])
        self.model.compile()
        self.model.fit()

# Define hyperparameters (there is one at the moment and it determines
# the type of data generated by `_generate_training_data`.
new_base = MyBaseMinutesModel(
    'human-readable-name',
    ms_per_observation=1000
)

# Add many speakers.
for folder in os.listdir('/path/to/speakers/'):
    # Assuming folder is speakers name.
    s = Speaker(folder) 

    # Assuming folder contains audio for speaker.
    s.add_audio('/path/to/speakers/' + folder)

    new_base.add_speaker(s)

new_base.fit(verbose=2)    # Takes a while :)
new_base.save()

Note that new_base.save() will save the fixed model parameters and keras model to the location specified by MINUTES_MODELS_DIRECTORY environment variable. If MINUTES_MODELS_DIRECTORY environment variable is not set by the user, this will default into the minutes repository /minutes/models/human-readable-name. Then its as simple as:

git checkout -b new-model-type
git add minutes/models/*
git commit -m "My new model got 99%!"
git push origin HEAD

We have work to do to standardize the naming of the output models. Currently it uses the name specified by the user, but it should have some other features to make it unique/informative/serialized.