Callback function for Keras

[yakigac]

This might be a pull-request than an issue, but I don't have any idea where to place this code. Please teach me the place or you can just close this issue.

This is a callback function of hyperdash for keras. It should be saved as hyperdash_callback.py in this case.

from keras.callbacks import Callback

class Hyperdash(Callback):
    def __init__(self, exp):
        super(Hyperdash, self).__init__()
        self.exp = exp

    def on_epoch_end(self, epoch, logs=None):
        val_acc = logs.get('val_acc')
        val_loss = logs.get('val_loss')

        if val_acc is not None:
            self.exp.metric("val_acc", val_acc)
        if val_loss is not None:
            self.exp.metric("val_loss", val_loss)

and use it like the following,

# other modules ...
from hyperdash import Experiment
from hyperdash_callback import Hyperdash

def main():
    exp = Experiment("Experiment 1")

    # prepare your train data and models here
    # ...

    hd_callback = Hyperdash(exp=exp)
    callbacks = [hd_callback] # it can be used with any other callbacks for keras
    history = model.fit(x_train, y_train, batch_size=32, verbose=1, epochs=50,
                        validation_split=0.1, shuffle=True,
                        callbacks=callbacks)

    exp.end()

if __name__ == "__main__":
    main()

[richardartoul]

Hey @yakigac sorry about the delay in getting back to you, this looks pretty cool!

We've been wanting to incorporate this feature for while so thanks for taking the first stab at it. Does the callback have to be explicitly passed into the model.fit() function for Keras? We've been debating if it would be possible to auto-register the callback.

If not this looks like a pretty solid API although I might rename the callback to something like

from hyperdash_callback import KerasCallback

keras_callback = KerasCallback(exp)

or something like that.

Do you want to build this into our SDK, or would you like me to do it? I'm happy either way.

Also @andrewschreiber what do you think?

[andrewschreiber]

Hey @yakigac thanks for taking the time to think this through! I think a keras callback is a solid idea. We might be able to build a callback subclass directly into the experiment object, for ease of use. Using your example:

# other modules ...
from hyperdash import Experiment

def main():
    exp = Experiment("Experiment 1")

    # prepare your train data and models here
    # ...

    callbacks = [exp.keras_callback] # it can be used with any other callbacks for keras
    history = model.fit(x_train, y_train, batch_size=32, verbose=1, epochs=50,
                        validation_split=0.1, shuffle=True,
                        callbacks=callbacks)

    exp.end()

if __name__ == "__main__":
    main()

What do you think? @richardartoul @yakigac

[yakigac]

@andrewschreiber I think your style is more useful than I wrote. But in this case, isn't it depending on keras.callbacks? I'm not sure but if so, is it acceptable?. (Please tell me if I wrong). Whether it is or not, I can make the contribution for it.

[yakigac]

I'm creating the feature here. But I don't enough time to test it now.

[richardartoul]

Cool this is being implemented in #113, you can track progress there

[richardartoul]

This feature was implemented in the latest version of Hyperdash. Documentation is still pending though