search

barisozmen / deepaugment

Discover augmentation strategies tailored for your dataset
MIT License
247 stars 41 forks source link

Using deepaugment with large custom dataset (using generator)? #26

Open nvinhphuc opened 5 years ago

nvinhphuc commented 5 years ago

According to my observation, I don't see deepaugment support big dataset (which I cannot load all images and labels at a time and have to use data generator)? If I'm missing something, can you show how to use this repo with custom dataset which I have to use data generator?

JordanMakesMaps commented 5 years ago

According to my observation, I don't see deepaugment support big dataset (which I cannot load all images and labels at a time and have to use data generator)? If I'm missing something, can you show how to use this repo with custom dataset which I have to use data generator?

Look at fit_with_generator() under childnn.py:

def fit_with_generator( self, datagen, X_val, y_val, train_data_size, epochs=None, csv_logger=None ):

    record = self.model.fit_generator(
        datagen,
        validation_data=(X_val, y_val),
        steps_per_epoch=train_data_size // self.config["child_batch_size"],
        epochs=epochs,
        shuffle=True,
        verbose=2,
        callbacks=[csv_logger],
    )
    return record.history
balajiselvaraj1601 commented 5 years ago

@JordanMakesMaps @barisozmen

Can u provide some tutorial to use Deepaugment with large dataset. I am still stuck.

Thanks in advance