aws / amazon-sagemaker-examples

Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.
https://sagemaker-examples.readthedocs.io
Apache License 2.0
10.04k stars 6.75k forks source link

How to specify the number of epoch using custom estimator.Tensorflow object? #659

Open ConvMech opened 5 years ago

ConvMech commented 5 years ago

Dear AWS team,

I am trying to train a custom tensoflow estimator with the . I am using pipemode dataset as the input.

I wish I can have an evaluation result at the end of each epoch during the training. Wondering what is the currect way to specify the hyper parameter epoch number?

from sagemaker.tensorflow import TensorFlow
import os
source_dir = os.path.join(os.getcwd(), 'source_dir')

cnn_estimator = TensorFlow(entry_point='estimator.py',
                        source_dir=source_dir,
                        role=role,
                        framework_version='1.12.0',
                        #checkpoint_path = model_artifacts_location,
                        input_mode='Pipe',
                        hyperparameters={'throttle_secs': 10, 'epoch':2},
                        output_path=model_artifacts_location,
                        code_location=custom_code_upload_location,
                        train_instance_count=1,
                        training_steps=None,
                        evaluation_steps=VALID_STEP,
                        train_instance_type='ml.c5.xlarge')

Method currently using inside the train_input_fn()

    ds = PipeModeDataset(channel)
    ds = ds.batch(2)
    ds = ds.map(parse) 
    ds = ds.batch(64)
    ds = ds.repeat(params['epoch'])

Thank you very much

danielkurniadi commented 4 years ago

I was wondering the same for tuning in the number of epochs. This should be very basic yet I need to search through beyong docs.