search

aws / sagemaker-spark

A Spark library for Amazon SageMaker.
https://aws.github.io/sagemaker-spark/
Apache License 2.0
298 stars 126 forks source link

Support new auto tuning #55

Open harthur opened 6 years ago

harthur commented 6 years ago

It looks like there's a nice Python API for "Automatic Model Tuning": https://aws.amazon.com/blogs/aws/sagemaker-automatic-model-tuning/. Would something like HyperparameterTuner be added to this Scala Spark library?

laurenyu commented 6 years ago

hi @harthur, thanks for your interest! Unfortunately, we're not sure at the moment if it's something that we'll add anytime soon, but we're always reprioritizing our backlog based on customer feedback.

sajjap commented 5 years ago

Hi @knakad, Thanks for a quick response. But this isn't related to auto model tuning. I was referring to something similar to the highlighted in below example from sagemaker-spark github page but with the XGBoostSageMakerEstimator

Instead of creating a KMeansSageMakerEstimator, you can create an equivalent SageMakerEstimator:

val estimator = new SageMakerEstimator( trainingImage = "382416733822.dkr.ecr.us-east-1.amazonaws.com/kmeans:1", modelImage = "382416733822.dkr.ecr.us-east-1.amazonaws.com/kmeans:1", requestRowSerializer = new ProtobufRequestRowSerializer(), responseRowDeserializer = new KMeansProtobufResponseRowDeserializer(), hyperParameters = Map("k" -> "10", "feature_dim" -> "784"), sagemakerRole = IAMRole(roleArn), trainingInstanceType = "ml.p2.xlarge", trainingInstanceCount = 1, endpointInstanceType = "ml.c4.xlarge", endpointInitialInstanceCount = 1, trainingSparkDataFormat = "sagemaker")