aws / sagemaker-experiments

Experiment tracking and metric logging for Amazon SageMaker notebooks and model training.
Apache License 2.0
125 stars 36 forks source link

Training job as a trial component is not accessible/visible in Sagemaker Studio pane #136

Open pprabakaran opened 3 years ago

pprabakaran commented 3 years ago

Describe the bug mnist-handwritten-digits-classification-experiment.ipynb script is run but the training jobs which are also configured to be a trial component is not visible in the Sagemaker Studio pane.

search_expression = {
    "Filters": [
        {
            "Name": "DisplayName",
            "Operator": "Equals",
            "Value": "Training",
        }
    ],
}

trial_component_analytics = ExperimentAnalytics(
    sagemaker_session=Session(sess, sm),
    experiment_name=mnist_experiment.experiment_name,
    search_expression=search_expression,
    sort_by="metrics.test:accuracy.max",
    sort_order="Descending",
    metric_names=["test:accuracy"],
    parameter_names=["hidden_channels", "epochs", "dropout", "optimizer"],
)

trial_component_analytics.dataframe()

The above script gives no output

To Reproduce To run the script as is mnist-handwritten-digits-classification-experiment.ipynb using Kernel Python 3 (Data Science) in Sagemaker Studio.

Expected behavior To view training jobs as trial components. Only the preprocessing trial component are visible.

Screenshots If applicable, add screenshots to help explain your problem.

image

Environment: mnist-handwritten-digits-classification-experiment.ipynb (from https://docs.aws.amazon.com/sagemaker/latest/dg/experiments-mnist.html) ran as it is in Sagemaker studio

jlloyd-widen commented 2 years ago

I'm seeing this too. I thought it might be because I was running the job with instance_type='local' but from the above it would appear that's not likely the case.