search

Azure / azureml-examples

Official community-driven Azure Machine Learning examples, tested with GitHub Actions.
https://docs.microsoft.com/azure/machine-learning
MIT License
1.75k stars 1.43k forks source link

Custom monitoring signal examples not working #2741

Open TF518 opened 1 year ago

TF518 commented 1 year ago

Operating System

Windows

Version Information

azure-cli: 2.53.0 azure-cli-core: 2.53.0 azure-cli-telemetry: 1.1.0 extensions: ml: 2.21.0

Steps to reproduce

  1. Follow instruction for 5. Create your own custom monitoring signal in /cli/monitoring

Expected behavior

Expected: A custom signal monitor is created and viewable from the Azure Machine Learning workspace monitoring dashboard

Actual behavior

CLI returns Met error <class 'AttributeError'>:'NoneType' object has no attribute 'values'

Addition information

Adding an inputs field to custom_monitoring.yaml changes the error message to Met error <class 'NotImplementedError'>:

# custom-monitoring.yaml
$schema: http://azureml/sdk-2-0/Schedule.json
name: my-custom-signal
trigger:
  type: recurrence
  frequency: day # can be minute, hour, day, week, month
  interval: 7 # #every day
create_monitor:
  compute:
    instance_type: "standard_e8s_v3"
    runtime_version: "3.2"
  monitoring_signals:
    customSignal:
      type: custom
      component_id: azureml:my_custom_signal:1.0.0
      inputs: # this is new 
        input_1:
          input_data:
            type: mltable
            path: azureml:Direct:1
          data_context: test
      input_data:
        test_data_1:
          input_data:
            type: mltable
            path: azureml:Direct:1
          data_context: test
        test_data_2:
          input_data:
            type: mltable
            path: azureml:Direct:1
          data_context: test
          data_window:
            trailing_window_size: P30D
            trailing_window_offset: P7D
          pre_processing_component: azureml:custom_preprocessor:1.0.0
      metric_thresholds:
        - metric_name: std_dev
          threshold: 2
  alert_notification:
    emails:
      - abc@example.com
sramayanam commented 1 year ago

I am getting the same error while trying to implement advanced model monitoring and same is true with basic. Can someone kindly advise Class MonitoringTarget: This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information. Met error <class 'NotImplementedError'>: