Azure / mlops-v2

Azure MLOps (v2) solution accelerators. Enterprise ready templates to deploy your machine learning models on the Azure Platform.
https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-management-and-deployment
MIT License
478 stars 225 forks source link

Getting error while creating Responsible Ai dashboard #67

Closed yogidosalwar closed 1 year ago

yogidosalwar commented 1 year ago

I am trying to use different algorithms for training. Model training and model registration part is successfully completed, but RAI insights dashboard constructor is failed. I am using the xgboost model for training.

Traceback (most recent call last):
  File "create_rai_insights.py", line 184, in <module>
    main(args)
  File "create_rai_insights.py", line 128, in main
    model_estimator = load_mlflow_model(my_run.experiment.workspace, model_id=model_id)
  File "/mnt/azureml/cr/j/exe/wd/rai_component_utilities.py", line 79, in load_mlflow_model
    return mlflow.pyfunc.load_model(model_uri)._model_impl
  File "/azureml-envs/responsibleai-0.18/lib/python3.8/site-packages/mlflow/pyfunc/__init__.py", line 484, in load_model
    model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
  File "/azureml-envs/responsibleai-0.18/lib/python3.8/site-packages/mlflow/sklearn/__init__.py", line 494, in _load_pyfunc
    return _load_model_from_local_file(path=path, serialization_format=serialization_format)
  File "/azureml-envs/responsibleai-0.18/lib/python3.8/site-packages/mlflow/sklearn/__init__.py", line 452, in _load_model_from_local_file
    return cloudpickle.load(f)
ModuleNotFoundError: No module named 'xgboost'

Thank You

setuc commented 1 year ago

Can you share which environment are you using? It's possible that the environment doesn't contain xgbosst. Consider adding that package as part of conda.yaml @yogidosalwar

yogidosalwar commented 1 year ago

@setuc Here is my conda.yaml . Thank you for quick response.

artifact

Here, i am stuck

RAI
setuc commented 1 year ago

Do you have your pipeline? Or did you just replace the model with XGBoost?

yogidosalwar commented 1 year ago

@setuc just i replace the model with XGBoost and added model dependencies in requirment.txt , train-conda.yml and train-monitor-conda.yml.

yogidosalwar commented 1 year ago

@setuc here is screen shot of DevOps

devops

Thank you.

setuc commented 1 year ago

Ahh. Got it. The responsbileAi modules user the environment that's different than the base container. It refers to the container from the Rai-vnext-Preview code. You can check the Rai component yaml file to see how the environment file is being built.

yogidosalwar commented 1 year ago

Ok. I will check. Thank you.

yogidosalwar commented 1 year ago

@setuc I went through the Rai-component.yml file and I made changes in requirment.txt which is part of Rai-vnext-Preview. But still, getting error in the Run Azure ML pipeline. If you have any input, please let me know.

Devops1

Thank you.

cindyweng commented 1 year ago

Can you see what's going on in the AML pipeline? Via the studio

yogidosalwar commented 1 year ago

@cindyweng @setuc I have RAI-vNext-Preview repo and MLOps-Test folder on my github. I made some changes as per my requirement in deploy-model-training-pipeline. yml, prep. py etc. and as well as I did changes in create_rai_insights.py. While running deploy-model-training-pipeline getting error while creating RAI Insights Dashboard Constructor. I went through the azure machine learning studio jobs and observe the code, but my changes are not showing there.

i did changes here parser.add_argument("--maximum_rows_for_test_dataset", type=int, default=5000) to parser.add_argument("--maximum_rows_for_test_dataset", type=int, default=10000)

responsibleai

below changes i did in deploy-model-training-pipeline.yml

    - repository: RAI-vNext-Preview  # Template Repo  #rai-vnext-preview
      name: yogidosalwar/RAI-vNext-Preview # need to change org name from "Azure" to your own org #rai-vnext-preview
      endpoint: github-connection # need to set up and hardcode
      type: github
      ref: main  

Thank you

yogidosalwar commented 1 year ago

@setuc @cindyweng Hi any update on above issue or suggestion??? Thank you