QA Notebooks in Synapse

microsoft / Azure-Analytics-and-AI-Engagement

Azure Analytics and AI Engagement

Other

140 stars 120 forks source link

QA Notebooks in Synapse #76

Closed daronyondem closed 3 years ago

daronyondem commented 4 years ago

Here is the list of notebooks used in Synapse, that are found in the DEMO video.

MFG_WITHOUT_HTAP
MFG_WITH_HTAP
8a MFG Cosmos...
1 Prepare data for Predictive Maintenance in Synapse
3 Prepare Data for Product Anomaly Detection in Synapse

xor-ranavikrant commented 4 years ago

QA done for MFG_WITHOUT_HTAP and MFG_WITH_HTAP notebooks.
8a MFG Cosmos is not present in synapse prod environment so not moved into packaging yet. Might be present in ML studio.

daronyondem commented 4 years ago

Pending QA for the notebooks listed here;

Assuming these are created by @sanjaydatasciencedojo Team (@tshrivasdsd might be the one who has to look into this)

1 Prepare data for Predictive Maintenance in Synapse
3 Prepare Data for Product Anomaly Detection in Synapse

The QA has to run on the environment that @xor-ranavikrant owns.

daronyondem commented 4 years ago

@RonStoehr Can you verify who has ownership for 8a MFG Cosmos notebook? This is used in the DEMO video, but not available in the current environment.

RonStoehr commented 4 years ago

@RonStoehr Can you verify who has ownership for 8a MFG Cosmos notebook? This is used in the DEMO video, but not available in the current environment.

Sent an email to Abhishek to check ownership.

xor-ranavikrant commented 4 years ago

Had a discussion with Tarun and Sanjay. Have verified the dependencies for the remaining 2 notebooks. They require a connection with Storage account which is already in place. The things that are missing are assets to be included in storage account. They are as follows:

machines.csv
errors.csv
maint.csv
failures.csv
telemetryp.csv (This file is 758 mb so cannot go on repository)

tshrivasdsd commented 4 years ago

We had a discussion with Rana and trust, we have been able to help him understand the dependencies on Azure Synapse MLNotebooks, Azure ML Studio Notebooks, Azure Custom Vision, Azure Cognitive Search, Azure Form Recognizer, Azure Translator, ADLS Gen2 Storages, and AKS.

RonStoehr commented 4 years ago

@RonStoehr Can you verify who has ownership for 8a MFG Cosmos notebook? This is used in the DEMO video, but not available in the current environment.

Sent an email to Abhishek to check ownership.

Abhisheck thinks it may be Click 2 Cloud, sent an email.

daronyondem commented 4 years ago

@xor-ranavikrant Once we have all dependencies in place don't we need a final round of QA for notebooks?

The 8a MFG Cosmos notebook was shown in the DEMO video, but we can't find it. Out of scope.

daronyondem commented 4 years ago

Here is the link for the file from the solliance subscription https://solliancepublicdata.blob.core.windows.net/cdp/manufacturing-csv/telemetryp.csv

xor-ranavikrant commented 4 years ago

Getting the following error while running the notebooks:1 Prepare data for Predictive Maintenance in Synapse and 2 Prepare data for anomaly in Synapse AnalysisException : 'java.lang.RuntimeException: Operation failed: "This request is not authorized to perform this operation using this permission.", 403, HEAD, https://dreamdemostrggen2r16gxwb.dfs.core.windows.net/tempdata/tmp/hive?upn=false&action=getStatus&timeout=90;' @tshrivasdsd could you suggest what the problem could be.

sanjaydatasciencedojo commented 4 years ago

@xor-ranavikrant It seems like a permission issue. We will need to provide "Storage Blob Contributor" access to that user for that storage.

daronyondem commented 4 years ago

Decision from meeting 8/25 9PM UTC

These four notebooks will be uploaded as they are with no modification.