[BUG]Unable to import name 'OpenAIEmbedding' from 'synapse.ml.cognitive'

[jyravi]

SynapseML version

0.11.0

System information

• Language version (e.g. python 3.8, scala 2.12): python 3.8, scala 2.12.15
• Spark Version (e.g. 3.2.3): 3.2
• Spark Platform (e.g. Synapse, Databricks): Synapse

Describe the problem

I am trying to create OpenAI Embedding as shown in the notebook - CognitiveServices - OpenAI.ipynb.

Code to reproduce issue

from synapse.ml.cognitive import OpenAIEmbedding

embedding = ( OpenAIEmbedding() .setSubscriptionKey(key) .setDeploymentName(deployment_name_embeddings) .setUrl("https://{}.openai.azure.com/".format(service_name)) .setTextCol("combined") .setErrorCol("error") .setOutputCol("embeddings") )

Other info / logs

No response

What component(s) does this bug affect?

• [X] area/cognitive: Cognitive project
• [ ] area/core: Core project
• [ ] area/deep-learning: DeepLearning project
• [ ] area/lightgbm: Lightgbm project
• [ ] area/opencv: Opencv project
• [ ] area/vw: VW project
• [ ] area/website: Website
• [ ] area/build: Project build system
• [ ] area/notebooks: Samples under notebooks folder
• [ ] area/docker: Docker usage
• [ ] area/models: models related issue

What language(s) does this bug affect?

• [ ] language/scala: Scala source code
• [X] language/python: Pyspark APIs
• [ ] language/r: R APIs
• [ ] language/csharp: .NET APIs
• [ ] language/new: Proposals for new client languages

What integration(s) does this bug affect?

• [X] integrations/synapse: Azure Synapse integrations
• [ ] integrations/azureml: Azure ML integrations
• [ ] integrations/databricks: Databricks integrations

[github-actions[bot]]

Hey @jyravi :wave:! Thank you so much for reporting the issue/feature request :rotating_light:. Someone from SynapseML Team will be looking to triage this issue soon. We appreciate your patience.

[serena-ruan]

Hi @jyravi Could you please double check if your SynpaseML version is v0.11.0? You could check by running:

import synapse.ml.core
synapse.ml.core.__spark_package_version__

[rabbyn]

Hello @serena-ruan , I'm facing the same error while using a fresh Synapse Spark Cluster I just deployed. By running the command you gave to @jyravi I got '0.10.1-69-84f5b579-SNAPSHOT' . So it seems that the SynapseML version currently deployed on Synapse Resource is not up-to-date to fully use Azure Open AI 👎

Support request ID | 2304070050000707

[anoopsr1]

This seems to be still a problem "ImportError: cannot import name 'OpenAIEmbedding' from 'synapse.ml.cognitive' (/home/trusted-service-user/cluster-env/env/lib/python3.10/site-packages/synapse/ml/cognitive/init.py)"