Closed jsteinberg-rbi closed 11 months ago
cc @eric-urban
Here's the code, literally a line-for-line copy of the code from the doc, except of course my deployment name is different:
poc-ada-002
and the env var names are adapted to what I use for all my other azure openai studio programs.
import os
import azure.cognitiveservices.speech as speechsdk
import openai
# This example requires environment variables named "OPEN_AI_KEY" and "OPEN_AI_ENDPOINT"
# Your endpoint should look like the following https://YOUR_OPEN_AI_RESOURCE_NAME.openai.azure.com/
openai.api_key = os.environ.get('AZURE_OPENAI_KEY')
openai.api_base = os.environ.get('AZURE_OPENAI_ENDPOINT')
openai.api_type = 'azure'
openai.api_version = '2022-12-01'
# This will correspond to the custom name you chose for your deployment when you deployed a model.
deployment_id='poc-ada-002'
# This example requires environment variables named "SPEECH_KEY" and "SPEECH_REGION"
speech_config = speechsdk.SpeechConfig(subscription=os.environ.get('AZURE_SUBSCRIPTION_KEY'), region='eastus')
audio_output_config = speechsdk.audio.AudioOutputConfig(use_default_speaker=True)
audio_config = speechsdk.audio.AudioConfig(use_default_microphone=True)
# Should be the locale for the speaker's language.
speech_config.speech_recognition_language="en-US"
speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config, audio_config=audio_config)
# The language of the voice that responds on behalf of Azure OpenAI.
speech_config.speech_synthesis_voice_name='en-US-JennyMultilingualNeural'
speech_synthesizer = speechsdk.SpeechSynthesizer(speech_config=speech_config, audio_config=audio_output_config)
# Prompts Azure OpenAI with a request and synthesizes the response.
def ask_openai(prompt):
# Ask Azure OpenAI
response = openai.Completion.create(engine=deployment_id, prompt=prompt, max_tokens=100)
text = response['choices'][0]['text'].replace('\n', ' ').replace(' .', '.').strip()
print('Azure OpenAI response:' + text)
# Azure text to speech output
speech_synthesis_result = speech_synthesizer.speak_text_async(text).get()
# Check result
if speech_synthesis_result.reason == speechsdk.ResultReason.SynthesizingAudioCompleted:
print("Speech synthesized to speaker for text [{}]".format(text))
elif speech_synthesis_result.reason == speechsdk.ResultReason.Canceled:
cancellation_details = speech_synthesis_result.cancellation_details
print("Speech synthesis canceled: {}".format(cancellation_details.reason))
if cancellation_details.reason == speechsdk.CancellationReason.Error:
print("Error details: {}".format(cancellation_details.error_details))
# Continuously listens for speech input to recognize and send as text to Azure OpenAI
def chat_with_open_ai():
while True:
print("Azure OpenAI is listening. Say 'Stop' or press Ctrl-Z to end the conversation.")
try:
# Get audio from the microphone and then send it to the TTS service.
speech_recognition_result = speech_recognizer.recognize_once_async().get()
# If speech is recognized, send it to Azure OpenAI and listen for the response.
if speech_recognition_result.reason == speechsdk.ResultReason.RecognizedSpeech:
if speech_recognition_result.text == "Stop.":
print("Conversation ended.")
break
print("Recognized speech: {}".format(speech_recognition_result.text))
ask_openai(speech_recognition_result.text)
elif speech_recognition_result.reason == speechsdk.ResultReason.NoMatch:
print("No speech could be recognized: {}".format(speech_recognition_result.no_match_details))
break
elif speech_recognition_result.reason == speechsdk.ResultReason.Canceled:
cancellation_details = speech_recognition_result.cancellation_details
print("Speech Recognition canceled: {}".format(cancellation_details.reason))
if cancellation_details.reason == speechsdk.CancellationReason.Error:
print("Error details: {}".format(cancellation_details.error_details))
except EOFError:
break
# Main
try:
chat_with_open_ai()
except Exception as err:
print("Encountered exception. {}".format(err))
@jsteinberg-rbi Thanks for your feedback! We will investigate and update as appropriate.
I have now tried this without changing anything except the deployment name. I am not confused about what the deployment name should be. I go to azure openai studio and get the deployment name from the console and assign it to the deployment_name
variable: pretty straightforward. But it still does not work.
Also I should've added: the transcription is working:
Azure OpenAI is listening. Say 'Stop' or press Ctrl-Z to end the conversation.
Recognized speech: Hey, let me get a cheeseburger and a Coca-Cola and a large fry stop.
Encountered exception. The completion operation does not work with the specified model, text-embedding-ada-002. Please choose different model and try again. You can learn more about which models can be used with each operation here: https://go.microsoft.com/fwlink/?linkid=2197993.
so I believe that confirms that my:
SPEECH_KEY=<redacted>
SPEECH_REGION=eastus
env vars are working just fine. the issue seems to be specifically against the azure openai service. like I said I have a gpt4 model deployed working in another program right now that is doing transcription and then llm analysis and it works great.
Hi @eric-urban Could you please review add comments on this, update as appropriate.
@jsteinberg-rbi Thank you for bringing this to our attention. I've delegated this to content author @eric-urban, who will review it and offer their insightful opinions.
@jsteinberg-rbi - The sample won't work with embedding models. Please try a completions model such as GPT 3.5 Turbo. We can update the documentation to clarify. I hope this helps!
In Azure OpenAI Studio when I go to create a deployment I do not have text-davinci-002 as a base model deployment choice, only gpt's and ada-002. I assume, from having reviewed OpenAI's documentation, that ada-002 should not only work, but has permanently replaced davinci-002 in Azure OpenAI Studio? So how do I either get ada-002 to work or get davinci-002 as a base model? Thanks!
I have copied the source code as instructed directly from the document below. The code runs fine and transcribes my speech to text and seems to be getting my various environment variables just fine. I also have another deployed gpt35 resource up and that whole program I wrote is working great, my point being I don't seem to be struggling to use the various resources, so I'm not sure what to do about the above.
The issue is occurring with this tutorial in case that isn't obvious from the metadata below:
https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/openai-speech?context=%2Fazure%2Fcognitive-services%2Fopenai%2Fcontext%2Fcontext&tabs=macos&pivots=programming-language-python
Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.