deepgram / deepgram-python-sdk

Official Python SDK for Deepgram's automated speech recognition APIs.
https://developers.deepgram.com
MIT License
179 stars 48 forks source link

`Object of type Sentiment is not JSON serializable` everytime Python parses the `sentiment=<Sentiment.UNKNOWN: ''>` #299

Closed roperi closed 4 months ago

roperi commented 4 months ago

What is the current behavior?

In Python, everytime i want to dump the response as dictionary (i.e. response.to_dict()) to a json file, I get a Object of type Sentiment is not JSON serializable error message.

I noticed that the response object always fails when trying to dump the 'sentiment' key with the value 'Sentiment.UNKNOWN''

Sentence(text="He's Korean.", start=127.06, end=127.88, sentiment=<Sentiment.UNKNOWN: ''>, sentiment_score=None)]

This forces me to resort to elaborate manouvers to avoid this error such as:

        if 'results' in response_dict and 'channels' in response_dict['results']:
            for channel in response_dict['results']['channels']:
                if 'alternatives' in channel:
                    for alternative in channel['alternatives']:
                        if 'words' in alternative:
                            for word in alternative['words']:
                                if 'sentiment' in word:
                                    sentiment_value = str(word['sentiment'])
                                    word['sentiment'] = sentiment_value
                        # Handle 'paragraphs'
                        if 'paragraphs' in alternative:
                            paragraphs = alternative['paragraphs']
                            # Handle 'sentiment' within 'paragraphs'
                            if 'sentiment' in paragraphs:
                                sentiment_value = str(paragraphs['sentiment'])
                                paragraphs['sentiment'] = sentiment_value
                            # Handle 'sentiment' within 'sentences' under 'paragraphs'
                            for paragraph in paragraphs.get('paragraphs', []):
                                for sentence in paragraph.get('sentences', []):
                                    if 'sentiment' in sentence:
                                        sentiment_value = str(sentence['sentiment'])
                                        sentence['sentiment'] = sentiment_value
                                if 'sentiment' in paragraph:
                                    sentiment_value = str(paragraph['sentiment'])
                                    paragraph['sentiment'] = sentiment_value

The checks above turn all UNKNWON sentiment values to a string and make my program work. But now that I added topics=True I'm once again getting the same Object of type Sentiment is not JSON serializable error.

Why is so hard to convert a Deepgram response to a dict and dump it to a json file in Python?

Steps to reproduce

response = deepgram.listen.prerecorded.v("1").transcribe_file(payload, options, timeout=300)
response_dict = response.to_dict()
with open(output_json, "w") as json_file:
    json.dump(response_dict, json_file, indent=4)

^ Fails with Object of type Sentiment is not JSON serializable for every Sentiment.UNKNOWN found in the response (there could be hundreds).

Expected behavior

Not failing with Object of type Sentiment is not JSON serializable. I don't know but sentiment value should be None or "" instead of the UNKNOWN object.

Please tell us about your environment

Debian 10 Python 3.10.12

Other information

This is just an excerpt of the response i'm getting (notice the sentiment value)

start=85.45, end=90.73, num_words=11.0, speaker=0, sentiment=<Sentiment.UNKNOWN: ''>, sentiment_score=0.0), Paragraph(sentences=[Sentence(text="We're Joan and Samantha.", start=90.73, end=92.35, sentiment=<Sentiment.UNKNOWN: ''>, sentiment_score=None)], start=90.73, end=92.35, num_words=4.0, speaker=0, sentiment=<Sentiment.UNKNOWN: ''>, sentiment_score=0.0), Paragraph(sentences=[Sentence(text='Are you French?', start=92.955, end=93.775, sentiment=<Sentiment.UNKNOWN: ''>, sentiment_score=None)], start=92.955, end=93.775, num_words=3.0, speaker=4, sentiment=<Sentiment.UNKNOWN: ''>, sentiment_score=0.0), Paragraph(sentences=[Sentence(text='No.', start=93.915, end=94.155, sentiment=<Sentiment.UNKNOWN: ''>, sentiment_score=None), Sentence(text="We're not from France.", start=94.155, end=95.195, sentiment=<Sentiment.UNKNOWN: ''>, sentiment_score=None), Sentence(text="We're from Belgium.", start=95.195, end=96.174995, sentiment=<Sentiment.UNKNOWN: ''>, sentiment_score=None), Sentence(text='Conversation 8.', start=100.21, end=101.43, sentiment=<Sentiment.UNKNOWN: ''>, sentiment_score=None), Sentence(text='Which country is Oleg from?', start=101.89, end=103.75, sentiment=<Sentiment.UNKNOWN: ''>, sentiment_score=None)], s
dvonthenen commented 4 months ago

This was reported earlier and it should be fixed in the latest release yesterday: https://github.com/deepgram/deepgram-python-sdk/issues/296

Can you verify it?

roperi commented 4 months ago

This was reported earlier and it should be fixed in the latest release yesterday: #296

Can you verify it?

Oops. Sorry for the duplicate. I swear i checked yesterday and couldn't find anything related (i guess i checked before it was reported).

I'll verify in a moment

dvonthenen commented 4 months ago

No worries. You probably (hopefully) grab the SDK just before this was published.

roperi commented 4 months ago

Hmmm. It seems I already have the latest release. pip install deepgram-sdk==3.* Requirement already satisfied: deepgram-sdk==3.* in /home/user/.virtualenvs/Deepgram/lib/python3.10/site-packages (3.1.3)

roperi commented 4 months ago

ok. I manually did a pip install deepgram-sdk==3.1.4 and it's working now! Thanks so much! Closing the issue

dvonthenen commented 4 months ago

all good. thanks for verifying this! good to know this is working/verified.