StreamSource is broken in latest SDK (3.1.5)

[mirceapasoi]

What is the current behavior?

StreamSource (previously called ReadStreamSource) doesn't seem to work properly.

Steps to reproduce

from deepgram import StreamSource
import io
src = StreamSource(stream=io.BufferedReader(raw=open("README.md", "r")))
"stream" in src
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[24], line 1
----> 1 "stream" in src

File ~/Development/lifestory/.venv/lib/python3.11/site-packages/deepgram/clients/analyze/v1/options.py:97, in StreamSource.__getitem__(self, key)
     96 def __getitem__(self, key):
---> 97     _dict = self.to_dict()
     98     return _dict[key]

File ~/Development/lifestory/.venv/lib/python3.11/site-packages/dataclasses_json/api.py:73, in DataClassJsonMixin.to_dict(self, encode_json)
     72 def to_dict(self, encode_json=False) -> Dict[str, Json]:
---> 73     return _asdict(self, encode_json=encode_json)

File ~/Development/lifestory/.venv/lib/python3.11/site-packages/dataclasses_json/core.py:414, in _asdict(obj, encode_json)
    412         value = getattr(obj, field.name)
    413     else:
--> 414         value = _asdict(
    415             getattr(obj, field.name),
    416             encode_json=encode_json
    417         )
    418     result.append((field.name, value))
    420 result = _handle_undefined_parameters_safe(cls=obj, kvs=dict(result),
    421                                            usage="to")

File ~/Development/lifestory/.venv/lib/python3.11/site-packages/dataclasses_json/core.py:432, in _asdict(obj, encode_json)
    430     return list(_asdict(v, encode_json=encode_json) for v in obj)
    431 else:
--> 432     return copy.deepcopy(obj)

File /opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py:161, in deepcopy(x, memo, _nil)
    159 reductor = getattr(x, "__reduce_ex__", None)
    160 if reductor is not None:
--> 161     rv = reductor(4)
    162 else:
    163     reductor = getattr(x, "__reduce__", None)

TypeError: cannot pickle '_io.BufferedReader' object

Expected behavior

It should work to instantiate StreamSource with an io.BufferedReader (and it does in older versions of the SDK).

Please tell us about your environment

Other information

I believe this is the change that broke it: https://github.com/deepgram/deepgram-python-sdk/commit/450211ee6618309382bde5c7a9982bb1f77aecac#diff-33563c4bf51563ef7c839d34ac0037cf2cff20456e65bcead1fda7f002696435

[davidvonthenen]

Taking a look at this.

[davidvonthenen]

The @dataclass was an experiment that I forgot about and unintentionally commited, but the older input should still work just like before.

This should still work:

# Copyright 2023 Deepgram SDK contributors. All Rights Reserved.
# Use of this source code is governed by a MIT license that can be found in the LICENSE file.
# SPDX-License-Identifier: MIT

import os
from dotenv import load_dotenv
import logging, verboselogs
from datetime import datetime, timedelta
from io import BufferedReader
from deepgram import DeepgramClientOptions
import logging

from deepgram import (
    DeepgramClient,
    DeepgramClientOptions,
    StreamSource,
    AnalyzeOptions,
    TextSource,
)

load_dotenv()

TEXT_FILE = "conversation.txt"

def main():
    try:
        # STEP 1 Create a Deepgram client using the API key in the environment variables
        config = DeepgramClientOptions(
            verbose=logging.SPAM,
        )
        deepgram = DeepgramClient("", config)

        # OR use defaults
        # deepgram = DeepgramClient()

        # STEP 2 Call the transcribe_file method on the prerecorded class
        stream = open(TEXT_FILE, "rb")

        payload: StreamSource = {
            "stream": stream,
        }

        options = AnalyzeOptions(
            language="en",
            intents=True,
        )

        response = deepgram.read.analyze.v("1").analyze_text(payload, options)
        print(response.to_json(indent=4))

    except Exception as e:
        print(f"Exception: {e}")

if __name__ == "__main__":
    main()

If the dataclass thing doesn't panout, I will revert the name back to ReadStreamSource and convert back to the TypedDict. Will have this done tomorrow.

[davidvonthenen]

@mirceapasoi I tested the above code with 3.1.6, it should work just fine.

[mirceapasoi]

@dvonthenen I am using transcribe_file, which seems to crash around here, due to the "key" in provided_source check calling __getitem__:

   if is_buffer_source(source):
            body = source["buffer"]
        elif is_readstream_source(source):
            body = source["stream"]
...
def is_buffer_source(provided_source: PrerecordedSource) -> bool:
    return "buffer" in provided_source

def is_readstream_source(provided_source: PrerecordedSource) -> bool:
    return "stream" in provided_source

[davidvonthenen]

I reverted it.... GitHub is dying this morning. Waiting for the outage-ish to end so I can post a new release. Closing since the update has been merged.