search

adafruit / Adafruit_CircuitPython_Requests

Requests-like interface for web interfacing
MIT License
51 stars 36 forks source link

Need supported way to "steal" socket from requests library for mp3 streaming #198

Open jepler opened 2 days ago

jepler commented 2 days ago

Right now I do this:


def get_mp3_stream():
    if STREAMING_URL.startswith("http:") or STREAMING_URL.startswith("https:"):
        return requests.get(STREAMING_URL, headers={"connection": "close"}).socket
    return open(STREAMING_URL, "rb")

but

The real requests library has a public raw property that calling code may use when the request includes stream=True. So simply renaming the socket property to raw would be one step in the right direction. (the stream argument is already supported in adafruit_requests)

        #: File-like object representation of response (for advanced usage).
        #: Use of ``raw`` requires that ``stream=True`` be set on the request.
        #: This requirement does not apply for use internally to Requests.
        self.raw = None

As for properly releasing the socket, I guess I need to restructure my code so it can hold onto the requests object as long as needed.

justmobilize commented 2 days ago

Isn't the raw object a urllib3.response.HTTPResponse? So not really the socket.

In json() we set:

self._raw = _RawResponse(self)

And _RawResponse has read. Could we do something similar if it's a stream?

jepler commented 2 days ago

It is a HTTPResponse object in standard Python, but it supports the read method (and not recv):

>>> response = requests.get("http://example.com", headers={'connection': 'close', 'accept-encoding': ''}, stream=True)
>>> response.raw
<urllib3.response.HTTPResponse object at 0x7f23ee80be50>
>>> response.raw.read(32)
b'<!doctype html>\n<html>\n<head>\n  '
>>> response.raw.recv
AttributeError: 'HTTPResponse' object has no attribute 'recv'

In CicuitPython a socket object supports recv and not read at the Python level, so the API still doesn't match. Small consolation then that for "my code" (MP3Decoder which uses C calls that work for a built in file or for a built in socket) doesn't care.

and I don't want to add a type to Requests that wraps a socket to provide read in terms of recv. For my purposes, an intermediate object, especially one coded in Python code, is undesirable, because dropping back into arbitrary Python code from a background task causes problems and we avoid it whenever possible.

jepler commented 2 days ago

if we just decide to document the socket property as a CircuitPython extension that's fine by me.

justmobilize commented 2 days ago

I understand not wanting to pass the wrapper to have read.

Although for CircuitPython, returning the socket as raw would break anything written in python...

justmobilize commented 2 days ago

Also, any desire to do this the standard way:

socket.fromfd(response.raw.fileno(), socket.AF_INET, socket.SOCK_STREAM)

I know fileno isn't currently supported...

dhalbert commented 2 days ago

This FAQ, https://requests.readthedocs.io/en/latest/community/faq/#encoded-data, says

You can get direct access to the raw response (and even the socket), if needed as well.

Is socket.fromfd() what they mean?

justmobilize commented 2 days ago

socket.fromfd() is what I've used in CPython.

jepler commented 2 days ago

file numbers really aren't a thing in circuitpython, though