MKuranowski / aiocsv

Python: Asynchronous CSV reading/writing
https://pypi.org/project/aiocsv/
MIT License
67 stars 9 forks source link

Support for non text mode buffers #2

Closed antoniovazquezblanco closed 3 years ago

antoniovazquezblanco commented 3 years ago

I am reading a CSV from an aiohttp request. I would love to use this async library with request.content as a buffer but it only returns bytes. It would be awesome to get support to decode the response from the request.

Thank you!

MKuranowski commented 3 years ago

aiocsv is built on top of Python's built-in csv module, which doesn't support operating on bytes. Really, I meant aiocsv to only be a buffer between asynchronous file-like objects and the synchronous csv module; data is passed as-is.

E.g. from csv.reader docs:

csvfile can be any object which supports the iterator protocol and returns a string each time its __next__() method is called

I don't want to add capabilities beyond what's offered by Python's built-in csv module, hence I'm closing this issue.


In synchronous code you can wrap binary streams in io.TextIOWrapper. Such class isn't available for asynchronous code, but it's not hard to write a simple one yourself with the help of codecs.IncrementalDecoder:

class AsyncTextReaderWrapper:
    def __init__(self, obj, encoding, errors="strict"):
        self.obj = obj

        decoder_factory = codecs.getincrementaldecoder(encoding)
        self.decoder = decoder_factory(errors)

    async def read(self, size):
        raw_data = await self.obj.read(size)

        if not raw_data:
            return self.decoder.decode(b"", final=True)

        return self.decoder.decode(raw_data, final=False)
antoniovazquezblanco commented 3 years ago

Thank you!