search

NOAA-OWP / DMOD

Distributed Model on Demand infrastructure for OWP's Model as a Service
Other
7 stars 15 forks source link

Look at replacing StringIO/BytesIO usage with simple Reader implementation #609

Open robertbartel opened 4 months ago

robertbartel commented 4 months ago

See here for original suggestion.

In short, consider moving to a custom implementation of the Reader protocol (or what Pandas would refer to as a "file-like object") in places where StringIO or BytesIO are used.

(Update: title, description, and first comment updated to include both StringIO and BytesIO.)

robertbartel commented 4 months ago

I am at least a bit curious the degree to which this is preferable to just using StringIO/BytesIO. Wouldn't mind a bit of discussion and references here if possible before we proceed with anything.

christophertubbs commented 4 months ago

This is one of those things that makes my head tilt like a dog. Why not use a BytesIO when it's a bytes object?

robertbartel commented 4 months ago

This is one of those things that makes my head tilt like a dog. Why not use a BytesIO when it's a bytes object?

Probably only because I wasn't thinking too hard about it, and the Pandas documentation for read_csv specifically cites StringIO as an example of a file-like. So it put a valid solution in my head, and I just went with it.

I probably should change that in the #566 review (and then update the issue title and description here; I think the general question probably still applies).

aaraney commented 4 months ago

Yeah, I concur, @robertbartel in regard to switching to BytesIO. I must have had a one tracked mind when reviewing! BytesIO didn't come to mind.