Closed gos-qs closed 2 years ago
We recommend using smart-open for this, for example:
with patch("builtins.open", smart_open.open):
stream = trading.streaming.create_historical_generator_stream(file_path='test.bz2')
..
Ok thanks, that package looks very useful for loads of my other stuff, is it worth adding a mention to the docs or examplestreaminghistorical.py ?
💯 I will add something now
For my use it is useful to be able to read the historical bz2 files directly into the historical stream generator rather than having to uncompress a file back to disk and then re-read it back in as raw text, and I couldn't find any existing way to do that, so I added HistoricalGeneratorStreamBz2 that does. Is this useful to anyone else and to include in the original? and if so, is the way i've implemented it ok? I figured the alternative would be to put the if-else in HistoricalGeneratorStream._read_loop, but it looked more straightforward to include a manual parameter, compression: str, to indicate files compressed but without an extension in the former method.
modified: betfairlightweight/endpoints/streaming.py modified: betfairlightweight/streaming/init.py modified: betfairlightweight/streaming/betfairstream.py
Thanks