LUCIT-Systems-and-Development / unicorn-binance-websocket-api

A Python SDK by LUCIT to use the Binance Websocket API`s (com+testnet, com-margin+testnet, com-isolated_margin+testnet, com-futures+testnet, com-coin_futures, us, tr, dex/chain+testnet) in a simple, fast, flexible, robust and fully-featured way.
https://unicorn-binance-websocket-api.docs.lucit.tech/
Other
680 stars 165 forks source link

Is there any method that can give us LIFO data from stream? #155

Closed JayDarji closed 2 years ago

JayDarji commented 3 years ago

Like, binance_websocket_api_manager.pop_stream_data_from_stream_buffer

From this method, we can get data of oldest stream with FIFO structure.

I only need last data from the stream every time, Is there any method that can give me only last data and deletes all older stream data? So the problem behind using this pop_stream_data_from_stream_buffer method, is I'm subscribing 80 streams in on thread and it get delayed of almost 1 hour to get latest data.

It keeps on popping out oldest data from this method.

cryptodali commented 3 years ago

Basically if I get your requirement, the idea is to get real-time data in the stream for quick processing rather than letting the pop_stream write and use FIFO b/c if the writing is slow in FIFO and you try to access it, you'll get old data.

Is there any way to make the buffer to almost 0 so we the pop_stream always contain only the latest data?

oliver-zehentleitner commented 3 years ago

I am going to implement a new stream_buffer handling, it should be easy to add an option to turn it to a LIFO stack: https://github.com/oliver-zehentleitner/unicorn-binance-websocket-api/issues/138

It should also be very less work to clean the stream_buffer before adding a new entry, but then you would need to create a separate streambuffer for each currency or you would lose a lot of good data...

oliver-zehentleitner commented 2 years ago

Released: https://github.com/oliver-zehentleitner/unicorn-binance-websocket-api/releases/tag/1.32.0