search

rosspalmer / bitQuant

Bitcoin data gathering, backtesting, and algorithmic trading solution
MIT License
23 stars 9 forks source link

Using since-parameter in subsequent cron calls #13

Closed multidis closed 9 years ago

multidis commented 9 years ago

After the issues are sorted out with request timing limits, it would be good if every trades-ping within a cron cycle queries only trades since the last timestamp of the previous ping. Most exchanges support that. Otherwise, redundant data is transferred when e.g. 5000-trades bitfinex ping is repeated within 10 sec and only a few new trades appear while a full 5000-block is still transferred.

rosspalmer commented 9 years ago

This is definitely a good idea but first I am going to look into to using the WebSocket API for live trade data. I have been doing some research and it is a much more efficient method since data packets are sent for each trade and don't require constant "pings". A few exchanges do not have WebSocket support as of yet so this change will be needed.

multidis commented 9 years ago

Indeed WebSockets are more efficient however the most important (volume-wise) exchange Bitfinex does not offer that.

rosspalmer commented 9 years ago

Added the "since limiter" feature in this commit. Some of the exchanges use a since that is based on trade id while others use timestamp. stype has been added to the API cmd dictionary to indicate which types of exchanges are which.

multidis commented 9 years ago

Tested - seems to work as expected. This is a helpful improvement, much less data is transferred in subsequent calls (in particular with bitfinex). Thanks!