tudorelu
commented
4 years ago Hi @twopirllc, this sounds interesting I'll have a look!
Closed twopirllc closed 3 years ago
tudorelu
commented
4 years ago Hi @twopirllc, this sounds interesting I'll have a look!
twopirllc
commented
4 years ago @tudorelu,
If you have any questions, let me know.
KJ
tudorelu
commented
4 years ago Definitely - just had a thought, wanted to see your take on it... you know with real time trading sometimes you want to compute an indicator value only for the last Candle for example - because you already computed it for the previous ones. How many indicators in your library would have that capability, or should most of them be rewritten to allow for that? (I imagine a lot of devs would like to have this ability in a TA library..)
twopirllc
commented
4 years ago @tudorelu,
Excellent question.
I imagine a lot of devs would like to have this ability in a TA library..
No doubt!
Yet, in my experience of both creating indicators and reverse engineering them from TA Lib, Trading View, et al, it is not possible for many indicators.
In fact it was a discussion on my repo as well. It's the first Pinned Issue: How to utilize this Framework for live ticks #87. Unfortunately, I haven't had much chance to personally address this issue more. Hopefully soon! 🤞
Traditional Broker Platforms like Trade Station, Think or Swim, Interactive Brokers as well as specialty charting platforms like MultiCharts, Sierra Charts et al are a mix of C, C++, C#, and Java typically. I have not found anything online yet on how any of them do it, but it's unlikely a concern considering their choice of language.
For the web trading platform Trading View, this is how they handle the Execution of Pine Script
More specifically:
I suppose it really depends on your requirements or limitations.
If the requirement is truly HFT, microseconds trading with a dedicated machine as close to the exchange as possible, then Python is obviously not the choice. However if it is tick data, then you should be fine in many cases with performant hardware. Lastly for 1 minute and higher time frames, I wouldn't be concerned.
For instance, I am running a 2013 Mac Book Pro with 4G RAM, 4 Cores. When using Pandas TA, I can process over a hundred different indicators in about 4secs with ohlcv data containing 700 rows. However in reality, one would have a small sample of indicators to use and it takes my machine microseconds.
For example, when I used this subset of indicators (max period of 200) with 700 bars I got around 50 ms (or 0.05 secs) to process them. When I take 210 bars, 10 extra bars for buffer, I get around 30 ms (or 0.03 secs). Even with my hardware limitation, TA speed bottlenecks are trivial.
But if you have a small portfolio to live track and calculate as a scanner list, of course it may take marginally longer to process them in a blocking manner.
How many indicators in your library would have that capability, or should most of them be rewritten to allow for that?
So in my opinion, I don't think it is necessary to rewrite the few that may benefit. Even with older hardware, TA is pretty fast.
Apologies if this was overkill... KJ
tudorelu
commented
4 years ago Definitely not an overkill, really thanks for the detailed explanation. It put me at ease with respect to the speed of calculation / perceived bottleneck as I haven't tested their speed myself.
I imagine that a well thought out algorithm can perform without issues in the current scenario, with a small number of pairs - however you're right, if you want to compute hundreds of indicators on dozens of pairs every minute - it will take longer than a minute :P so rewriting some of them in a recursive fashion to compute 'in real time' - at least for the indicators that it is possible - might be a use case for python, slightly different than the HFT algo which I totally agree that python wouldn't be the language for.
Again thanks for this answer and your effort of converting TA-lib to python
On Mon, Sep 7, 2020 at 9:19 AM Kevin Johnson notifications@github.com wrote:
@tudorelu https://github.com/tudorelu,
Excellent question.
I imagine a lot of devs would like to have this ability in a TA library..
No doubt!
Yet, in my experience of both creating indicators and reverse engineering them from TA Lib https://sourceforge.net/p/ta-lib/code/HEAD/tree/trunk/ta-lib/c/src/ta_func/, Trading View, et al, it is not possible for many indicators.
In fact it was a discussion on my repo as well. It's the first Pinned Issue: How to utilize this Framework for live ticks #87 https://github.com/twopirllc/pandas-ta/issues/87. Unfortunately, I haven't had much chance to personally address this issue more. Hopefully soon! 🤞 Brokers, Charting Software, and Web Platforms
Traditional Broker Platforms like Trade Station, Think or Swim, Interactive Brokers as well as specialty charting platforms like MultiCharts, Sierra Charts et al are a mix of C, C++, C#, and Java typically. I have not found anything online yet on how any of them do it, but it's unlikely a concern considering their choice of language.
For the web trading platform Trading View, this is how they handle the Execution of Pine Script https://www.tradingview.com/pine-script-docs/en/v4/Quickstart_guide.html#execution-model-of-pine-scripts
More specifically: [image: Screen Shot 2020-09-06 at 9 05 07 AM] https://user-images.githubusercontent.com/18198392/92333600-b0502300-f03b-11ea-9884-1613392b8a12.png Requirements or Limitations
I suppose it really depends on your requirements or limitations.
If the requirement is truly HFT, microseconds trading with a dedicated machine as close to the exchange as possible, then Python is obviously not the choice. However if it is tick data, then you should be fine in many cases with performant hardware. Lastly for 1 minute and higher time frames, I wouldn't be concerned.
For instance, I am running a 2013 Mac Book Pro with 4G RAM, 4 Cores. When using Pandas TA, I can process over a hundred different indicators in about 4secs with ohlcv data containing 700 rows. However in reality, one would have a small sample of indicators to use and it takes my machine microseconds.
For example, when I used this subset of indicators (max period of 200) with 700 bars I got around 50 ms (or 0.05 secs) to process them. When I take 210 bars, 10 extra bars for buffer, I get around 30 ms (or 0.03 secs). Even with my hardware limitation, TA speed bottlenecks are trivial.
[image: Screen Shot 2020-09-06 at 9 24 35 AM] https://user-images.githubusercontent.com/18198392/92336172-9ff77280-f052-11ea-9048-c39ccce67bac.png
But if you have a small portfolio to live track and calculate as a scanner list, of course it may take marginally longer to process them in a blocking manner.
How many indicators in your library would have that capability, or should most of them be rewritten to allow for that?
So in my opinion, I don't think it is necessary to rewrite the few that may benefit. Even with older hardware, TA is pretty fast.
Apologies if this was overkill... KJ
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tudorelu/pyjuque/issues/5#issuecomment-687930786, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACNIYPGYJUS2T2KO2J75PRDSEQKH3ANCNFSM4QTJDQLQ .
twopirllc
commented
4 years ago @tudorelu,
Definitely not an overkill, really thanks for the detailed explanation. It put me at ease with respect to the speed of calculation / perceived bottleneck as I haven't tested their speed myself.
Cool.
So rewriting some of them in a recursive fashion to compute 'in real time' - at least for the indicators that it is possible - might be a use case for python
Yeah.
I imagine that a well thought out algorithm can perform without issues in the current scenario, with a small number of pairs - however you're right, if you want to compute hundreds of indicators on dozens of pairs every minute
Right. I am also building out a Watchlist Class that manages a group of tickers and fetches data from AlphaVantage (currently), and processes TA assigned to it. It's a work in progress with the goal to add more data sources, brokers, and an event loop for live or simulation.
Again thanks for this answer and your effort of converting TA-lib to python
- I am almost certainly using this library for pyjuque as I wanted to change pyti for a long time - I might even give it a try to update some indicators to real-time!
Thanks! Yeah, TA Lib also does not easily install some systems. So I made Pandas TA to have the same level of accuracy as TA Lib, ease of use, and compatibility with the Python Data Science / ML / FinTech ecosystem. I have other cool features in mind to add. I just need to make time to program them. Looking forward to what you produce.
Thanks, KJ
TekpreXyz
commented
3 years ago hi all nice guy! do you know this project? https://github.com/freqtrade/freqtrade Maybe you can take some ispiration from.
TekpreXyz
commented
3 years ago
Hello @tudorelu,
I came across your Github account through another user and saw that you are creating an Algo Trading System and posting via YouTube. Awesome!
I noticed that you are using pyti for TA but it seems to have gone stale nearly two years ago. I was wondering if you would be interested in trying and incorporating Pandas TA into your project? It is built to be compatible with Pandas as an DataFrame extension, correlated with TA Lib (for common indicators), other popular indicators like TTM Squeeze and Trend, and much more! Also check out some Examples for implementation and features.
Thanks for your time, Kevin Johnson