How to utilize this Framework for live ticks

[22Vignesh-M]

Hi, I'm trying to calculate indicators for Live tick data,

You have done a great work!, its very much helpful and makes life easy ;)

Thanks & Regards, Vignesh Murugan

[twopirllc]

Hello @22Vignesh-M,

Your speed depends on your sources, dimensions and hardware of course. Pandas TA just runs Pandas compatible TA features comparable with TA Lib. It is agnostic to how you want to process the data. You might want to check other Pandas Frameworks that are built for multiprocessing, event processing, streaming, ETL, et al for more efficient solutions.

I have not tried this out, but you can try to help speed up your processing using a temporary DataFrame that is updated every tick and is slightly larger than the your indicator of maximum period. For example, like this example pseudo code:

import pandas as pd
import pandas_ta as ta

momo_bands_sma_ta = [
    {"kind":"sma", "length": 50},
    {"kind":"sma", "length": 200},
    {"kind":"bbands", "length": 20},
    {"kind":"macd"},
    {"kind":"rsi"},
    {"kind":"log_return", "cumulative": True}, # Creates new column: "CUMLOGRET_1"
    {"kind":"sma", "close": "CUMLOGRET_1", "length": 5, "suffix": "CUMLOGRET"},
]
momo_bands_sma_strategy = ta.Strategy(
    "Momo, Bands and SMAs and Cumulative Log Returns", # name
    momo_bands_sma_ta, # ta
    "MACD and RSI Momo with BBANDS and SMAs 50 & 200 and Cumulative Log Returns" # description
)

# Since SMA requires 200 periods above, we will set last to 200
last = 200
# Indicator length buffer
last_buffer = 10
rows = last + last_buffer

histdf = YourHistoricalSource()
has_new_tick = True

while has_new_tick:
    lastdf = histdf.tail(rows).copy() # Get the last 210 rows from histdf
    lastdf.ta.strategy(momo_bands_sma_strategy) # Process the Strategy with lastdf
    histdf.append(lastdf.iloc[-1], ignore_index=True) # Append new TA to histdf
    histdf.to_<Storage>() # Save to your Storage if you require it

    new_tick = YourTickSource() # Request new tick for your Broker
    if new_tick has "ohlcv": # Or whatever satisfies your requirements
        has_new_tick = True
    else:
        has_new_tick = False

Let me know what you find or come up with. Hope this helps!

Don't forget to ⭐ if you find the library useful.

Thanks, KJ

[22Vignesh-M]

Hi @twopirllc,

This is exactly what I wanted, but I'm getting some fluctuations in values for some indicators which depend on the OHLC form the beginning of the data frame.

Example Screenshots:

Computations on full Data Frame:

Computations on Live Ticks:

Regards, Vignesh Murugan

[twopirllc]

Hello @22Vignesh-M,

Thanks for replying and providing more info.

However, can you share some of your code so I can see what is happening? It looks like your MACD is incomplete, it requires a minimum of 26 periods. I would at least look back 30 periods/rows maybe even 50 since you have a small subset of indicators.

Regards, KJ

[twopirllc]

Hello @22Vignesh-M,

I don't know what you use for charting, but mplfinance just added Animation Support. Check it out, maybe it will help. 🤷‍♂️

Regards, KJ

[22Vignesh-M]

Hello @twopirllc,

Here is the code that I have tried for live tick calculation.

import pandas as pd
import pandas_ta as ta
from alphaVantageAPI.alphavantage import AlphaVantage

#AlphaVantage
def farm(ticker = 'AAPL', drop=['dividend', 'split_coefficient']):
    AV = AlphaVantage(api_key="YOUR API KEY", premium=False, clean=True, output_size='full')
    df = AV.data(symbol=ticker, function='D')
    df.set_index(pd.DatetimeIndex(df['date']), inplace=True) if not df.ta.datetime_ordered else None
    df.drop(['dividend', 'split_coefficient'], axis=1, inplace=True) if 'dividend' in df.columns and 'split_coefficient' in df.columns else None
    df.name = ticker
    return df

#Historical Data
ticker = 'GOOGL'
df = farm(ticker)

#Live Tick Iterator
def live_df_simulation(df, start, n, end):
    while True:
        if start+n < end:
            yield df.iloc[start: start+n]
            start += 1
        else:
            yield pd.DataFrame()

#Strategy Definition
strategy = [
    {"kind":"sma"},
    {"kind":"bbands"},
    {"kind":"macd"},
    {"kind":"rsi"},
    {"kind":"supertrend"},
    {"kind":"mfi"}
]

strategy = ta.Strategy('live_indicators', strategy, 'technical indicators live calculation')
df_hist = df.copy(deep=True)

#Calclating Indicators
df_hist.ta.strategy(strategy)

#Live Tick Simulation
start = 300 #Starts from index 300
end = len(df)-105
n = 30 

#df_slice iterates through 300-399, 301-400, 302-401 .... 3918-4017
df_slice = live_df_simulation(df, start, n, end)

#Live DataFrame
df_live = pd.DataFrame()
tmp = next(df_slice)

#Live Calculation
while tmp.empty != True:
    tmp.ta.strategy(strategy)
    df_live = df_live.append(tmp.iloc[-1])
    tmp = next(df_slice)

df_hist = df_hist[['open', 'high', 'low', 'close']+['SUPERT_7_3.0', 'RSI_14', 'SMA_10', 'MFI_14', 'MACD_12_26_9']]
df_live = df_live[['SUPERT_7_3.0', 'RSI_14', 'SMA_10', 'MFI_14', 'MACD_12_26_9']]

#Comparison b/w single_run calculation and live_tick calculation
comparision = df_hist.merge(df_live, how='inner', left_index=True, right_index=True)

#if we increase the number of rows n=100 or 200, there is a change in values only after the decimal point

And lots of thanks for suggesting a charting solution, I'm currently using bokeh for live streaming and charting.

Thanks & Regards, Vignesh Murugan

[twopirllc]

Hello @22Vignesh-M,

Thanks for getting back to me with some example code to help. I'll take a look as soon as I can.

I'm current using bokeh live streaming and charting.

Excellent choice.

Thanks, KJ

[pbrumblay]

Thank you for this discussion. This was something I stumbled on and you folks helped me out. Here's a simple example I'm going to run with: https://gist.github.com/pbrumblay/b8fee2a425be6bfc88fc43ff15fb0199 (not using strategy yet but I will...).

I'm not going to worry about optimizing just yet but if pandas is really copying the whole DF there might be a performance problem in there depending on DF size (as @twopirllc mentioned above).

[twopirllc]

@pbrumblay,

Thanks for using Pandas TA! Glad it helped you out.

Thanks, KJ

[sasha-id]

I guess most efficient way to achive streaming processing is to utilize:

• pandas.DataFrame.rolling: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.rolling.html
• pandas.DataFrame.expanding: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.expanding.html

[rahulmr]

@twopirllc @22Vignesh-M This is extremely cool stuff. It will be great guys if you can create example for alpaca paper trading account. It will be of great use, once you code it for alpaca others can code similarly for other brokers. If I happen to do this myself, I will definitely post it here. Also, the watchlist example but for live market with alpaca example. Alpaca Paper trading is available for free and even oanda demo account is free.