Live OHLCV resampling

[femtotrader]

Hello,

following discussion https://github.com/nardew/talipp/issues/115#issuecomment-1962685822 from @varunsingla211 I also wonder if this library shouldn't provide "a feature which could convert real time tick data into candles of different time intervals" ie live OHLCV resampling.

Any opinion ?

Kind regards

[varunsingla211]

It will make things easier for me for sure.

[nardew]

Can you guys describe the interface of this feature? How would it look like?

[femtotrader]

My opinion is that we should have 2 kind of objects one for calculating OHLC values and another one for summing volume.

timeframe = datetime.timedelta(minutes=5)
ohlc = TimeResamplingOHLC(timeframe=timeframe)
v = TimeResamplingSum(timeframe=timeframe)

now = datetime.datetime.utcnow()
ohlc.add(now, 60_900)
v.add(now, 1.8)

This lead to the concept of timeframe...

For "simple" timeframe it's not a big deal... we can using datetime.timedelta

• 1Min
• 5Min
• 15Min
• 30Min
• 1H
• 4H
• 1D

It may be harder to support other kind of timeframe (weekly, monthly... which one ? month start or month end ? ...)

This is where Python Pandas come with Date Offset ... https://pandas.pydata.org/docs/reference/offset_frequency.html

For example timeframe could be

timeframe = pd.tseries.frequencies.to_offset("5T")

"5T" can be replaced by many other options.

The main problem I see is probably that we don't want to have such a big dependency.

Any opinion?

We could have access to

ohlc[-1].open
ohlc[-1].high
ohlc[-1].low
ohlc[-1].close

and for volume simply

v[-1]

[nardew]

Thanks @femtotrader .

Let's take TimeResamplingOHLC as an example. I think I got what you want to achieve - you keep feeding it values but it resamples them according to the specified timeframe. How will you use this object with indicators later on?

Imagine we have stoch = Stoch(1, 1). Currently you call stoch.add(ohlcv) each time there are new real-time data. How should adding of a new value work with TimeResamplingOHLC? The only solution that comes to my mind is something like

ohlc = TimeResamplingOHLC(timeframe=timeframe)
ret = ohlc.add(now, 10) // this could return some value which denotes that the "previous" timeframe was "closed", meaning it can be added into the indicator
if ret == PREV_CLOSED:
    stoch.add(previous ohlc)

Is it what you had in mind?

Regarding pandas you are right, I do not want to have it as a dependency.

[varunsingla211]

We can have that, or the indicators can also work on live ticks.

So lets say i am resampling on 1 minute time frame And right now i am at 30th second of the latest minute, indicator can be updated with whatever ohlc is at this moment ( there is already a function to update last value) if the ohlc changes ( lets say on 30.03 seconds, the update function gets called and it updates the value of indicator as well ) Once the minute is complete, the last value in the managed sequence of indicator gets fixed and another value gets added as the open of next minute. (at the starting of next minute when there is only single tick, ohlc will be same as open)

This will be the ultimate real time thing.

Here user can chose whether they want to keep updating indicator on real time or they want to see the updated value on the finishing of the candle

[nardew]

@varunsingla211 So if we are "within" the current timeframe we keep updating, once we leave it we append and then same process applies. Correct?

That's something different than @femtotrader had in mind but at first sight I like this approach better.

[varunsingla211]

Yes.

[femtotrader]

My opinion is that we also have to deal with indicator chaining ie we need to feed tick price to TimeResamplingOHLC and it should transmit by either updating or adding to chained indicators.

[varunsingla211]

Yes we will need to transmit the updated value from child indicator to parent indicator.

I dont know but if we update the last value of child indicator, does the parent indicator gets revaluated ?

If it is happening, then we will only need to call the update function and things will work fine. If not, then we will have to write a function (update_parent) which will be called from within the update function of the child indicator.

[nardew]

Agree, I do not see a problem with chaining. The "first" indicator which is fed the input data will make the decision about append/update and rest of the indicators in the chain will act based on it.

It will not work if you chain indicators where each expects a different timeframe. This is so bizarre that I am not even considering this option...

[nardew]

@femtotrader @varunsingla211 I implemented a working version and I would like to release an RC for some indicators. Would you fancy giving it a try? If yes, let me know for which indicators.

[varunsingla211]

Woah.. I would like to try it for macd, rsi, stochastic and stochastic rsi. Lets try it for some chained indicators also. So EMA of RSI can be of use for me.

On Sun, Mar 10, 2024 at 2:31 AM nardew @.***> wrote:

@femtotrader https://github.com/femtotrader @varunsingla211 https://github.com/varunsingla211 I implemented a working version and I would like to release an RC for some indicators. Would you fancy giving it a try? If yes, let me know for which indicators.

— Reply to this email directly, view it on GitHub https://github.com/nardew/talipp/issues/116#issuecomment-1986979246, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMPS7N36WKAGVTQFP2EQNADYXN2CXAVCNFSM6AAAAABDYXXZLKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBWHE3TSMRUGY . You are receiving this because you were mentioned.Message ID: @.***>

-- Varun Singla (VRS)

[nardew]

I added the new functionality into the indicators you mentioned. The testing version can be installed from below branch:

pip install git+https://github.com/nardew/talipp.git@resampling

Here is an example:

from talipp.input import SamplingPeriodType

obv = OBV(input_sampling=SamplingPeriodType.SEC_1)

For all available periods (they are predefined) check definition of the enum.

Keep in mind that sampling works on indicators which accept OHLCV candles as their input, it does not work with float (for obvious reason).

If one wants to add sampling for indicators based on floats, then it is required to wrap the float into OHLCV object (and populate at least close and time) and then use input_modifier to extract the value expected by the indicator.

from talipp.input import SamplingPeriodType
from talipp.ohlcv import ValueExtractor

rsi = RSI(input_values=<list of OHLCV>, input_sampling=SamplingPeriodType.SEC_5, input_modifier=ValueExtractor.extract_close) 

Instead of provided ValureExtractor class you can use any lambda you wish.

Happy to hear a feedback.

[varunsingla211]

Thanks a lot. I will give it a go tomorrow and let u know about it.

One thing that i can tell from the look of it is,

1. user will always give the float values not the ohlcv values.
2. OHLCV values should be derived from the library function only based on the resampling period ( for this we need a method in the ohlcv.py class to convert the floats into ohlcv)

An example :

User will receive live data values via a websocket in this form :

time price volume 01-01-2024 09:30:00 5055 567 01-01-2024 09:30:01 5052 534 01-01-2024 09:30:01 5000 700
01-01-2024 09:30:02 5024 500 .. .. ..

And so on it will keep receiving and keep feeding it to a listener in the library. The listener will call a function which will convert this data to ohlcv candles in the same way ( updating or adding the candle based on timestamp). And that output of ohlcv converter function should be fed to indicators in this manner.

rsi = RSI(input_values=<list of OHLCV>, input_sampling=SamplingPeriodType.SEC_5, input_modifier=ValueExtractor.extract_close)

[nardew]

If you keep receiving floats, what prevents you from doing

rsi.add(OHLCV(close=your_float, time=your_time))

This should be pretty sufficient I guess.

[nardew]

Delivered with 2.1.0.