CMF very different from TradeView

schwaa commented 2 years ago

Which version are you running? The lastest version is on Github. Pip is for major releases.

import pandas_ta as ta
print(ta.version)

pandas-ta .3.2b0 Do you have TA Lib also installed in your environment?

$ pip list

TA-Lib = .4.19

Upgrade.

$ pip install -U git+https://github.com/twopirllc/pandas-ta

upgraded to .3.14b0 Same CMF result.

Describe the bug A clear and concise description of what the bug is.

Using CMF 20 day on AHPI with 30min bars. CMS return is .4 from pandas ta. The CMF 20 day on TradeView is -.2x I'm looking to be close not exact, the issue is one is > 0 the other is < 0.

To Reproduce Provide sample code. cmfdf = df.ta.cmf(length=20) print(cmfdf) last_row = cmfdf.iloc[-1] print(last_row)

Expected behavior A clear and concise description of what you expected to happen. Expected a negative based on Tradeview - but got .4

Screenshots If applicable, add screenshots to help explain your problem.

https://www.tradingview.com/x/oKiSGrOM/

Additional context Add any other context about the problem here.

Could be TradeView is wrong, or that I'm not understanding how the two calculations are different - like if CMF in the ta lib is 0-100 and tradeview is centered on 0?
the ta lib - NOT TA-LIB does also give .4. - I have .7 of the ta pypl installed.

------- CMFDF ------- time 2021-11-22 14:00:00-05:00 NaN 2021-11-22 15:30:00-05:00 NaN 2021-11-23 11:30:00-05:00 NaN 2021-11-23 14:00:00-05:00 NaN 2021-11-23 14:30:00-05:00 NaN ... 2021-12-30 11:30:00-05:00 -0.021492 2021-12-30 12:30:00-05:00 0.276578 2021-12-30 13:30:00-05:00 0.378808 2021-12-30 14:30:00-05:00 0.384685 2021-12-30 15:00:00-05:00 0.400801 Name: CMF_20, Length: 261, dtype: float64

Thanks for using Pandas TA!

twopirllc commented 2 years ago

Hello @schwaa,

What ohlcv source you are using to calculate cmf using Pandas TA? Is it from TradingView or is it another data source? It would also be great if you could provide a correlation test by following this https://github.com/twopirllc/pandas-ta/issues/107#issuecomment-685834922 and show the correlation.

Kind Regards, KJ

schwaa commented 2 years ago

Thank you for the reply - Happy New Year! I'm lame so I'm looking at this a little more today, but I'll have to give up soon and try again tomorrow. I'm using a different data source - paid feed from Alpaca.TradeView is saying I need to upgrade to Pro+ in order easily export the data for the chart, so I have to see about that as well. I do pay for TradeView, but I guess not the right level. I'll keep working on it.

twopirllc commented 2 years ago

@schwaa,

Happy New Year!

Yes, this is a common Issue with people comparing apples and oranges and claiming the library is inaccurate. As you probably know, financial data is not uniform among exchanges, brokers and other data sources. Nor are we privy to how they are cleaned and sanitized or is post edited for other reasons. But when there are inaccuracies, I try to remedy it to within 0.99 correlation. Furthermore unlike the rigor of Mathematics standards, there are no standard in TA indicators. Many indicators have different initialization/bootstrapping reasons for different trading platforms, even TA Lib mentions this in some indicator comments.

I have a Pro+ account, I will do a correlation test when I get some time after addressing other pressing Issues. In the mean time, you are welcome to fork and submit a PR to fix the bug. 😎 Many will appreciate it.

NOT TA-LIB does also give .4

Lastly, TA Lib hasn't been updated in years. The version I use for TA Lib correlation, 0.4.21, does not contain cmf. I suppose they removed it after 0.4.19? Do you have the TA Lib source code for cmf? If so, please post it.

Thanks, KJ

schwaa commented 2 years ago

Hello,I realized there were some volume gaps in the original test case, so to remove that issue, I started to look at XLE. That gets tons of volume. And I was looking at today.

This is TradeView: Sorry I don't have the detailed data. CMF goes from negative to positive today. Even at 11:40 - XLE CMF is still negative using the pandas-Ta. --=== Start XLE ===--- Fetching new bars for XLE at 2022-01-03T11:40:08.997395 checking for buy and sell signals Start Flow Buy function ADD CMF open 56.805000 high 56.980000 low 56.770000 close 56.980000 volume 16997.000000 CMF -0.177575 Name: 2022-01-03 11:00:00-05:00, dtype: float64

I'm still willing to admit I could totally be in the wrong. I have several pandas-ta functions running, but I haven't used CMF too much in the past and never from Pandas-ta.More output from my run. XLE - DF being used as the data input.

open high low close volume time 2021-12-31 13:15:00-05:00 55.435 55.485 55.435 55.460 9941 2021-12-31 13:30:00-05:00 55.500 55.540 55.465 55.490 3789 2021-12-31 13:45:00-05:00 55.515 55.615 55.490 55.615 13798 2021-12-31 14:00:00-05:00 55.605 55.605 55.550 55.555 8565 2021-12-31 14:15:00-05:00 55.525 55.580 55.515 55.550 5614 2021-12-31 14:30:00-05:00 55.535 55.535 55.385 55.385 9576 2021-12-31 14:45:00-05:00 55.365 55.410 55.355 55.410 15689 2021-12-31 15:00:00-05:00 55.435 55.520 55.425 55.485 13324 2021-12-31 15:15:00-05:00 55.475 55.670 55.475 55.670 13262 2021-12-31 15:30:00-05:00 55.630 55.750 55.620 55.745 26646 2021-12-31 15:45:00-05:00 55.740 55.785 55.480 55.490 146494 2022-01-03 09:30:00-05:00 55.595 56.485 55.560 56.485 19878 2022-01-03 09:45:00-05:00 56.480 56.680 56.070 56.655 37623 2022-01-03 10:00:00-05:00 56.675 56.915 56.600 56.875 33228 2022-01-03 10:15:00-05:00 56.890 57.005 56.865 56.900 15644 2022-01-03 10:30:00-05:00 56.920 56.950 56.740 56.850 13763 2022-01-03 10:45:00-05:00 56.900 56.950 56.790 56.790 6983 2022-01-03 11:00:00-05:00 56.805 56.910 56.780 56.785 6263 2022-01-03 11:15:00-05:00 56.805 56.980 56.770 56.980 10734 2022-01-03 11:30:00-05:00 56.970 56.990 56.720 56.850 17965

CMF for timeperiod 2021-12-31 15:30:00-05:00 0.403507 2021-12-31 15:45:00-05:00 -0.228853 2022-01-03 09:30:00-05:00 -0.167205 2022-01-03 09:45:00-05:00 -0.055273 2022-01-03 10:00:00-05:00 0.021792 2022-01-03 10:15:00-05:00 -0.007157 2022-01-03 10:30:00-05:00 -0.010907 2022-01-03 10:45:00-05:00 -0.021565 2022-01-03 11:00:00-05:00 -0.046548 2022-01-03 11:15:00-05:00 -0.026218 2022-01-03 11:30:00-05:00 -0.032900

I would work a code fix, if I found the problem. I know that was requested. I've made small commits before, so I'm happy to, I just haven't figured out where the gap is yet.

Brian(schwaa)

schwaa commented 2 years ago

This is really stupid of me, but I must on some level be doing something wrong. This code works:

import pandas_ta as ta
symbol='XLE'
df = get_15bar_from15(symbol)
ccidf = df.ta.cci(length=20)
print(ccidf)

import pandas_ta as pta
symbol='XLE'
df = get_15bar_from15(symbol)
ccidf = df.pta.cci(length=20)
print(ccidf)

Have I done some common mistake that people mess up or something? Just changing the import as.. should not have killed this. I have no other TA import. thanks again.

twopirllc commented 2 years ago

@schwaa,

Yes, the first code block is correct. In _pandasta/core.py, the DataFrame Extension is called ta and the methods/indicators of that extension must use it (unless you modify the extension name locally).

Pandas TA DataFrame Extension definition: @pd.api.extensions.register_dataframe_accessor("ta")

Thus the second code block won't work because pta is not the name of the DataFrame Extension. However, if you want to use import pandas_ta as pta you can but it applies only for the Standard Programming Convention (like TA Lib) which is more explicit instead of letting the DataFrame Extension automatically handle the default input series (ohlcv).

import pandas_ta as pta

symbol='XLE'
df = get_15bar_from15(symbol)

# Equivalent
ccidf = df.ta.cci(length=20)  # DataFrame Extension
print(ccidf)
ccidf = pta.cci(df.high, df.low, df.close, length=20)  # Standard (like TA Lib)
print(ccidf)

See Programming Conventions on the README for examples.

Hope this helps!

KJ

schwaa commented 2 years ago

Great - That make and I was using ta before, I just changed it try to try some different things to figure out how to understand what is happening.My biggest challenge right now, is that I don't understand the different in the computations with TradeView and I can't get any symbols to match even remotely. I've been trying all the sector ETF's and I get close.XLE, XLP, XLRE, XLU, etc Brian

twopirllc commented 2 years ago

Hey @schwaa,

Ok, so I exported Daily SPY since inception with TV's CMF with length 20. Here are the correlation results and additional beginning and ending rows for comparison.

Hope this helps!

Kind Regards, KJ

schwaa commented 2 years ago

Yes - Thank you -To summarize - you would say - if the data is the same, the result will be the same, so my SIP data, must be different from TradeView. Sounds like what you are thinking?

twopirllc commented 2 years ago

@schwaa,

That's correct. Like I said above in https://github.com/twopirllc/pandas-ta/issues/461#issuecomment-1003620059, "... financial data is not uniform among exchanges, brokers and other data sources. Nor are we privy to how they are cleaned, sanitized or is post edited for other reasons."

As popular and ubiquitous as TV is, TA Lib is still the de facto TA Library that most users use and compare other exchange/brokerage TA values against; especially the retail algorithmic trading community. Since TA Lib is an independent library, it is easy to test that Pandas TA indicators are highly correlated with each other... unlike Binance, TradeStation, TDA, et al where there data and indicators are not accessible through an API due to terms of service etc. Luckily TV can be tested albeit manually with an expensive a Pro+ account.

What's important is you do not compare identical indicators with two different sources and that your trading system uses the data that you trade with. If your TV Binance data is not equal to your external Binance data, then there will be some deviations.

Hope this makes sense. KJ

twopirllc / pandas-ta

CMF very different from TradeView #461