ranaroussi / yfinance

Download market data from Yahoo! Finance's API
https://aroussi.com/post/python-yahoo-finance
Apache License 2.0
12.42k stars 2.25k forks source link

Peg Ratio value way off #903

Closed rickturner2001 closed 2 years ago

rickturner2001 commented 2 years ago

The Peg Ratio value for "FE", First Energy seems to be wrong, according to Google Finance the value is 2.99 but yfinance returns 43.37 on my script. This is how i got this value:

import yfinance as yf

symbol = yf.Ticker("FE")

infos = (symbol.info)

print(infos['pegRatio'])

Output: 43.37

It works just fine with other stock which is why i don't really get what the problem seems to be.

asafravid commented 2 years ago

You need yahoo finance not Google finance - pay attention

בתאריך יום ד׳, 8 בדצמ׳ 2021 ב-20:23 מאת rickturner2001 < @.***>:

The Peg Ratio value for "FE", First Energy seems to be wrong, according to Google Finance the value is 2.99 but yfinance returns 43.37 on my script. This is how i got this value:

import yfinance as yf

symbol = yf.Ticker("FE")

infos = (symbol.info)

print(infos['pegRatio'])

Output: 43.37

It works just fine with other stock which is why i don't really get what the problem seems to be.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ranaroussi/yfinance/issues/903, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK7ZJS5SKO36IQ7AJPH33S3UP6PBHANCNFSM5JUO4H3Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Asaf - אסף

rickturner2001 commented 2 years ago

I might be mistaken but it seems logical to me that the peg Ratio should be fairly similar from different platforms. regardless, 43 is way over the normal value for peg ratio and anyway in the yahoofinance.com website the peg ratio is 2.99.

asafravid commented 2 years ago

You are correct, I also get that 43.37 value in yfinance and yahoo finance shows PEG Ratio (5 yr expected) | 3.02

maybe a yfinance issue I'll step-by-step debug inside and update findings, nice catch

asafravid commented 2 years ago

This is the returned html (relevant part) requested: line 544 in sessions.py, callstack is image

{},"lastSplitFactor":null,"legalType":null,"lastDividendDate":{"raw":1635984000,"fmt":"2021-11-04"},"morningStarOverallRating":{},"earningsQuarterlyGrowth":{"raw":0.02,"fmt":"2.00%"},"priceToSalesTrailing12Months":{},"dateShortInterest":{"raw":1636934400,"fmt":"2021-11-15"},"pegRatio":{"raw":43.37,"fmt":"43.37"},"ytdReturn":{},"forwardPE":{"raw":15.511812,"fmt":"15.51"},"maxAge":1,"lastCapGain":{},"shortPercentOfFloat":{"raw":0.0346,"fmt":"3.46%"},"sharesShortPriorMonth":{"raw":15382744,"fmt":"15.38M","longFmt":"15,382,744"},"impliedSharesOutstanding":{},"category":null,"fiveYearAverageReturn":{}},"details":{},"summaryProfile":{"zip":"44308","sector":"Utilities","fullTimeEmployees":12153,"longBusinessSummary":"FirstEnergy Corp., through its subsidiaries, generates, transmits, and distributes electricity in the United States. The company operates through Regulated

asafravid commented 2 years ago

So I think yfinance is kind of mislead here by the returned data.... it just takes it as it is I suspect the session headers now... I'm going to simulate chrome browser headers and retry

asafravid commented 2 years ago

Its as if the API of Yahoo Finance provides wrong data (specifically for the pegRatio) when comparing to the Website Contents Maybe a Yahoo Finance API versioning issue? @ranaroussi

asafravid commented 2 years ago

These are the headers used by yfinance (not sure headers are related here though, I now suspect the API itself..)

user_agent_headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}

asafravid commented 2 years ago

Nope, nothing to do with headers - I tried latest chrome (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36 from http://www.useragentstring.com/index.php?id=19919 but no change, pegRatio still the wrong value Need to check different Yahoo Finance API versions...

asafravid commented 2 years ago

This is the get request yfinance issues, seems that all is well with it, I'll feed it in my browser to see reault

image image

asafravid commented 2 years ago

The website feeds from query2 (seen on taskbar): https://finance.yahoo.com/quote/FE/key-statistics?p=FE

asafravid commented 2 years ago

From my 2 cents of debugging I see no yfinance issue here... Yahoo finance provides an HTML with different value (specifically for pegRatio) than one would see in the website... so still maybe an API issue but none that I can point at as of yet

image

asafravid commented 2 years ago

Super Strange

asafravid commented 2 years ago

@rickturner2001 can you find another example (ticker/quote) of another company with this difference in pegRatio? Maybe there’s some calculation which yahoo finance applies so as to get from 43.37 to ~3.0 Therefore if we have more examples we can further investigate this My guess is that yahoo finance takes 43.37 and applies some function it to reach 3.0 but this is just a guess Html contains pegRatio to be 43.37 so need more examples to proceed (unless someone has other ideas)

rickturner2001 commented 2 years ago

@asafravid Thank you so much for taking the time to properly debug this whole thing. The fact that the API returns a value that differs from the one in the website is quite unusual, especially because i am almost sure that with the .infomethod the API should in fact be scraping the value from the website itself, so i really don't know what this issue it being caused by. Anyway, I'm going to be working with the API the whole week so I will surely get back to you.

rickturner2001 commented 2 years ago

I have found a few more tickers that seem to have the same issue:

**FORMAT:**
(Ticker, value, real value from https://finance.yahoo.com/quote/{TICKER}/key-statistics?p={TICKER})

CNP, 10.55 , 5.07

FAST, 6.24, 4.09

GILD, 6.52, 0.52

SJM, 9.51, 7.96 (maybe not as relevant)

NI, 5.39, 2.57

This might be the most interesting case

PPL, -1.55, N/A

This is all i have gather. The fact that PPL on the website has not been assigned a value makes me think that maybe we're looking at the wrong label, but then again all the other keys in the dictionary returned from the .info function are reported in the same page.

asafravid commented 2 years ago

Thanks, I’ll take a look at the other tickets you’ve sent Question: how many tickets did you check for this anomaly? Do you use a script? I’m asking since my stock scanner can scan all nasdaq traded stocks automatically If you have a script to read this peg ratio from Google finance then I can do the comparison for all nasdaq stocks and specifically one can take the Google finance value rather than the yfinance value (for the peg ratio) for those anomaly cases (kind of looking for a workaround now) For other tickets is it 1to1 bit exact pegRatio equals website one? Also we should follow up in like 1 month or 1 quarter (unless this is resolved) as peg ratio is price to earnings / growth and price to earnings changes daily (as price changes daily) But question is - is it calculated daily as well by yahoo finance? Let’s also see tomorrow for those tickets and of course for FE

rickturner2001 commented 2 years ago

I have checked for a hundred stocks or so and the pegRatios were all different from the value in the website, even if some by a few decimals, none were matching. I am sorry but I actually mislead you by saying that i was comparing the Yfinance data with the google finance data when i was in fact only searching through the Google engine and getting the data from Yahoo Finance itself, not Google Finance (which by the way doesn't provide the Peg Ratios). To answer your question, I tried scraping the data directly from Yahoo Finance by using BeautifulSoupand Requests but I was getting bad responses (503, 404 and 403) so i decided to use Puppeteer.js to scrape the data.

The problem is that none of the different sources (yahofinance.com, finviz.com, nasdaq.com and zacks.com) have the peg Ratio we are looking for, even though the 5 years expectation is the closest value.

Should i try and find a relation between the data I am getting from the API with the one in the website?

asafravid commented 2 years ago

Thanks @rickturner2001 this is very useful and important information In my yfinance-based open source stock scanner I calculate the PEG ratio manually in case pegRatio doesn't exist But it seems I need to use the manual calculation only You can take a look: https://github.com/asafravid/sss Let's wait for some more opinions on this issue but question - perhaps you can check and compare (in the meanwhile) for non-US stocks? Same different values? Must be some funcation() Yahoo applies to get the pegRatio... maybe the projected pegRatio (in all websites including yahoo finance) vs. Current pegRatio? Just a thought there

rickturner2001 commented 2 years ago

I think you are right and waiting a few more opinions is going to be the best option. I will get back to you with the non-US stock results. By the way congrats on your screener

asafravid commented 2 years ago

One thing to notice (and yet to be proved empirically) is that the real value is (as per your examples above) always lower than the yfinance value (except the PPL which for the sake of simplicity let's ignore for now): I.e. Actual value is always lower (better) than the reported yfinance value (except the negative+ N/A exception) Can this be proved? Can you summarize (upload) all your checks? Let's see it the yfinance value is always higher (less attracitve fundamentaly) than the actual value @rickturner2001 let me know please

rickturner2001 commented 2 years ago

I'm sorry i couldn't get back to you sooner but I had some problems trying to automatically scrape the data from yahoo finance, apparently they do prevent people form easily scraping the data from the website. anyway, here are 50 stocks i went through. The Format is the same I used above

MMM, 2.24, N/A
AMD, 1.56, 1.23
ALLE, 3.21, 2.65
AEP, 3.21, 2.88
ADI, 1.62, 2.29
AIZ, 0.95, N/A
BAC, 0.52, 0.96
BK, 1.34, 1.28 
CTLT, 2.29, 2.80
CF, 0.24, 0.30
CTAS, 3.42, 4.60
CL, 3.96, 3.95
COST, 3.77, 5.08
DE, 1.31, 1.74
DISH, -0.34, 1.29
DD, 1.25, 1.01
EMR, 1.76, 1.85
RE, 0.17, 0.46
FAST, 6.24, 4.09
F, 0.14, 0.17
IT, 1.7, N/A
GS, 0.37, 1.78
HOLX, 4.98, 0.86
IBM, 1.81, 2.05
IFF, 6.4, 0.73
JKHY, 2.41, 2.50
KEYS, 2.36, 2.33
LW, 3.1, 2.35
L, None, N/A
MLM, 1.73, N/A
MTD, 2.58, 3.02
MPWR, 2.99, 2.34
NEM, None, 0.79
NLOK, 1.12, 0.67
OMC, 1.29, N/A
PYPL, 2.99, 2.14
PXD, 0.2, 0.29
PTC, 1.74, 2.31
RJF, 0.72, 2.01
ROL, 7, N/A
STT, 0.83, N/A
TPR, 1.37, N/A
HIG, 1.23, N/A
TRMB, 3.61, 2.86
UNP, 1.52, N/A
VZ, 2.9, 2.92
WAB, 2.9, N/A
WU, 1.14, 1.11
YUM, 1.86, 1.93

None of these are perfectly matching, which I guess is fine but some are just plain wrong. The value from the website isn't even being consistent as sometimes it is higher and some times it is lower. I strongly believe that this is in fact a different value form the one displayed in the website, which sounds obvious given the fact that the values are all different but what i am really suggesting is that your theory about a function being applied to each value seems unlikely now looking at the final result.

asafravid commented 2 years ago

Thanks From your data my assumption is incorrect The yfinance value is not always higher than the website value But I will look at these symbols (scan them) and maybe I’ll see how they are calculated

rickturner2001 commented 2 years ago

shall we assume that the N/A values are given for a lack of data?

asafravid commented 2 years ago

I suppose so Need to recheck in like a month/quarter or so

asafravid commented 2 years ago

I’ll check if some correlation exists statistically You provided around 40 so if you have more it will help the correlation analysis

asafravid commented 2 years ago

image

asafravid commented 2 years ago

I filtered out all N/A and None entries and sorted by the ratio between the values Around 16/40 (40%) of the values are within a 20% error Let's see the same statistics for 400 symbols (or as many as you can add here - that would be much appreciated) image

rickturner2001 commented 2 years ago

Here are a few more tickers. Please keep me updated if something new comes up.

Note that these are all securities in the S&P500,

MMM, 2.23, 2.14
AOS, 3.51, 2.42
ABT, 1.99, 2.58
ABBV, 2.19, 1.28
ABMD, 7.51, 2.75
ACN, 3.33, 3.36
ATVI, 1.18, 1.01
ADM, 1.32, 3.02
ADBE, 2.77, 2.25
AAP, 1.18, 1.12
AMD, 1.49, 1.22
AES, 1.87, 1.81
AFL, 2.41, 0.93
A, 2.96, 2.14
APD, 2.18, 1.53
AKAM, 1.64, 1.58
ALK, 1.06, 1.3
ALB, 2.15, 2.22
ALGN, 1.4, 3.78
ALLE, 3.23, 2.68
LNT, 3.64, 3.27
ALL, -1.62, 1.95
GOOGL, 1.29, 0.85
GOOG, 1.3, 0.86
MO, 1.87, 7.63
AMZN, 2.39, 2.07
AEE, 2.86, 2.83
AEP, 3.23, 2.88
AXP, 0.39, 0.66
AIG, 0.38, 0.55
AMT, 2.84, 2.59
AWK, 5.05, 4.65
ABC, 0.97, 1.47
AME, -24.59, 2.08
AMGN, 2.14, 1.6
APH, 2.38, 2.03
ADI, 1.67, 2.36
ANSS, 4.98, 4.23
ANTM, 1.15, 1.28
AON, 1.81, 2.01
APA, -0.27, 3.07
AAPL, 1.4, 3.75
AMAT, 0.96, 2.05
APTV, 0.85, 1.02
ANET, 2.73, 1.85
AJG, 2.6, 2.09
AIZ, 1, 1.46
T, 3.05, 9.17
ATO, 2.47, 2.31
ADSK, 2.17, 1.34
ADP, 2.97, 3.28
AZO, 1.22, 1.53
AVB, 17.21, 5.73
AVY, 2.57, 2.92
BKR, 0.09, 0.33
BLL, 1.82, 4.01
BAC, 0.52, 0.97
BBWI, 1.6, 1.1
BAX, 1.99, 1.5
BDX, 1.96, 1.79
BBY, 1.3, 1.61
BIO, 3.21, 3.5
TECH, 4.39, 1.68
BIIB, -2.25, 6.24
BLK, 1.49, 2.7
BK, 1.17, 1.33
BKNG, 0.27, 0.21
BWA, 0.47, 0.5
BSX, 1.31, 1.52
AVGO, 1.19, 1.28
BR, 2.26, 2.29
BRO, 2.13, 4.41
CHRW, 1.6, 1.53
CDNS, 4.45, 4.67
CPB, 3.77, 5.02
COF, 0.14, 0.44
CAH, 1.41, 0.5
KMX, 0.9, 1.28
CARR, 1.39, 1.69
CTLT, 2.41, 2.8
CAT, 0.57, 1.09
CBOE, 9.1, 1.75
CBRE, 1.83, 1.0
CDW, 2.07, 1.53
CE, 0.33, 0.58
CNC, 1.13, 1.05
CNP, 3.92, 5.2
CERN, 1.98, 2.13
CF, -2.02, 0.3
CRL, 2.63, 1.67
SCHW, 1.09, 1.1
CHTR, 1.05, 0.54
CVX, -2.97, 0.53
CMG, 1.28, 1.34
CB, 0.56, 0.86
CHD, 3.84, 4.34
CI, 0.78, 0.83
CTAS, 3.31, 4.65
CSCO, 2.69, 2.99
C, 0.25, 0.69
CFG, -3.08, 0.94
CTXS, 1.89, 3.69
CLX, 20.57, 6.29
CME, 5.92, 4.99
CMS, 4.1, 4.02
KO, 2.48, 2.38
CTSH, 1.59, 1.16
CL, 3.38, 3.95
CMCSA, 1.06, 1.48
CMA, -0.87, 0.76
CAG, 7.4, 2.64
COP, -6.62, 8.41
ED, 9.07, 4.29
STZ, 2.33, 2.85
CPRT, 1.62, 1.14
GLW, 0.77, 0.91
CTVA, 1, 1.03
COST, 3.98, 5.08
CTRA, None, 0.14
CCI, 3.67, 3.63
CSX, 1.39, 1.71
CMI, 0.8, 1.15
CVS, 1.76, 1.92
DHI, 1.23, 2.62
DHR, 2.03, 2.32
DRI, 0.69, 1.54
DVA, 0.84, 0.94
DE, 0.49, 1.74
DVN, 0.41, 1.62
DXCM, 13.32, 8.25
FANG, 0.14, 0.35
DLR, 2.64, 1.91
DFS, 0.13, 0.38
DISCA, 1.3, 0.53
DISCK, 0.52, 0.51
DISH, -0.55, 1.29
DG, 3.33, 2.44
DLTR, 2.19, 1.73
D, 3.07, 2.57
DPZ, 3.09, 2.89
DOV, 1.6, 1.7
DTE, 7.75, 2.95
DUK, 3.76, 3.82
DD, 1.25, 1.05
DXC, 0.35, 0.28
EMN, 0.64, 1.32
ETN, 1.39, 1.73
EBAY, 1.64, 3.29
ECL, 2.78, 2.35
EIX, 3.19, 3.78
EW, 3.31, 2.31
EA, 0.84, 7.19
LLY, 2.23, 1.73
EMR, 2.01, 1.85
ENPH, 1.98, 1.82
ETR, 4.93, 2.68
EOG, 0.15, 1.11
EFX, 2.72, 1.92
EQIX, 3.85, 4.0
EQR, 9.33, 4.81
ESS, 7.52, 4.77
EL, 2.47, 4.42
ETSY, 1.34, 2.56
EVRG, 3.61, 1.41
ES, 3.62, 2.25
EXC, -38.34, 7.95
EXPD, 5.38, 2.5
EXR, 6.14, 3.27
FFIV, 1.5, 1.3
FB, 0.93, 0.92
FAST, 5.69, 4.09
FRT, 9.28, 2.62
FDX, 1.08, 1.06
FIS, 1.12, 1.14
FITB, -3.6, 0.9
FRC, 1.66, 1.5
FE, -8.54, 3.02
FISV, 1.1, 1.01
FLT, 1.35, 1.18
FMC, 1.71, 1.38
F, 0.11, 0.17
FTNT, 4.88, 3.95
FTV, 2.7, 1.59
FBHS, 1.96, 1.37
BEN, 0.92, 0.53
FCX, 0.68, 0.2
GPS, 2.37, 4.15
GRMN, 4.27, 1.83
IT, 2.44, 1.99
GNRC, 5.59, 1.69
GD, 2.11, 2.32
GIS, 3.36, 4.1
GM, 0.59, 0.67
GPC, 4.16, 1.53
GILD, 7.81, 0.52
GPN, 0.97, 0.72
GS, 0.37, 1.78
HAL, 0.35, 0.44
HAS, 1.17, 1.79
HCA, 1.09, 0.89
PEAK, 26.92, 3.89
HSIC, 1.11, 2.11
HPE, 0.59, 2.24
HLT, 0.2, 0.22
HOLX, 2.52, 0.88
HD, 2.12, 2.11
HON, 2.24, 1.84
HRL, 3.29, 3.97
HWM, 1.02, 0.8
HPQ, 0.48, 4.74
HUM, 1.44, 1.32
HBAN, -6.21, 0.62
HII, 21.69, 0.73
IBM, 0.8, 2.15
IEX, 2.95, 2.57
IDXX, 4.76, 6.02
INFO, 3.48, 3.22
ITW, 1.82, 1.92
ILMN, 3.4, 2.96
INCY, 1.21, 0.91
IR, 1.64, 1.26
INTC, 1.12, 2.86
ICE, 2.79, 2.68
IFF, 3.24, 0.73
IP, 0.48, 1.35
IPG, 0.93, 2.43
INTU, 3.46, 3.19
IVZ, 0.37, 0.77
IPGP, 0.8, 2.52
IQV, 1.55, 1.04
IRM, 5.14, 1.08
JBHT, 1.33, 1.68
JKHY, 2.37, 2.5
J, 1.53, 1.54
SJM, 9.47, 7.96
JNJ, 2.07, 1.63
JCI, 1.34, 1.72
JPM, 1, 1.4
JNPR, 2.03, 1.34
KSU, 2.21, 2.55
K, 4.76, 2.43
KEY, 0.79, 0.92
KEYS, 2.43, 2.33
KIM, 4, 2.1
KMI, 1.68, 0.41
KLAC, 1.19, 1.8
KHC, -4.89, 2.8
KR, 1.91, 4.11
LHX, None, 1.17
LH, -1.15, 2.07
LRCX, 1.27, 1.5
LW, 3.1, 2.35
LDOS, 1.41, 2.34
LEN, 0.33, 0.59
LNC, 0.18, 1.24
LIN, None, 2.22
LKQ, 0.44, 1.08
LMT, 3.14, 8.53
L, None, 2.69
LOW, 1.11, 1.37
LYB, 0.1, 1.51
MTB, 0.84, 1.31
MKTX, 17.19, 4.46
MAR, 0.23, 0.23
MMC, 2.16, 2.48
MLM, 1.78, 1.38
MAS, 1.2, 1.92
MA, 1.58, 1.33
MTCH, 2.35, 1.67
MKC, 5.26, 5.72
MCD, 1.28, 2.12
MDT, 1.72, 2.05
MRK, 0.82, 1.17
MET, 1.02, 1.8
MTD, 2.67, 3.12
MGM, 0.27, 4.2
MCHP, 1.14, 1.66
MU, 0.21, 1.03
MSFT, 2.24, 2.83
MHK, 3.31, 0.93
MDLZ, 2.3, 3.28
MPWR, 2.83, 2.34
MNST, 2.48, 3.4
MCO, 2.95, 2.73
MS, 2.37, 6.2
MSI, 2, 1.97
MSCI, 3.73, 3.74
NDAQ, 1.88, 2.93
NTAP, 1.49, 1.04
NFLX, 1.31, 1.49
NWL, 3.55, 1.42
NEM, -10.54, 0.79
NWSA, 1.56, 2.3
NWS, None, 2.32
NEE, 4.37, 3.71
NLSN, 2.48, 1.77
NKE, 2.23, 3.57
NI, 5.34, 2.57
NSC, 1.48, 1.69
NTRS, 1.11, 1.12
NOC, 2.2, 9.53
NLOK, 1.1, 0.67
NRG, 0.27, 0.53
NUE, 0.2, 0.75
NVDA, 1.66, 3.25
NVR, 3.2, 4.89
NXPI, 1.11, 1.17
ORLY, 1.53, 1.44
ODFL, 1.57, 1.93
OMC, 1.29, 0.97
OKE, 1.66, 1.34
ORCL, 1.74, 2.39
OTIS, 2.59, 2.67
PCAR, 0.74, 0.94
PKG, 1.08, 1.83
PH, 1.78, 1.35
PAYX, 4.93, 4.64
PAYC, 4.2, 2.86
PYPL, 2.99, 2.15
PNR, 1.42, 1.59
PBCT, 0.82, 5.52
PEP, 2.57, 3.13
PFE, 0.92, 1.58
PM, 1.38, 1.31
PXD, 0.2, 0.29
PNC, -3.63, 0.6
POOL, 2.09, 1.81
PPG, 2.35, 2.33
PFG, 0.65, 1.5
PG, 3.39, 5.6
PRU, 0.75, 1.46
PTC, 1.74, 2.31
PSA, 2.37, 2.8
PHM, 0.4, 2.5
PVH, -2.51, 4.58
QRVO, 1, 1.08
QCOM, 0.55, 1.07
PWR, 1.61, 0.75
RL, 0.22, 0.38
RJF, 0.72, 2.01
RTX, 0.85, 0.69
O, 8.95, 4.62
REG, 4.16, 1.67
REGN, 3.05, 1.18
RF, 0.19, 0.71
RSG, 3.22, 3.51
RMD, 2.04, 3.79
RHI, 0.76, 1.35
ROK, 2.88, 3.72
ROL, 7, 6.22
ROP, 2.68, 2.47
ROST, 0.29, 0.21
SPGI, 3.66, 3.08
CRM, 5.79, 3.49
SBAC, 0.66, 2.09
SLB, 0.42, 0.4
STX, 0.75, 1.29
SRE, 3.85, 2.25
NOW, 4.6, 2.59
SHW, 2.81, 5.27
SPG, 2.82, 1.81
SWKS, 1.04, 1.12
SNA, 1.64, 1.58
SO, 3.09, 3.47
SWK, 1.15, 1.37
SBUX, 0.66, 3.04
STT, 0.83, 1.17
STE, 2.82, 2.83
SYK, 2.23, 2.16
SIVB, 2.32, 2.21
SYF, 0.2, 0.73
SNPS, 3.09, 2.86
SYY, 0.42, 0.67
TMUS, 1.38, 1.1
TROW, 1.09, 1.68
TTWO, 2.88, 2.57
TPR, 1.37, 1.58
TGT, 1.43, 1.78
TEL, 2.12, 2.32
TDY, 1.61, 0.98
TFX, 2.83, 2.28
TER, 1.49, 1.37
TSLA, 2.69, 2.81
TXN, 2.4, 2.17
TXT, 0.8, 0.53
COO, 3.37, 2.5
HIG, 1.23, 1.41
HSY, 2.93, 4.43
MOS, 1.02, 0.21
TRV, 1.6, 1.09
DIS, 1.46, 1.11
TMO, 5.11, 8.26
TJX, 0.19, 0.16
TSCO, 2.76, 2.79
TT, 1.58, 2.28
TDG, 4.06, 1.62
TRMB, 3.61, 2.86
TFC, 1.23, 1.04
TYL, 7.2, 3.25
TSN, 1.38, 1.42
USB, 0.93, 1.27
ULTA, 0.45, 0.64
UAA, 1.48, 5.43
UA, 1.78, 4.46
UNP, 1.52, 1.94
UPS, 1.11, 1.49
URI, 0.98, 1.15
UNH, 1.74, 1.65
UHS, 1.66, 1.87
VRSN, 5.1, 7.7
VRSK, 4.54, 3.04
VZ, 2.9, 2.85
VRTX, 1.65, 0.31
VFC, 0.48, 0.66
VIAC, -3.54, 0.8
V, 1.95, 1.54
VMC, 2.09, 1.41
WRB, 1.91, 0.72
GWW, 1.48, 1.92
WAB, 2.9, 3.92
WMT, 2.95, 3.12
WM, 2.14, 2.93
WAT, 4.25, 2.59
WEC, 3.68, 2.97
WFC, 0.09, 0.22
WELL, 11.07, 1.8
WST, 2.17, 3.29
WDC, 0.13, 0.39
WU, 1.14, 1.11
WRK, 0.63, 2.09
WY, 2.12, 5.4
WHR, 1.06, 1.65
WMB, 4.22, 2.03
WLTW, 2.31, 0.78
XEL, 3.75, 3.12
XLNX, 4.88, 4.75
XYL, 2.34, 2.61
YUM, 1.86, 1.97
ZBRA, 3.34, 1.76
ZBH, 1.7, 1.99
ZION, -0.27, 0.91
ZTS, 3.41, 3.23
asafravid commented 2 years ago

Thanks I'll analyze those as well

asafravid commented 2 years ago

https://docs.google.com/spreadsheets/d/1NEZtD7f85AXUdys02WWcL6sCFX6D8lJFdbl6gBjb4K0/edit?usp=sharing Its a little comforting to see that a good portion of the values are within a marginal error (say 20%) image BTW @rickturner2001 how did you scrape the data? you can DM me if you kindly wish to share

asafravid commented 2 years ago

Some more histogram Points of View: image

image

image

image

rickturner2001 commented 2 years ago

Thanks for the stats.

There definitely is some sort of pattern, though i would say that the fact that most values have a 20% error isn't really helping us, in the sense that we really can't do much with that. I honestly would have hoped to see a more linear pattern but something weird is going on here, what are your thoughts? Anyway, I'll be sending you the script i used to check for the values

rickturner2001 commented 2 years ago

It's actually a very simple script so I will paste it:

You will surely notice that i am not scraping from yahoo-finance.com. The page I'm scraping the PEG Ratio values from has the exact same values as yahoo-finance.com.

I used bs4 and I;m sure it could have been done in a more elegant fashion though i will admit I am not used to scraping data from a website that doesn't use classes or ids in its html elements so i had to scrape the values in a little bit of a hard coded way. Nevertheless it still works just fine so here you go.


# Declare the tickers to check in the symbols list as strings or even better use a csv file.
# If you decide to use a .csv file remember to change the for loop (e.g. for symbol in symbols['Symbol'])
symbols = []

for symbol in symbols:
    # Dot notation on symbols (BRK.B) will return null values if used with yfinance 
    if not "." in symbol:
        # Getting the yfinance PEG Ratio
        api_peg = yf.Ticker(symbol).info['pegRatio']
        URL = f'https://stockanalysis.com/stocks/{symbol}/statistics/'
        r = requests.get(URL)
        if r.status_code == 200:
            soup = BeautifulSoup(r.content, 'html.parser')
            my_table = soup.findAll('table', {'class': 'text-sm xs:text-base StatsWidget_statstable__9KlU0'})

            for i in my_table:
                if 'PEG' in str(i):
                    ratio_iloc = str(i).find('PEG Ratio')
                    peg_ratio = str(i)[ratio_iloc + 39:ratio_iloc + 43]
                    try:
                        peg_ratio = float(peg_ratio)
                        print(f"{symbol}, {api_peg}, {peg_ratio}")
                    except:
                        # First exception: double digits and negative values.
                        try:
                            peg_ratio = str(i)[ratio_iloc + 40:ratio_iloc + 45]
                            peg_ratio = float(peg_ratio)
                            print(f"{symbol}, {api_peg}, {peg_ratio}")
                        except Exception as e:
                            # Empty td tag in the page means N/A value
                            if str(peg_ratio) == '</td>':
                                peg_ratio = 'N/A'
                                print(f"{symbol}, {api_peg}, {peg_ratio}")
                            else:
                                print('Something went wrong trying to convert the PEG Ratio: ',symbol,  peg_ratio)
asafravid commented 2 years ago

The “.” Can be replaced by the “-“ in Yahoo Finance

I have a list of all ~7000 nasdaq stocks (auto d/l from the nasdaq FTP server - see in my repo), I’ll try to run it for them and see if the statistics hold

will update soon

eabase commented 2 years ago

Have you guys looked into rounding issues? I would suspect all sort of funny things could happen when you start double rounding (due to definition of PEG) while using different SW libraries (1st from Morningstar, then from Yahoo, then from your Excel or Python).

Since you already have your spreadsheet up there, I guess we could try with some random data, to see if different rounding methods (and order of rounding) would create something similar.

asafravid commented 2 years ago

It’s not a rounding issue AFAIK, see the beginning of the debugging

בתאריך יום א׳, 12 בדצמ׳ 2021 ב-19:27 מאת eabase @.***>:

Have you guys looked into rounding https://en.wikipedia.org/wiki/Rounding issues? I would suspect all sort of funny things could happen when you start double rounding (due to definition of PEG) while using different SW libraries (1st from Morningstar, then from Yahoo, then from your Excel or Python).

Since you already have your spreadsheet up there, I guess we could try with some random data, to see if different round would create something similar.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ranaroussi/yfinance/issues/903#issuecomment-991937686, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK7ZJSZNJ5LHRDPP4JKLBILUQTLQFANCNFSM5JUO4H3Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Asaf - אסף

eabase commented 2 years ago

How can I get the HTML part of the OP request?

asafravid commented 2 years ago

Can you elaborate what you need?

asafravid commented 2 years ago

The following code shall fetch all NASDAQ stock symbols and we'll have a few thousands of stocks for the statistics https://github.com/asafravid/sss/blob/master/sss_peg_ratios.py

I gave there all credit to you @rickturner2001 Its running now and once its complete I'll check the stats

eabase commented 2 years ago

Now, I get 43.29.... (Was the data updated?)

rickturner2001 commented 2 years ago

@asafravid Thanks for the credit, I really do appreciate it.

@eabase Where are you getting 43.29 from? Can you please be more specific?

asafravid commented 2 years ago

@eabase the PEG ration can change as it depends on price to earnings Did both yfinance and website value change proportionally? This can be simply checked by comparing the scanned values weekly - I’ll do that

eabase commented 2 years ago

@asafravid @rickturner2001 Yeah, ok. I updated all packages, then ran the script from OP.
Unfortunately I did not pay close attention to what was the website value at the time.

I also tried some variation of rounding in Excel, but then the largest change I could get was about a few %. The rounding issues are more "severe" in python because of the floating point arithmetic. But did not try anything else.

Other reasons?
Then I am also wondering if some website dev decided to cheat and possibly just truncate the data value instead of rounding? Or if the regex used in this tool, is getting fooled by some character encoding, (UTF-8/16 vs ASCII)?

Click here to see pip list ```bash $ pip list Package Version ------------------ --------- aiosmb 0.2.50 aiowinreg 0.0.7 asn1crypto 1.4.0 asysocks 0.1.2 beautifulsoup4 4.10.0 certifi 2021.10.8 chardet 4.0.0 charset-normalizer 2.0.9 colorama 0.4.4 idna 3.3 lxml 4.6.4 MechanicalSoup 1.1.0 minidump 0.0.21 minikerberos 0.2.14 msldap 0.3.30 multitasking 0.0.10 numpy 1.21.4 oscrypto 1.2.1 pandas 1.3.5 pip 21.3.1 prompt-toolkit 3.0.24 pycryptodomex 3.12.0 pypykatz 0.5.2 python-dateutil 2.8.2 pytz 2021.3 requests 2.26.0 setuptools 59.5.0 six 1.16.0 soupsieve 2.3.1 termcolor 1.1.0 tqdm 4.62.3 urllib3 1.26.7 wcwidth 0.2.5 wesng 1.0.2.1 winacl 0.1.1 winsspi 0.0.9 yfinance 0.1.67 ```
eabase commented 2 years ago

Not sure if the following is helpful at all, since I'm new to this, so...

I just saw this on Wikipedia for PEG, not sure if this can be related. image

Further down the text reads:

"When the PEG is quoted in public sources it makes a great deal of difference whether the earnings used in calculating the PEG is the past year's EPS, the estimated future year's EPS, or even selected analysts' speculative estimates of growth over the next five years. Use of the coming year's expected growth rate is considered preferable as the most reliable of the future-looking estimates. Yet which growth rate was selected for calculating a particular published PEG ratio may not be clear, or may require a close reading of the footnotes for the given figure."


What values would you use for manually calculating PEG from the Reuters page? https://www.reuters.com/companies/FE.N/key-metrics

Here's Morningstar: http://financials.morningstar.com/valuation/price-ratio.html?t=FE&region=usa&culture=en-US

eabase commented 2 years ago

I think you should look into the N/A values and look if the real ingredients are there. If I was programming something, I'd use N/A if I got oo or division by zero etc. If there is some kind of rounding/truncation error somewhere, then symbols giving that would be a good place to look first.

Never mind, I just looked at your scraping code. 🤦‍♂️

rickturner2001 commented 2 years ago

@eabase The problem is not getting the Peg Ratio per se, rather understanding what seems to be inherently wrong with yfinance since it's one the main assets for financial analysis in python.

Given the first example ('FE' PEG Ratio) wouldn't you exclude a rounding issue from the get-go? I certainly would.

eabase commented 2 years ago

@rickturner2001 No, that was clear all along. But it is still funny how it is presented differently from different sources. Given that yfinance is using multiple source, for their data, including calculating it on their own, I supposed we can't exclude anything. It's the 1st time I'm back on this repo since 2+ years, and was curious, and certainly lack understanding on how yfiance and yahoo web pages collect their data. It shouldn't be that hard to track down the source for the data, and if it is, find someone in yahoo who can look it up.

asafravid commented 2 years ago

@eabase - No one in Yahoo is obligated to look it up nor do I think you can "find someone in yahoo who can look it up":

Yahoo Finance is unrelated to yfinance - see the yfinance readme:

IMPORTANT LEGAL DISCLAIMER Yahoo!, Y!Finance, and Yahoo! finance are registered trademarks of Yahoo, Inc.

yfinance is not affiliated, endorsed, or vetted by Yahoo, Inc. It's an open-source tool that uses Yahoo's publicly available APIs, and is intended for research and educational purposes.

You should refer to Yahoo!'s terms of use (here, here, and here) for details on your rights to use the actual data downloaded. Remember - the Yahoo! finance API is intended for personal use only.

rickturner2001 commented 2 years ago

@eabase Yahoo Finance does not provide the wrong values to begin with so it wouldn't make much sense to reach out to someone who associates with Yahoo Finance other then the fact that they might be qualified to solve the issue or even give an explanation for this peculiar bug. Other than that I don't see why any of the employees should even bother helping since yfinance and Yahoo Finance are not affiliated

eabase commented 2 years ago

Looking at the query mentioned above, we find the correct peg ratio but with a different label. Reading the raw curl query returns many "peg", but only 3 pegRatios:

# curl.exe https://finance.yahoo.com/quote/{FE}/key-statistics?p={FE} |grep --color=always -i "pegRatio*"
"pegRatio":{"raw":-2.33,"fmt":"-2.33"},
"trailingPegRatio":[{"dataId":14021,"asOfDate":"2021-12-10","periodType":"TTM","reportedValue":{"raw":3.0477,"fmt":"3.05"}}],
"quarterlyPegRatio":[{"dataId":14021,"asOfDate":"2020-09-30","periodType":"3M","reportedValue":{"raw":2.8618,"fmt":"2.86"}},

Website seem to be using the trailingPegRatio, whereas your API is using pegRatio.

PS. So , yeah, it would be nice to know the definitions of these...