search

ranaroussi / yfinance

Download market data from Yahoo! Finance's API
https://aroussi.com/post/python-yahoo-finance
Apache License 2.0
14.93k stars 2.45k forks source link

ISIN in .info returns 404 #1744

Closed SteynGuelen closed 11 months ago

SteynGuelen commented 1 year ago

Describe bug

Running .info with the ISIN as parameter returns a 404. Symbol works, ISIN does not.

Simple code that reproduces your problem

ticker_yahoo = yf.Ticker("US0079031078")
ticker_info = ticker_yahoo.info
print(ticker_info)

returns: HTTPError: 404 Client Error: Not Found for url: https://query2.finance.yahoo.com/v10/finance/quoteSummary/US0079031078?modules=financialData%2CquoteType%2CdefaultKeyStatistics%2CassetProfile%2CsummaryDetail&ssl=true&crumb=nkKfTFXt4wT

Debug log

DEBUG    get_raw_json(): https://query2.finance.yahoo.com/v10/finance/quoteSummary/US0079031078
DEBUG    Entering get()
DEBUG     url=https://query2.finance.yahoo.com/v10/finance/quoteSummary/US0079031078
DEBUG     params={'modules': 'financialData,quoteType,defaultKeyStatistics,assetProfile,summaryDetail', 'ssl': 'true'}
DEBUG     Entering _get_cookie_and_crumb()
DEBUG      cookie_mode = 'basic'
DEBUG      Entering _get_cookie_and_crumb_basic()
DEBUG       loaded persistent cookie
DEBUG       reusing cookie
DEBUG       crumb = 'nkKfTFXt4wT'
DEBUG      Exiting _get_cookie_and_crumb_basic()
DEBUG     Exiting _get_cookie_and_crumb()
DEBUG     toggling cookie strategy csrf -> basic
DEBUG     Entering _get_cookie_and_crumb()
DEBUG      cookie_mode = 'basic'
DEBUG      Entering _get_cookie_and_crumb_basic()
DEBUG       loaded persistent cookie
DEBUG       reusing cookie
DEBUG       crumb = 'nkKfTFXt4wT'
DEBUG      Exiting _get_cookie_and_crumb_basic()
DEBUG     Exiting _get_cookie_and_crumb()
DEBUG    Exiting get()

Bad data proof

No response

yfinance version

0.2.32

Python version

3.11

Operating system

No response

ValueRaider commented 1 year ago

Your debug log is messy, it should look like this:

DEBUG    get_raw_json(): https://query2.finance.yahoo.com/v10/finance/quoteSummary/US0079031078
DEBUG    Entering get()
DEBUG     url=https://query2.finance.yahoo.com/v10/finance/quoteSummary/US0079031078
DEBUG     params={'modules': 'financialData,quoteType,defaultKeyStatistics,assetProfile,summaryDetail', 'ssl': 'true'}
DEBUG     Entering _get_cookie_and_crumb()
DEBUG      cookie_mode = 'basic'
...

Post if you figure out why your log is different.

SteynGuelen commented 1 year ago

Your debug log is messy, it should look like this:

DEBUG    get_raw_json(): https://query2.finance.yahoo.com/v10/finance/quoteSummary/US0079031078
DEBUG    Entering get()
DEBUG     url=https://query2.finance.yahoo.com/v10/finance/quoteSummary/US0079031078
DEBUG     params={'modules': 'financialData,quoteType,defaultKeyStatistics,assetProfile,summaryDetail', 'ssl': 'true'}
DEBUG     Entering _get_cookie_and_crumb()
DEBUG      cookie_mode = 'basic'
...

Post if you figure out why your log is different.

Happened because I'm using the library in a Jupyter notebook. Ran it in a new python script, main post has been edited with the clean log format. My apologies.

ValueRaider commented 1 year ago

I see a problem, it never invoked the csrf cookie method.

ValueRaider commented 11 months ago

I see a problem, it never invoked the csrf cookie method.

I think I fixed this, try #1759

Urgent to test & merge because yahooquery broke just now. yfinance good for up to 24 hours because reuses cookie.

SteynGuelen commented 11 months ago

same issue even after pulling new branch

import yfinance as yf

print(yf.__version__)

yf.enable_debug_mode()

ticker_yahoo = yf.Ticker("US0079031078")
ticker_yahoo.info

returns: 

DEBUG    get_raw_json(): https://query2.finance.yahoo.com/v10/finance/quoteSummary/US0079031078
DEBUG    Entering get()
DEBUG     url=https://query2.finance.yahoo.com/v10/finance/quoteSummary/US0079031078
DEBUG     params={'modules': 'financialData,quoteType,defaultKeyStatistics,assetProfile,summaryDetail', 'ssl': 'true'}
DEBUG     Entering _get_cookie_and_crumb()
DEBUG      cookie_mode = 'basic'
DEBUG      Entering _get_cookie_and_crumb_basic()
DEBUG       loaded persistent cookie
DEBUG       reusing cookie
DEBUG       crumb = 'iUSsnoS0aBq'
DEBUG      Exiting _get_cookie_and_crumb_basic()
DEBUG     Exiting _get_cookie_and_crumb()
DEBUG     response code=404
DEBUG     toggling cookie strategy basic -> csrf
DEBUG     Entering _get_cookie_and_crumb()
DEBUG      cookie_mode = 'csrf'
DEBUG      Entering _get_crumb_csrf()
DEBUG       loaded persistent cookie
DEBUG      Exiting _get_crumb_csrf()
DEBUG      toggling cookie strategy csrf -> basic
DEBUG      Entering _get_cookie_and_crumb_basic()
DEBUG       loaded persistent cookie
DEBUG       reusing cookie
DEBUG       crumb = 'iUSsnoS0aBq'
DEBUG      Exiting _get_cookie_and_crumb_basic()
DEBUG     Exiting _get_cookie_and_crumb()
DEBUG     response code=404
DEBUG    Exiting get()

and then a 404 Client Error: Not Found for url: https://query2.finance.yahoo.com/v10/finance/quoteSummary/US0079031078?modules=financialData%2CquoteType%2CdefaultKeyStatistics%2CassetProfile%2CsummaryDetail&ssl=true&crumb=iUSsnoS0aBq

Installed using: pip install git+https://github.com/ranaroussi/yfinance.git@hotfix/cookie-fallback-strategy --no-cache-dir

ValueRaider commented 11 months ago

@SteynGuelen I've pushed a commit to hopefully log why strategy is toggling. Instead of posting below, can you edit with new log output? Keep thread tidy.

ValueRaider commented 11 months ago

Better, but still an unexplained decision in log here:

DEBUG      Exiting _get_crumb_csrf()
DEBUG      toggling cookie strategy csrf -> basic

Edit: I might have seen why. New commit to test.

Are you in USA?

SteynGuelen commented 11 months ago

New log:

DEBUG    get_raw_json(): https://query2.finance.yahoo.com/v10/finance/quoteSummary/US0079031078
DEBUG    Entering get()
DEBUG     url=https://query2.finance.yahoo.com/v10/finance/quoteSummary/US0079031078
DEBUG     params={'modules': 'financialData,quoteType,defaultKeyStatistics,assetProfile,summaryDetail', 'ssl': 'true'}
DEBUG     Entering _get_cookie_and_crumb()
DEBUG      cookie_mode = 'basic'
DEBUG      Entering _get_cookie_and_crumb_basic()
DEBUG       fetched basic cookie = <Cookie A3=d=AQABBETubmUCEHE-FNbKOtfHiHdUQrT481EFEgEBAQE_cGV4Ze2NzSMA_eMAAA&S=AQAAAobVCwUVwY-NWWfFQiFHgL8 for .yahoo.com/>
DEBUG       reusing cookie
DEBUG       crumb = 'QJaLmgXtj29'
DEBUG      Exiting _get_cookie_and_crumb_basic()
DEBUG     Exiting _get_cookie_and_crumb()       
DEBUG     response code=404
DEBUG     toggling cookie strategy basic -> csrf
DEBUG     Entering _get_cookie_and_crumb()      
DEBUG      cookie_mode = 'csrf'
DEBUG      Entering _get_crumb_csrf()
DEBUG       csrfToken = UrlLPz_5xA_LlWjV7sl--V_zmyI9gjoo
DEBUG       sessionId='3_cc-session_14476d26-3dcc-44ca-91ac-5ef206c332a3
DEBUG       crumb = 'QJaLmgXtj29'
DEBUG      Exiting _get_crumb_csrf()     
DEBUG     Exiting _get_cookie_and_crumb()
DEBUG     response code=404
DEBUG    Exiting get()

Europe based. Can try with a USA VPN later today.

ValueRaider commented 11 months ago

Interesting you're in Europe and neither strategy works. Only issue with CSRF I'm aware of is doesn't work in USA (then the basic strategy should work).

bot-unit commented 11 months ago

@ValueRaider It's not crumb error. It's error in base class. At the file base.py lines 73 - 76 

self._analysis = Analysis(self._data, ticker)
self._holders = Holders(self._data, ticker)
self._quote = Quote(self._data, ticker)
self._fundamentals = Fundamentals(self._data, ticker)

change to

self._analysis = Analysis(self._data, self.ticker)
self._holders = Holders(self._data, self.ticker)
self._quote = Quote(self._data, self.ticker)
self._fundamentals = Fundamentals(self._data, self.ticker)
SteynGuelen commented 11 months ago

Can confirm, this fix from @bot-unit works as intended.

DEBUG    get_raw_json(): https://query2.finance.yahoo.com/v10/finance/quoteSummary/AMD
DEBUG    Entering get()
DEBUG     url=https://query2.finance.yahoo.com/v10/finance/quoteSummary/AMD
DEBUG     params={'modules': 'financialData,quoteType,defaultKeyStatistics,assetProfile,summaryDetail', 'ssl': 'true'}
DEBUG     Entering _get_cookie_and_crumb()
DEBUG      cookie_mode = 'basic'
DEBUG      Entering _get_cookie_and_crumb_basic()
DEBUG       loaded persistent cookie
DEBUG       reusing cookie
DEBUG       crumb = 'QJaLmgXtj29'
DEBUG      Exiting _get_cookie_and_crumb_basic()
DEBUG     Exiting _get_cookie_and_crumb()
DEBUG     response code=200
DEBUG    Exiting get()
DEBUG    Entering get()
DEBUG     url=https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/AMD?symbol=AMD&type=trailingPegRatio&period1=1686009600&period2=1701820800
DEBUG     params=None
DEBUG     Entering _get_cookie_and_crumb()
DEBUG      cookie_mode = 'basic'
DEBUG      Entering _get_cookie_and_crumb_basic()
DEBUG       reusing cookie
DEBUG       reusing crumb
DEBUG      Exiting _get_cookie_and_crumb_basic()
DEBUG     Exiting _get_cookie_and_crumb()
DEBUG     response code=200
DEBUG    Exiting get()
{'address1': '2485 Augustine Drive', 'city': 'Santa Clara', 'state': 'CA', 'zip': '95054', 'country': 'United States', 'phone': '408 749 4000', 'website': 'https://www.amd.com', 'industry': 'Semiconductors', 'industryKey': 'semiconductors', 'industryDisp': 'Semiconductors', 'sector': 'Technology', 
'sectorKey': 'technology', 'sectorDisp': 'Technology', 'longBusinessSummary': 'Advanced Micro Devices, Inc. operates as a semiconductor company worldwide. It operates in four segments: Data Center, Client, Gaming, and Embedded segments. The company offers x86 microprocessors and graphics processing units (GPUs) as an accelerated processing unit, chipsets, data center, and professional GPUs; and embedded processors, and semi-custom system-on-chip (SoC) products, microprocessor and SoC development services and technology, data processing unites, field programmable gate arrays (FPGA), and adaptive SoC products. It also provides processors under the AMD Ryzen, AMD Ryzen PRO, Ryzen Threadripper, Ryzen Threadripper PRO, AMD Athlon, AMD Athlon PRO, and AMD PRO A-Series brand names; graphics under the AMD Radeon graphics and AMD Embedded Radeon graphics; and professional graphics under the AMD Radeon Pro graphics brand name. In addition, the company offers data center graphics under the Radeon Instinct and Radeon PRO V-series brands, as well as servers under the AMD Instinct accelerators brand; server microprocessors under the AMD EPYC brands; embedded processor solutions under the AMD Athlon, AMD Geode, AMD Ryzen, AMD EPYC, AMD R-Series, and G-Series brands; FPGA products under the Virtex-6, Virtex-7, Virtex UltraScale+, Kintex-7, Kintex UltraScale, Kintex UltraScale+, Artix-7, Artix UltraScale+, Spartan-6, and Spartan-7 brands; adaptive SOCs under the Zynq-7000, Zynq UltraScale+ MPSoC, Zynq UltraScale+ RFSoCs, Versal HBM, Versal Premium, Versal Prime, Versal AI Core, Versal AI Edge, Vitis, and Vivado brands; and compute and network acceleration board products under the Alveo brand. It serves original equipment and design manufacturers, public cloud service providers, system integrators, independent distributors, online and brick and mortar retailers, and add-in-board manufacturers through its direct sales force, independent distributors, and sales representatives. The company was incorporated in 1969 and is headquartered in Santa Clara, California.', 'fullTimeEmployees': 25000, 'companyOfficers': [{'maxAge': 1, 'name': 'Dr. Lisa T. Su Ph.D.', 'age': 52, 'title': 'Chair & CEO', 'yearBorn': 1970, 'fiscalYear': 2022, 'totalPay': 2120679, 'exercisedValue': 61926388, 'unexercisedValue': 84294872}, {'maxAge': 1, 'name': 'Mr. Victor  Peng', 'age': 62, 'title': 'President', 'yearBorn': 1960, 'fiscalYear': 2022, 'totalPay': 6091409, 'exercisedValue': 0, 'unexercisedValue': 0}, {'maxAge': 1, 'name': 'Mr. Mark D. Papermaster', 'age': 60, 'title': 'CTO and Executive VP of Technology & Engineering', 'yearBorn': 1962, 'fiscalYear': 2022, 'totalPay': 1140933, 'exercisedValue': 4836541, 'unexercisedValue': 22460436}, {'maxAge': 1, 'name': 'Mr. Paul Darren Grasby', 'age': 52, 'title': 'Executive Vice President of Strategic Partnerships & President EMEA', 'yearBorn': 1970, 'fiscalYear': 2022, 'totalPay': 921473, 'exercisedValue': 1077205, 'unexercisedValue': 432646}, {'maxAge': 1, 'name': 'Ms. Jean X. Hu', 'age': 59, 'title': 'Executive VP, CFO & Treasurer', 'yearBorn': 1963, 'fiscalYear': 2022, 'exercisedValue': 0, 'unexercisedValue': 0}, {'maxAge': 1, 'name': 'Mr. Keivan  Keshvari', 'title': 'Senior Vice President of Global Operations', 'fiscalYear': 
2022, 'exercisedValue': 0, 'unexercisedValue': 0}, {'maxAge': 1, 'name': 'Ms. Darla M. Smith', 'age': 56, 'title': 'Corporate VP & Chief Accounting Officer', 'yearBorn': 1966, 'fiscalYear': 2022, 'exercisedValue': 0, 'unexercisedValue': 0}, {'maxAge': 1, 'name': 'Mr. Hasmukh  Ranjan', 'title': 'Senior VP & Chief Information Officer', 'fiscalYear': 2022, 'exercisedValue': 0, 'unexercisedValue': 0}, {'maxAge': 1, 'name': 'Mr. Harry A. Wolin', 'age': 59, 'title': 'Senior VP, General Counsel & Corporate Secretary', 'yearBorn': 1963, 'fiscalYear': 2022, 'totalPay': 961942, 'exercisedValue': 9188269, 'unexercisedValue': 9460937}, {'maxAge': 1, 'name': 'Ms. Ruth  Cotter', 'title': 'Senior Vice President of Marketing, Communications & Human Resources', 'fiscalYear': 2022, 'exercisedValue': 0, 'unexercisedValue': 0}], 'auditRisk': 8, 'boardRisk': 5, 'compensationRisk': 7, 'shareHolderRightsRisk': 3, 'overallRisk': 5, 'governanceEpochDate': 1701388800, 'compensationAsOfEpochDate': 1672444800, 'maxAge': 86400, 'priceHint': 2, 'previousClose': 121.39, 'open': 119.75, 'dayLow': 116.37, 'dayHigh': 119.9, 'regularMarketPreviousClose': 121.39, 'regularMarketOpen': 119.75, 'regularMarketDayLow': 116.37, 'regularMarketDayHigh': 119.9, 'exDividendDate': 798940800, 'payoutRatio': 0.0, 'beta': 1.674, 'trailingPE': 1077.909, 'forwardPE': 31.959568, 'volume': 39807240, 'regularMarketVolume': 39807240, 'averageVolume': 52022411, 'averageVolume10days': 38406660, 'averageDailyVolume10Day': 38406660, 'bid': 118.0, 'ask': 118.1, 'bidSize': 800, 'askSize': 1200, 'marketCap': 191549833216, 'fiftyTwoWeekLow': 60.05, 'fiftyTwoWeekHigh': 132.83, 'priceToSalesTrailing12Months': 8.663101, 'fiftyDayAverage': 109.2496, 'twoHundredDayAverage': 104.4007, 'trailingAnnualDividendRate': 0.0, 'trailingAnnualDividendYield': 0.0, 'currency': 'USD', 'enterpriseValue': 188626698240, 'profitMargins': 0.0094099995, 'floatShares': 1605385868, 'sharesOutstanding': 1615500032, 'sharesShort': 40903051, 'sharesShortPriorMonth': 31516655, 'sharesShortPreviousMonthDate': 1697155200, 'dateShortInterest': 1700006400, 'sharesPercentSharesOut': 0.0253, 'heldPercentInsiders': 0.00477, 'heldPercentInstitutions': 0.74466, 'shortRatio': 0.69, 'shortPercentOfFloat': 0.0255, 'impliedSharesOutstanding': 1615500032, 'bookValue': 34.037, 'priceToBook': 3.4835622, 'lastFiscalYearEnd': 1672444800, 'nextFiscalYearEnd': 1703980800, 'mostRecentQuarter': 1696032000, 'earningsQuarterlyGrowth': 3.53, 'netIncomeToCommon': 208000000, 'trailingEps': 0.11, 'forwardEps': 3.71, 'pegRatio': 4.67, 'lastSplitFactor': '2:1', 'lastSplitDate': 966902400, 'enterpriseToRevenue': 8.531, 'enterpriseToEbitda': 61.926, '52WeekChange': 0.68734884, 'SandP52WeekChange': 0.15947187, 'exchange': 'NMS', 'quoteType': 'EQUITY', 'symbol': 'AMD', 'underlyingSymbol': 'AMD', 'shortName': 'Advanced Micro Devices, Inc.', 'longName': 'Advanced Micro Devices, Inc.', 'firstTradeDateEpochUtc': 322151400, 'timeZoneFullName': 'America/New_York', 'timeZoneShortName': 'EST', 'uuid': '48af4341-f745-363f-945f-a838eeabb062', 'messageBoardId': 'finmb_168864', 'gmtOffSetMilliseconds': -18000000, 'currentPrice': 118.57, 'targetHighPrice': 200.0, 'targetLowPrice': 60.0, 'targetMeanPrice': 130.43, 'targetMedianPrice': 130.0, 'recommendationMean': 2.0, 
'recommendationKey': 'buy', 'numberOfAnalystOpinions': 40, 'totalCash': 5784999936, 'totalCashPerShare': 3.581, 'ebitda': 3046000128, 'totalDebt': 2862000128, 'quickRatio': 1.421, 'currentRatio': 2.188, 'totalRevenue': 22111000576, 'debtToEquity': 5.206, 'revenuePerShare': 13.697, 'returnOnAssets': -0.00083, 'returnOnEquity': 0.0038, 'grossProfits': 12051000000, 'freeCashflow': 2758500096, 'operatingCashflow': 1852999936, 'earningsGrowth': 3.5, 'revenueGrowth': 0.042, 'grossMargins': 0.50319, 'ebitdaMargins': 0.13776, 'operatingMargins': 0.03862, 'financialCurrency': 'USD', 'trailingPegRatio': 1.6627}
bot-unit commented 11 months ago

Pull request is #1760

But for future, it's better to have classmethod from_isin