search

ranaroussi / yfinance

Download market data from Yahoo! Finance's API
https://aroussi.com/post/python-yahoo-finance
Apache License 2.0
13k stars 2.3k forks source link

.info returns 404 on all tickers #1729

Closed ssvetli closed 10 months ago

ssvetli commented 11 months ago

@ValueRaider hijacking top post

Describe bug

Call to .info returns 404 error on all tickers. Example url: https://query2.finance.yahoo.com/v6/finance/quoteSummary/SPY?modules=financialData&modules=quoteType&modules=defaultKeyStatistics&modules=assetProfile&modules=summaryDetail&ssl=true Using browsers on multiple connections/devices, yahoo output "Will be right back... Our engineers are working quickly to resolve the issue."

Data could have moved to new link

Simple code that reproduces your problem

import yfinance as yf yf_ticker=yf.Ticker("SPY") yf_ticker.info

Debug log

HTTPError: 404 Client Error: Not Found for url: https://query2.finance.yahoo.com/v6/finance/quoteSummary/SPY?modules=financialData&modules=quoteType&modules=defaultKeyStatistics&modules=assetProfile&modules=summaryDetail&ssl=true

Bad data proof

No response

yfinance version

0.2.31

Python version

No response

Operating system

No response

brld commented 10 months ago

Seems like the endpoint fix was also broken as of today. Trying to use the https://e1.ycpi.vip.dca.yahoo.com/ endpoint from Lucas' fix returns a 404 now.

HTTPError: 404 Client Error: Not Found for url: https://e1.ycpi.vip.dca.yahoo.com/v6/finance/quoteSummary/AAPL?modules=financialData&modules=quoteType&modules=defaultKeyStatistics&modules=assetProfile&modules=summaryDetail&ssl=true

silentsight commented 10 months ago

I think @zq99 is on to something. His approach works for me. After appending the crumb from a request to: https://query2.finance.yahoo.com/v1/test/getcrumb to this URL: https://query2.finance.yahoo.com/v10/finance/quoteSummary/AAPL?module=summaryDetail&ssl=true&crumb=<crumbfrompreviouscall> I get a valid result. Without having any modifications in /etc/hosts. I should say that I am in the US, but if I use the v6 API and an unmodified hosts file I still get 404 errors.

I have been having a brief look at the code. The 'quoteSummary' v6 URL is declared in the module quote.py.

However, when I was looking at the version history of this module, it seems that the original URL was actually pointing to v10 rather than v6.

A change from v10 to v6 was made on Jul 13th with commit 509a109

So maybe there was some kind of issue with the v10 url.

So I followed some crumbs and a reddit user, this is a workaround (not sure if it covers all cases):

Changed line 24 of quote.py _BASICURL to _BASICURL = "https://query2.finance.yahoo.com/v8/finance/chart/" And ln 603 & 605 changed result["quoteSummary"] with result["chart"] same for result.get("quoteSummary" for result.get("chart"

Not sure if it will help using chart instead of quote

wowo commented 10 months ago

@silentsight I'm afraid it won't help with ticker.info issue

StanLackey commented 10 months ago

It stopped working for me Oct 31. I am in the US (NY). fast_info does not work either. Does anyone know, is this a known issue or maybe something I need to do?

ValueRaider commented 10 months ago

Please move fast_info discussion to a separate Issue / Discussion. If what you're actually talking about is the branch feature/cookie-and-crumb, then post in #1657.

I'm not unhappy, just trying to maintain order, otherwise we get a single thread with 200+ comments of interleaved conversations. That's a lot to read before replying.

I will lock this for a short while to give people time to read.

patfla commented 10 months ago

fast_info vs info seemed worth investigating. Found that there was only one instance of .info() in my program and I was able to replace that with something else.

Elsewhere, things like stock.history(), to get actual prices (where type stock = yfinance.ticker.Ticker), still works and my program is running again. It has a list of all members of QQQ and retrieves intraday prices.

So I take it that only parts of the API aren't working - which maybe I should have inferred from the topic title.

brian-dm100 commented 10 months ago

I'm not in the US, using this url the info is returned. Perhaps the base_url and ticker.py url need changing.

https://query2.finance.yahoo.com/v7/finance/options/bt-a.l

{"optionChain":{"result":[{"underlyingSymbol":"BT-A.L","expirationDates":[],"strikes":[],"hasMiniOptions":false,"quote":{"language":"en-US","region":"US","quoteType":"EQUITY","typeDisp":"Equity","quoteSourceName":"Delayed Quote","triggerable":false,"customPriceAlertConfidence":"LOW","currency":"GBp","exchangeTimezoneShortName":"GMT","gmtOffSetMilliseconds":0,"esgPopulated":false,"regularMarketChangePercent":-1.463416,"regularMarketPrice":111.1,"messageBoardId":"finmb_171577","exchangeTimezoneName":"Europe/London","market":"gb_market","exchange":"LSE","shortName":"BT GROUP PLC ORD 5P","longName":"BT Group plc","marketState":"POSTPOST","tradeable":false,"cryptoTradeable":false,"priceHint":2,"regularMarketChange":-1.6500015,"regularMarketTime":1698860724,"regularMarketDayHigh":113.45,"regularMarketDayRange":"111.0 - 113.45","regularMarketDayLow":111.0,"regularMarketVolume":32525515,"regularMarketPreviousClose":112.75,"bid":110.1,"ask":125.0,"bidSize":0,"askSize":0,"fullExchangeName":"LSE","financialCurrency":"GBP","regularMarketOpen":113.1,"averageDailyVolume3Month":30413774,"averageDailyVolume10Day":32775872,"fiftyTwoWeekLowChange":88.118,"fiftyTwoWeekLowChangePercent":3.8342178,"fiftyTwoWeekRange":"22.982 - 165.49","fiftyTwoWeekHighChange":-54.390007,"fiftyTwoWeekHighChangePercent":-0.32866037,"fiftyTwoWeekLow":22.982,"fiftyTwoWeekHigh":165.49,"fiftyTwoWeekChangePercent":-11.741686,"earningsTimestamp":1698908400,"earningsTimestampStart":1698908400,"earningsTimestampEnd":1698908400,"trailingAnnualDividendRate":0.077,"trailingPE":5.8473682,"dividendRate":0.08,"trailingAnnualDividendYield":6.8292685E-4,"dividendYield":6.83,"epsTrailingTwelveMonths":0.19,"epsForward":18.45,"epsCurrentYear":18.24,"priceEpsCurrentYear":6.0910087,"sharesOutstanding":9943310336,"bookValue":1.49,"fiftyDayAverage":115.447,"fiftyDayAverageChange":-4.347,"fiftyDayAverageChangePercent":-0.037653644,"twoHundredDayAverage":131.6465,"twoHundredDayAverageChange":-20.546501,"twoHundredDayAverageChangePercent":-0.15607329,"marketCap":11047018496,"forwardPE":0.0602168,"priceToBook":74.56376,"sourceInterval":15,"exchangeDataDelayedBy":15,"averageAnalystRating":"2.4 - Buy","firstTradeDateMilliseconds":583743600000,"symbol":"BT-A.L"},"options":[]}],"error":null}}

LordByron1788 commented 10 months ago

Please move fast_info discussion to a separate Issue / Discussion. If what you're actually talking about is the branch feature/cookie-and-crumb, then post in #1657.

I'm not unhappy, just trying to maintain order, otherwise we get a single thread with 200+ comments of interleaved conversations. That's a lot to read before replying.

I will lock this for a short while to give people time to read.

@ValueRaider If you had not hijacked the first post in this issue to talk about 'fast_info', then if would not have had a single mention in this thread.

fast_info has nothing to do with this issue, and you should not have been mentioned it.

Ticker.fast_info has subset of Ticker.info

Your comment is incorrect. Please remove it, to avoid confusing people further. I suggest you read and understand the codebase before changing other people's comments regarding this issue.

brian-dm100 commented 10 months ago

yesterday it was working fine with a US VPN, today is not. Locally it is not working either and am based in the US/EAST

The page is genuinely under maintenance I guess https://query2.finance.yahoo.com/v6/finance/quoteSummary/AAPL?modules=financialData&modules=quoteType&modules=defaultKeyStatistics&modules=assetProfile&modules=summaryDetail&ssl=true

That message is the same as 10 days ago.

The data is returned using the modified URL ...

https://query2.finance.yahoo.com/v7/finance/options/AAPL?modules=financialData&modules=quoteType&modules=defaultKeyStatistics&modules=assetProfile&modules=summaryDetail&ssl=true

ValueRaider commented 10 months ago

@LordByron1788 Are you dumb? You literally posted after someone showing info contains price-related data. I assure you that fast_info is a subset of info - I wrote it. I grant you it's a small subset and some key names different because it was rushed out in Spring last time info failed, but it helped a lot of users.

brld commented 10 months ago

The data is returned using the modified URL ...

https://query2.finance.yahoo.com/v7/finance/options/AAPL?modules=financialData&modules=quoteType&modules=defaultKeyStatistics&modules=assetProfile&modules=summaryDetail&ssl=true

Much of the data appears to be similar but there are still some absences. sector industry and news data are all missing and possibly others as well.

MackieBlackburn commented 10 months ago

The main things I was using from info were "yield" and "beta3Year", which are not in .fast_info. As mentioned by @brian-dm100, this URL has yield but not beta: https://query2.finance.yahoo.com/v7/finance/options/bt-a.l

Is there another source of beta info?

psychoz971 commented 10 months ago

Hi, Here a script I wrote to fix the problem without the yfinance package in my own project. If a developer want to internalize it in yfinance be free:

import requests
import urllib.parse
class YFinance:
    user_agent_key = "User-Agent"
    user_agent_value = ("Mozilla/5.0 (Windows NT 6.1; Win64; x64) "
                        "AppleWebKit/537.36 (KHTML, like Gecko) "
                        "Chrome/58.0.3029.110 Safari/537.36")

    def __init__(self, ticker):
        self.yahoo_ticker = ticker

    def __str__(self):
        return self.yahoo_ticker

    def _get_yahoo_cookie(self):
        cookie = None

        headers = {self.user_agent_key: self.user_agent_value}
        response = requests.get("https://fc.yahoo.com",
                                headers=headers,
                                allow_redirects=True)

        if not response.cookies:
            raise Exception("Failed to obtain Yahoo auth cookie.")

        cookie = list(response.cookies)[0]

        return cookie

    def _get_yahoo_crumb(self, cookie):
        crumb = None

        headers = {self.user_agent_key: self.user_agent_value}

        crumb_response = requests.get(
            "https://query1.finance.yahoo.com/v1/test/getcrumb",
            headers=headers,
            cookies={cookie.name: cookie.value},
            allow_redirects=True,
        )
        crumb = crumb_response.text

        if crumb is None:
            raise Exception("Failed to retrieve Yahoo crumb.")

        return crumb

    @property
    def info(self):
        # Yahoo modules doc informations :
        # https://cryptocointracker.com/yahoo-finance/yahoo-finance-api
        cookie = self._get_yahoo_cookie()
        crumb = self._get_yahoo_crumb(cookie)
        info = {}
        ret = {}

        headers = {self.user_agent_key: self.user_agent_value}

        yahoo_modules = ("summaryDetail,"
                         "financialData,"
                         "indexTrend,"
                         "defaultKeyStatistics")

        url = ("https://query1.finance.yahoo.com/v10/finance/"
               f"quoteSummary/{self.yahoo_ticker}"
               f"?modules={urllib.parse.quote_plus(yahoo_modules)}"
               f"&ssl=true&crumb={urllib.parse.quote_plus(crumb)}")

        info_response = requests.get(url,
                                     headers=headers,
                                     cookies={cookie.name: cookie.value},
                                     allow_redirects=True)

        info = info_response.json()
        info = info['quoteSummary']['result'][0]

        for mainKeys in info.keys():
            for key in info[mainKeys].keys():
                if isinstance(info[mainKeys][key], dict):
                    try:
                        ret[key] = info[mainKeys][key]['raw']
                    except (KeyError, TypeError):
                        pass
                else:
                    ret[key] = info[mainKeys][key]

        return ret

I'm using the API v10 and only get values from this modules : "summaryDetail," "financialData," "indexTrend," "defaultKeyStatistics".

If you need more modules add it in the yahoo_modules

brld commented 10 months ago

Hi, Here a script I wrote to fix the problem without the yfinance package in my own project. If a developer want to internalize it in yfinance be free: ... I'm using the API v10 and only get values from this modules : "summaryDetail," "financialData," "indexTrend," "defaultKeyStatistics".

If you need more modules add it in the yahoo_modules

Just tried it and it seems to be a true and working fix. I don't know enough about the codebase but I think it should work OOTB if integrated.

gene-git commented 10 months ago

@psychoz971 question - using your code snippet on ticker='^IXIC' is there a way to get current value of index?

brld commented 10 months ago

@psychoz971 question - using your code snippet on ticker='^IXIC' is there a way to get current value of index?

This is what I used for a while before learning there was a currentPrice attribute for many symbols:

last = round(ticker.history(period='1d')['Close'].iloc[0], 2)

It also has the added benefit of not depending on the quoteSummary endpoint which has obviously broken multiple times recently as it uses history instead of info.

brian-dm100 commented 10 months ago

OK, lot's of comments and possibilities! Is there way to retrieve the company name?

sanyearng commented 10 months ago

👍

On Thu, Nov 2, 2023 at 3:25 PM prabhakaran2812 @.***> wrote:

@psychoz971 https://github.com/psychoz971 I tried below code. I am sure im missing something yf = YFinance('AAPL') yf.info()

I am getting below error https://localhost:8080/# in _get_yahoo_cookie(self) 15 16 headers = {self.user_agent_key: self.user_agent_value} ---> 17 response = requests.get("https://fc.yahoo.com/", 18 headers=headers, 19 allow_redirects=True)

NameError: name 'requests' is not defined

— Reply to this email directly, view it on GitHub https://github.com/ranaroussi/yfinance/issues/1729#issuecomment-1791419427, or unsubscribe https://github.com/notifications/unsubscribe-auth/A5QKGW24NI74BCOXW3KM2Y3YCPXTJAVCNFSM6AAAAAA6IUG7XOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJRGQYTSNBSG4 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

patfla commented 10 months ago

Company name: not that I can see. I'm printing out all the field names (data items) I find below. If PASS then that field wasn't, for example, of type dict and the field is unaccepted (not in the output).

Company name appears neither in the accepted nor unaccepted values. Not though that isn't in there somewhere in the API anyway - but not the code as it stands currently. As best I can determine.

[note prabhakaran2812, sanyearng: code needs import requests, urllib]

D:\Projects\QQQ>python yf2.py

d:\projects\qqq\yf2.py(5)() -> test = YFinance2('GOOG') (Pdb) c maxAge priceHint previousClose open dayLow dayHigh regularMarketPreviousClose regularMarketOpen regularMarketDayLow regularMarketDayHigh dividendRate PASS: dividendRate dividendYield PASS: dividendYield exDividendDate PASS: exDividendDate payoutRatio fiveYearAvgDividendYield PASS: fiveYearAvgDividendYield beta trailingPE forwardPE volume regularMarketVolume averageVolume averageVolume10days averageDailyVolume10Day bid ask bidSize askSize marketCap yield PASS: yield ytdReturn PASS: ytdReturn totalAssets PASS: totalAssets expireDate PASS: expireDate strikePrice PASS: strikePrice openInterest PASS: openInterest fiftyTwoWeekLow fiftyTwoWeekHigh priceToSalesTrailing12Months fiftyDayAverage twoHundredDayAverage trailingAnnualDividendRate trailingAnnualDividendYield navPrice PASS: navPrice currency fromCurrency toCurrency lastMarket coinMarketCapLink volume24Hr PASS: volume24Hr volumeAllCurrencies PASS: volumeAllCurrencies circulatingSupply PASS: circulatingSupply algorithm maxSupply PASS: maxSupply startDate PASS: startDate tradeable maxAge priceHint enterpriseValue forwardPE profitMargins floatShares sharesOutstanding sharesShort sharesShortPriorMonth sharesShortPreviousMonthDate dateShortInterest sharesPercentSharesOut heldPercentInsiders heldPercentInstitutions shortRatio shortPercentOfFloat PASS: shortPercentOfFloat beta impliedSharesOutstanding morningStarOverallRating PASS: morningStarOverallRating morningStarRiskRating PASS: morningStarRiskRating category bookValue priceToBook annualReportExpenseRatio PASS: annualReportExpenseRatio ytdReturn PASS: ytdReturn beta3Year PASS: beta3Year totalAssets PASS: totalAssets yield PASS: yield fundFamily fundInceptionDate PASS: fundInceptionDate legalType threeYearAverageReturn PASS: threeYearAverageReturn fiveYearAverageReturn PASS: fiveYearAverageReturn priceToSalesTrailing12Months PASS: priceToSalesTrailing12Months lastFiscalYearEnd nextFiscalYearEnd mostRecentQuarter earningsQuarterlyGrowth revenueQuarterlyGrowth PASS: revenueQuarterlyGrowth netIncomeToCommon trailingEps forwardEps pegRatio lastSplitFactor lastSplitDate enterpriseToRevenue enterpriseToEbitda 52WeekChange SandP52WeekChange lastDividendValue PASS: lastDividendValue lastDividendDate PASS: lastDividendDate lastCapGain PASS: lastCapGain annualHoldingsTurnover PASS: annualHoldingsTurnover maxAge symbol peRatio pegRatio estimates maxAge currentPrice targetHighPrice targetLowPrice targetMeanPrice targetMedianPrice recommendationMean recommendationKey numberOfAnalystOpinions totalCash totalCashPerShare ebitda totalDebt quickRatio currentRatio totalRevenue debtToEquity revenuePerShare returnOnAssets returnOnEquity grossProfits freeCashflow operatingCashflow earningsGrowth revenueGrowth grossMargins ebitdaMargins operatingMargins profitMargins financialCurrency marketCap = {'raw': 1601643872256, 'fmt': '1.6T', 'longFmt': '1,601,643,872,256'}

mykolaz commented 10 months ago

https://github.com/ranaroussi/yfinance/issues/1729#issuecomment-1792468854

Great solution!

Can confirm this update works off GCP's us-east-4 zone.

Thank you!

Jshen123 commented 10 months ago

Hi @ValueRaider @psychoz971, thank you both for your amazing work!

I have created a PR that integrates both of your solutions. It should be able to fairly reliably get the info data.

prabhakaran2812 commented 10 months ago

https://github.com/ranaroussi/yfinance/issues/1729#issuecomment-1790830481

@psychoz971 Works like a charm, Thank you. To the yfinance admins, if there is an option to say manipulate the BASE URL / URL constructs it will be great.

kuantumk commented 10 months ago

Hi, Here a script I wrote to fix the problem without the yfinance package in my own project. If a developer want to internalize it in yfinance be free:

class YFinance:
    user_agent_key = "User-Agent"
    user_agent_value = ("Mozilla/5.0 (Windows NT 6.1; Win64; x64) "
                        "AppleWebKit/537.36 (KHTML, like Gecko) "
                        "Chrome/58.0.3029.110 Safari/537.36")

    def __init__(self, ticker):
        self.yahoo_ticker = ticker

    def __str__(self):
        return self.yahoo_ticker

    def _get_yahoo_cookie(self):
        cookie = None

        headers = {self.user_agent_key: self.user_agent_value}
        response = requests.get("https://fc.yahoo.com",
                                headers=headers,
                                allow_redirects=True)

        if not response.cookies:
            raise Exception("Failed to obtain Yahoo auth cookie.")

        cookie = list(response.cookies)[0]

        return cookie

    def _get_yahoo_crumb(self, cookie):
        crumb = None

        headers = {self.user_agent_key: self.user_agent_value}

        crumb_response = requests.get(
            "https://query1.finance.yahoo.com/v1/test/getcrumb",
            headers=headers,
            cookies={cookie.name: cookie.value},
            allow_redirects=True,
        )
        crumb = crumb_response.text

        if crumb is None:
            raise Exception("Failed to retrieve Yahoo crumb.")

        return crumb

    @property
    def info(self):
        # Yahoo modules doc informations :
        # https://cryptocointracker.com/yahoo-finance/yahoo-finance-api
        cookie = self._get_yahoo_cookie()
        crumb = self._get_yahoo_crumb(cookie)
        info = {}
        ret = {}

        headers = {self.user_agent_key: self.user_agent_value}

        yahoo_modules = ("summaryDetail,"
                         "financialData,"
                         "indexTrend,"
                         "defaultKeyStatistics")

        url = ("https://query1.finance.yahoo.com/v10/finance/"
               f"quoteSummary/{self.yahoo_ticker}"
               f"?modules={urllib.parse.quote_plus(yahoo_modules)}"
               f"&ssl=true&crumb={urllib.parse.quote_plus(crumb)}")

        info_response = requests.get(url,
                                     headers=headers,
                                     cookies={cookie.name: cookie.value},
                                     allow_redirects=True)

        info = info_response.json()
        info = info['quoteSummary']['result'][0]

        for mainKeys in info.keys():
            for key in info[mainKeys].keys():
                if isinstance(info[mainKeys][key], dict):
                    try:
                        ret[key] = info[mainKeys][key]['raw']
                    except (KeyError, TypeError):
                        pass
                else:
                    ret[key] = info[mainKeys][key]

        return ret

I'm using the API v10 and only get values from this modules : "summaryDetail," "financialData," "indexTrend," "defaultKeyStatistics".

If you need more modules add it in the yahoo_modules

@psychoz971 Thanks! For some reason, "assetProfile" cannot work with other module in the same module list. But it can work if it's the only module in the list. Do you know why?

psychoz971 commented 10 months ago

@psychoz971 Thanks! For some reason, "assetProfile" cannot work with other module in the same module list. But it can work if it's the only module in the list. Do you know why?

@kuantumk everythink is good for me if I add "assetProfile". Here the script with all modules of the previews info definition, i only add assetProfile:

class YFinance:
    user_agent_key = "User-Agent"
    user_agent_value = ("Mozilla/5.0 (Windows NT 6.1; Win64; x64) "
                        "AppleWebKit/537.36 (KHTML, like Gecko) "
                        "Chrome/58.0.3029.110 Safari/537.36")

    def __init__(self, ticker):
        self.yahoo_ticker = ticker

    def __str__(self):
        return self.yahoo_ticker

    def _get_yahoo_cookie(self):
        cookie = None

        headers = {self.user_agent_key: self.user_agent_value}
        response = requests.get("https://fc.yahoo.com",
                                headers=headers,
                                allow_redirects=True)

        if not response.cookies:
            raise Exception("Failed to obtain Yahoo auth cookie.")

        cookie = list(response.cookies)[0]

        return cookie

    def _get_yahoo_crumb(self, cookie):
        crumb = None

        headers = {self.user_agent_key: self.user_agent_value}

        crumb_response = requests.get(
            "https://query1.finance.yahoo.com/v1/test/getcrumb",
            headers=headers,
            cookies={cookie.name: cookie.value},
            allow_redirects=True,
        )
        crumb = crumb_response.text

        if crumb is None:
            raise Exception("Failed to retrieve Yahoo crumb.")

        return crumb

    @property
    def info(self):
        # Yahoo modules doc informations :
        # https://cryptocointracker.com/yahoo-finance/yahoo-finance-api
        cookie = self._get_yahoo_cookie()
        crumb = self._get_yahoo_crumb(cookie)
        info = {}
        ret = {}

        headers = {self.user_agent_key: self.user_agent_value}

        yahoo_modules = ("financialData,"
                         "quoteType,"
                         "defaultKeyStatistics,"
                         "assetProfile,"
                         "summaryDetail")

        url = ("https://query1.finance.yahoo.com/v10/finance/"
               f"quoteSummary/{self.yahoo_ticker}"
               f"?modules={urllib.parse.quote_plus(yahoo_modules)}"
               f"&ssl=true&crumb={urllib.parse.quote_plus(crumb)}")

        info_response = requests.get(url,
                                     headers=headers,
                                     cookies={cookie.name: cookie.value},
                                     allow_redirects=True)

        info = info_response.json()
        info = info['quoteSummary']['result'][0]

        for mainKeys in info.keys():
            for key in info[mainKeys].keys():
                if isinstance(info[mainKeys][key], dict):
                    try:
                        ret[key] = info[mainKeys][key]['raw']
                    except (KeyError, TypeError):
                        pass
                else:
                    ret[key] = info[mainKeys][key]

        return ret
psychoz971 commented 10 months ago

@psychoz971 Works like a charm, Thank you. To the yfinance admins, if there is an option to say manipulate the BASE URL / URL constructs it will be great.

@prabhakaran2812 if you need it I think you can add new properties base_urland cookie_url and replace it in the YFinance class.

psychoz971 commented 10 months ago

@psychoz971 question - using your code snippet on ticker='^IXIC' is there a way to get current value of index?

@gene-git I make only the info endpoint to debug my own program and I'm thinking info doesn't have the currentPrice, sorry!

kuantumk commented 10 months ago

https://github.com/ranaroussi/yfinance/issues/1729#issuecomment-1792468854

@psychoz971 Many thanks! It's working today. So the code is fine and it might be the API issue yesterday.

mathindy commented 10 months ago

I'm trying to implement @psychoz971 YFinance class. Can somebody post a short example of some main program code that uses this?

mykolaz commented 10 months ago

I'm trying to implement @psychoz971 YFinance class. Can somebody post a short example of some main program code that uses this?

Just grab the main function from the Class and modify for out-of-class usage.

`USER_AGENT = 'User-Agent' USER_AGENT_VALUE = ("Mozilla/5.0 (Windows NT 6.1; Win64; x64) " "AppleWebKit/537.36 (KHTML, like Gecko) " "Chrome/58.0.3029.110 Safari/537.36")

def _get_yahoo_cookie():

 cookie = None

 headers = {USER_AGENT : USER_AGENT_VALUE}
 response = requests.get("https://fc.yahoo.com",
                            headers=headers,
                            allow_redirects=True)

 if not response.cookies:

      raise Exception("Failed to obtain Yahoo auth cookie.")

 cookie = list(response.cookies)[0]

 print ('Obtained cookie')

 return cookie

def _get_yahoo_crumb(cookie):

 crumb = None

 headers = {USER_AGENT: USER_AGENT_VALUE}

 crumb_response = requests.get(
        "https://query1.finance.yahoo.com/v1/test/getcrumb",
        headers=headers,
        cookies={cookie.name: cookie.value},
        allow_redirects=True,
    )

 crumb = crumb_response.text

 if crumb is None:

      raise Exception("Failed to get Yahoo crumb.")

 return crumb`

Or you can just use the class

client = YFinance('AAPL').info

mathindy commented 10 months ago

@mykolaz Thank you for this help. I figured out how to get .info to work with the cookies&crumbs fix but can no longer get the .Ticker method and associated .history.

brld commented 10 months ago

@mykolaz Thank you for this help. I figured out how to get .info to work with the cookies&crumbs fix but can no longer get the .Ticker method and associated .history.

The separate class that was provided only contains the endpoint for .info. If you want to use .history() then you just have to use the standard yfinance provided Ticker class which still works fine for that use case. Until the changes are implemented into the library, what I did (since I don't have enough knowledge of the codebase to make my own fork / pull request at the moment) is just make two Ticker classes and call history or info appropriately depending on which one I need.

StanLackey commented 10 months ago

I am managing to work around .info no longer working to some extent. From aIl the above activity and proposed workarounds I'm getting the impression that .info isn't going to be fixed. I am now using fast info and .history.
My question is, I'd like to retrieve quantities like dividend yield, capitalization, PE and target mean. I'm not a sophisticated enough python programmer to use the cookie and crumb related workarounds. Can I get those other quantities in a more straightforward way?

brian-dm100 commented 10 months ago

And the company name, if there's a way to retrieve that, that would be great.

patfla commented 10 months ago

Names, both long and short, seem to be in there now with the change above.

        yahoo_modules = ("financialData,"
                         "quoteType,"
                         "defaultKeyStatistics,"
                         "assetProfile,"
                         "summaryDetail")
mathindy commented 10 months ago

I'd love it if there was comment from a Yahoo engineer or somebody with knowledge about exactly what was changed/broken. My guess is that everyone who has imported the Python yfinance library now has broken code. If I manually load one of the API's URL then I see that Yahoo says "Will be right back... Thank you for your patience. Our engineers are working quickly to resolve the issue." but I worry that maybe Yahoo has killed specific yfinance features because maybe it was pissing off the companies who sell this data for a fee? My hope is that Yahoo engineers really are working on this and it will soon be back to the way it was.

prabhakaran2812 commented 10 months ago

@gene-git I make only the info endpoint to debug my own program and I'm thinking info doesn't have the currentPrice, sorry!

@gene-git @brld - Try this , there are mostly 3 values for quoteType - EQUITY, ETF, INDEX, if equity get currentPrice, if ETF get navPrice, if INDEX get previousClose

prabhakaran2812 commented 10 months ago

@prabhakaran2812 if you need it I think you can add new properties base_urland cookie_url and replace it in the YFinance class.

I got that, but I want yfinance to provide property overriding, which is not the current case. I can't do a = yf.Ticker('AAPL') ; a._BASEURL = https://xyz in yfinance

gene-git commented 10 months ago

@prabhakaran2812 - thanks but:

if INDEX get previousClose

Not what I need. I need current value of index not yesterdays close.

brian-dm100 commented 10 months ago

@prabhakaran2812 - thanks but:

if INDEX get previousClose

Not what I need. I need current value of index not yesterdays close.

Using yfinance, no fixes, you can still retrieve the current price. As far as I can ascertain, yfinance 0.2.31 and previous versions can no longer retrieve .info but other functions still work.

gene-git commented 10 months ago

@brian-dm100 Thanks - I'm trying to understand the code and how yahoo API works (or doesn't) - which yahoo modules and which api v11? v10? v7? something else maybe - can be used to retrieve current value of ^IXIC?

Separately, are you guys aware of decent docs of yahoo api ?

mykolaz commented 10 months ago

I am managing to work around .info no longer working to some extent. From aIl the above activity and proposed workarounds I'm getting the impression that .info isn't going to be fixed. I am now using fast info and .history. My question is, I'd like to retrieve quantities like dividend yield, capitalization, PE and target mean. I'm not a sophisticated enough python programmer to use the cookie and crumb related workarounds. Can I get those other quantities in a more straightforward way?

@mykolaz Thank you for this help. I figured out how to get .info to work with the cookies&crumbs fix but can no longer get the .Ticker method and associated .history.

The separate class that was provided only contains the endpoint for .info. If you want to use .history() then you just have to use the standard yfinance provided Ticker class which still works fine for that use case. Until the changes are implemented into the library, what I did (since I don't have enough knowledge of the codebase to make my own fork / pull request at the moment) is just make two Ticker classes and call history or info appropriately depending on which one I need.

@mykolaz Thank you for this help. I figured out how to get .info to work with the cookies&crumbs fix but can no longer get the .Ticker method and associated .history.

The separate class that was provided only contains the endpoint for .info. If you want to use .history() then you just have to use the standard yfinance provided Ticker class which still works fine for that use case. Until the changes are implemented into the library, what I did (since I don't have enough knowledge of the codebase to make my own fork / pull request at the moment) is just make two Ticker classes and call history or info appropriately depending on which one I need.

It is not that difficult of a workaround. Just use the snippet of code to declare functions that return cookie and crumb. Then use the cookie and crumb like this:

def _get_info(ticker):

     cookie = _get_yahoo_cookie()
     crumb = _get_yahoo_crumb(cookie)

     headers = {USER_AGENT: USER_AGENT_VALUE}

     # Initialized dictionaries
     info = {}
     ret = {}

     yahoo_modules = ("summaryDetail,"
                    "financialData,"
                    "indexTrend,"
                    "defaultKeyStatistics")

     url = ("https://query1.finance.yahoo.com/v10/finance/"
               f"quoteSummary/{ticker}"
               f"?modules={urllib.parse.quote_plus(yahoo_modules)}"
               f"&ssl=true&crumb={urllib.parse.quote_plus(crumb)}")

     info_response = requests.get(url,
                                   headers=headers,
                                   cookies={cookie.name: cookie.value},
                                   allow_redirects=True)

     info = info_response.json()
     info = info['quoteSummary']['result'][0]

     for mainKeys in info.keys():
          for key in info[mainKeys].keys():
               if isinstance(info[mainKeys][key], dict):
                    try:
                         ret[key] = info[mainKeys][key]['raw']
                    except (KeyError, TypeError):
                         pass
               else:
                    ret[key] = info[mainKeys][key]

     return ret

Then use the _get_info function in a variable that you used to do with yf.Ticker({Ticker}).info

ValueRaider commented 10 months ago

I think time for polling to see what works / doesn't work, for who and where.

ValueRaider commented 10 months ago

@psychoz971 solution fixes Ticker.info in USA - vote 👍 or 👎

ValueRaider commented 10 months ago

@psychoz971 solution fixes Ticker.info in Europe - vote 👍 or 👎

ValueRaider commented 10 months ago

@psychoz971 solution fixes Ticker.info in Asia - vote 👍 or 👎

francemur commented 10 months ago

Time zone other than Eastern Time might affect data consistency. So if the query is being made outside the US, this might affect results. Still, it's not working with a VPN. This one is not the same, but it works:

https://query1.finance.yahoo.com/v8/finance/chart/SPY?region=US&lang=en-US&includePrePost=false&interval=2m&useYfid=true&range=1d&corsDomain=finance.yahoo.com&.tsrc=finance

However, this one doesn't

https://query1.finance.yahoo.com/v8/finance/quoteSummary/SPY?modules=financialData&modules=quoteType&modules=defaultKeyStatistics&modules=assetProfile&modules=summaryDetail&ssl=true

ValueRaider commented 10 months ago

@patfla This info data isn't new.

ValueRaider commented 10 months ago

@arjune1987 You installed the broken version and are surprised it is broken? Maybe you can't help.

arjune1987 commented 10 months ago

@arjune1987 You installed the broken version and are surprised it is broken? Maybe you can't help.

sorry for my novice mistakes, I finally got it working - I did not understand the fix was in a different version. I successfully installed yfinance-0.2.31b2 and can confirm it is working - thankyou and I am sorry about my mistakes, this is why I offer $ because I am still newbie developer.

ValueRaider commented 10 months ago

yfinance-0.2.31b2 can't possibly fix this issue. It's basically same as latest official version.