search

ranaroussi / yfinance

Download market data from Yahoo! Finance's API
https://aroussi.com/post/python-yahoo-finance
Apache License 2.0
14.89k stars 2.44k forks source link

Feature Proposal: Support Sector and Industry data from Markets Tab #2058

Closed ericpien closed 2 months ago

ericpien commented 2 months ago

The following data is available on: https://finance.yahoo.com/sectors/:

Resolves:

Sample code:

import yfinance as yf
tech = yf.Sector('technology')
software = yf.Industry('software-infrastructure')

# Common information
tech.key
tech.name
tech.symbol
tech.ticker
tech.overview
tech.top_companies
tech.research_reports

# Sector information
tech.top_etfs
tech.top_mutual_funds
tech.industries

# Industry information
software.sector_key
software.sector_name
software.top_performing_companies
software.top_growth_companies

# Ticker to Sector and Industry
msft = yf.Ticker('MSFT')
tech = yf.Sector(msft.info.get('sectorKey'))
software = yf.Industry(msft.info.get('industryKey'))

# Sector and Industry to Ticker
tech_ticker = tech.ticker
tech_ticker.info
software_ticker = software.ticker
software_ticker.history()
sshcli commented 2 months ago

Thank you @ericpien 🙌

ValueRaider commented 2 months ago

I ran tech.symbol a few times, not many, and triggered a messy error:

ERROR    Failed to get sector data for 'technology' reason: 401 Client Error: Unauthorized for url: https://query1.finance.yahoo.com/v1/finance/sectors/technology?formatted=true&withReturns=true&lang=en-US&region=US&crumb=4WOcuFKaIqF
DEBUG    Got response: 
DEBUG    -------------
Traceback (most recent call last):
  File "/home/test/yfinance-ericpien/yfinance/domain/sector.py", line 53, in _fetch_and_parse
    result = self._fetch(self._query_url, self.proxy)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/test/yfinance-ericpien/yfinance/domain/domain.py", line 59, in _fetch
    result = self._data.get_raw_json(query_url, user_agent_headers=self._data.user_agent_headers, params=params_dict, proxy=proxy)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/test/yfinance-ericpien/yfinance/data.py", line 399, in get_raw_json
    response.raise_for_status()
  File "/home/test/.local/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://query1.finance.yahoo.com/v1/finance/sectors/technology?formatted=true&withReturns=true&lang=en-US&region=US&crumb=4WOcuFKaIqF

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/test/yfinance-ericpien/./test.py", line 12, in <module>
    print(tech.symbol)
          ^^^^^^^^^^^
  File "/home/test/yfinance-ericpien/yfinance/domain/domain.py", line 34, in symbol
    self._ensure_fetched(self._symbol)
  File "/home/test/yfinance-ericpien/yfinance/domain/domain.py", line 97, in _ensure_fetched
    self._fetch_and_parse()
  File "/home/test/yfinance-ericpien/yfinance/domain/sector.py", line 66, in _fetch_and_parse
    logger.debug(f" {data}")
                     ^^^^
UnboundLocalError: cannot access local variable 'data' where it is not associated with a value

Also, you can put hardcoded stuff like SECTOR_INDUSTY_MAPPING in const.py, keep the logic clean.

ericpien commented 2 months ago

@ValueRaider thanks for reviewing. I made the changes for const.py and the UnboundedLocalError portion of the log you shared.

I can't reproduce the HTTPError portion of the log. I ran the code below successfully which rules out the error being related to "running it a few times". Can you try again?

for s in const.SECTOR_INDUSTY_MAPPING:
    sector = yf.Sector(s)
    print(f'sector symbol: {sector.symbol}')
    for i in const.SECTOR_INDUSTY_MAPPING[s]:
        industry = yf.Industry(i)
        print(f'industry symbol: {industry.symbol}')
ValueRaider commented 2 months ago

Your change has improved the error - it still happens randomly but is cleaner. Probably related to my spam, I'm doing other fetches.

One more error:

tech = yf.Sector('technology')
tech.research_reports
  File "yfinance/domain/domain.py", line 54, in research_reports
    self._ensure_fetched(self._research_reports)
                         ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Sector' object has no attribute '_research_reports'. Did you mean: '_research_report'?
ericpien commented 2 months ago

Thanks. Corrected the param name and checked the other params are correct as well.

ValueRaider commented 2 months ago

I'm hoping to release in a week, I'm just finishing testing improvements to dividend repair.