atreadw1492 / yahoo_fin

Scrape stock price history from new (Spring 2017) Yahoo Finance layout
MIT License
285 stars 125 forks source link

get_income_statement: string indices must be integers #94

Open garroshub opened 1 year ago

garroshub commented 1 year ago

Under Stock_info:

image

Can not get data, I checked line by line, and when running json_info["incomeStatementHistory"]["incomeStatementHistory"], it will report must be integers.

Then I checked _parse_json function: image The function can not get the quotesummarystore part. It says they must be integers, and cannot use string indices to get them.

Does anyone meet a similar issue?

dkivi commented 1 year ago

Same issue here with for example:

aapl_earnings_hist = si.get_earnings_history("aapl")

xyzitfinance commented 1 year ago

Hi,

Seems that the issue is not related only to some functions.

Seems that is something in the api response that is coming from Yahoo Finance.

Let's take the following example:

With the help of __parse_earningsjson(url)**** we retrive information from Yahoo Finance.

image

the problem: is in the api response because key = "stores" is somehow encrypted.

image

In order to figure out if is something related to yahoo_fin (expected to not be since no updates happened) I checked the information in Postman and run an GET URL API:

image

Seems that Yahoo Fin is starting to encrypt some information (maybe I;m wrong).

garroshub commented 1 year ago

Hi,

Seems that the issue is not related only to some functions.

Seems that is something in the api response that is coming from Yahoo Finance.

Let's take the following example:

  • in stock_info.py we are having the following function = _get_earningshistory .

With the help of __parse_earningsjson(url)**** we retrive information from Yahoo Finance.

image

the problem: is in the api response because key = "stores" is somehow encrypted.

image

In order to figure out if is something related to yahoo_fin (expected to not be since no updates happened) I checked the information in Postman and run an GET URL API:

image

Seems that Yahoo Fin is starting to encrypt some information (maybe I;m wrong).

If it is because Yahoo Finance encrypts some information, do you know of any user-side solutions?

xyzitfinance commented 1 year ago

Nope, but seems that not only yahoo_fin is having this issue.

Maybe to owner of the project is more familiar than me with the Yahoo Finance flow and with python.

danthings commented 1 year ago

Ok, so after some research, indeed the issue is coming from the Yahoo API.

I found that also other people that are using other libraries are having the some issues.

I don t know if is fine to specify the library but somehow they succeed to fix it.

The fix relates to the fact that they are decrypting the information with some AES parser & cipher.

I m not familiar to much with python, but in their cases seems to work and I have no idea if this library is still updated or not

garroshub commented 1 year ago

Ok, so after some research, indeed the issue is coming from the Yahoo API.

I found that also other people that are using other libraries are having the some issues.

I don t know if is fine to specify the library but somehow they succeed to fix it.

The fix relates to the fact that they are decrypting the information with some AES parser & cipher.

I m not familiar to much with python, but in their cases seems to work and I have no idea if this library is still updated or not

I am not familiar with AES parser and cypher. Perhaps you could provide the links you referenced that address similar issues.

lrauhockey commented 1 year ago

here is the pull request from yfinance that has fixed the encryption issue - https://github.com/ranaroussi/yfinance/pull/1253

sonso-1 commented 1 year ago

here is the pull request from yfinance that has fixed the encryption issue - ranaroussi/yfinance#1253

I was experiencing the issue when calling get_earnings_for_date(). Found a workaround using a snippet of the fix from yfinance.

Below is the workaround for my use case. This is not a proper fix, only posting it here for real developers to come up with a proper fix and update the yahoo_fin project. Until then, this workaround may work for others as well.

  1. Replace the stock_info.py in your python project (...\Lib\site-packages\yahoo_fin\stock_info.py) with the attached updated version
  2. Install additional python modules to handle the decryption import hashlib from base64 import b64decode from cryptography.hazmat.primitives import padding from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes

"changelog"

stock_info.py.txt

collinL33T commented 1 year ago

Just move all your stuff to yfinance, this dev gave up on this project already. The module has not been updated in almost 2 yrs. Since the Yahoo API issue occurred last week, yfinance has already issued out 2 patches to remedy the same issue we are requesting to fix right now.