oscar0812 / pyfinviz

A python package to scrape data from finviz.com
Other
36 stars 8 forks source link

Issue with Quote from pyfinviz.quote #7

Closed KamranAbbaszadeIbrahim closed 1 year ago

KamranAbbaszadeIbrahim commented 1 year ago

Hi,

The Quote object is not able to perform properly when I run next snippet of code from Readme

from pyfinviz.quote import Quote

quote = Quote(ticker="AMZN")

# available variables:
print(quote.exists)  # check if fetch was successful (STOCK may not exist)
print(quote.ticker)  # AMZN
print(quote.exchange)  # NASD
print(quote.company_name)  # Amazon.com, Inc.
print(quote.sectors)  # ['Consumer Cyclical', 'Internet Retail', 'USA']
print(quote.fundamental_df)  # Index    P/E EPS (ttm) Insider Own  ...  SMA50  SMA200     Volume  Change
print(quote.outer_ratings_df)  # 0   Nov-04-20     Upgrade  ...                Hold → Buy  $3360 → $4000
print(quote.outer_news_df)  # 0   Jan-04-21 10:20PM  ...                   Bloomberg
print(quote.income_statement_df)  # 1      12/31/2019  ...                    22.99206
print(quote.insider_trading_df)  # 0         WILKE JEFFREY A  ...  http://www.sec.gov/Archives/edgar/data/1018724...

getting next error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
[<ipython-input-6-8ccf082c50ac>](https://localhost:8080/#) in <module>
      1 from pyfinviz.quote import Quote
      2 
----> 3 quote = Quote(ticker="AMZN")
      4 
      5 # available variables:

1 frames
[/usr/local/lib/python3.7/dist-packages/pyfinviz/quote.py](https://localhost:8080/#) in __get_outer_news_df__(soup)
     46 
     47             prev_date = date_
---> 48             info_ = [date_, news_a.text, news_a['href'], news_from.text]
     49             outer_news_info.append({tags__[i]: info_[i] for i in range(0, len(tags__))})
     50 

AttributeError: 'NoneType' object has no attribute 'text'

I used the lates release Eight.

Oazubuko commented 1 year ago

I wrote a function

from pyfinviz.quote import inspect src =inspect.getsource(Quote) print(src)

to print the quote import module to the command line, which gives me

class Quote:

    @staticmethod
    def __get_outer_ratings_df__(soup):
        outer_ratings_table = soup.find('table', class_='fullview-ratings-outer')
        # might not have outer ratings
        if outer_ratings_table is None:
            return None

        outer_ratings_trs = outer_ratings_table.find_all('tr', recursive=False)

        outer_ratings_info = []
        tags__ = ['Date', 'Status', 'Outer', 'Rating', 'Price']
        for tr in outer_ratings_trs:
            o_tds_text = [td.text for td in tr.find_all('td')[1:]]
            outer_ratings_info.append({tags__[i]: o_tds_text[i] for i in range(0, len(tags__))})

        return pd.DataFrame(outer_ratings_info)

    @staticmethod
    def __get_outer_news_df__(soup):
        outer_news_table = soup.find('table', class_='fullview-news-outer')
        # might not have news
        if outer_news_table is None:
            return None

        outer_news_trs = outer_news_table.find_all('tr', recursive=False)

        outer_news_info = []
        tags__ = ['Date', 'Headline', 'URL', 'From']
        prev_date = None
        for tr_ in outer_news_trs:
            o_tds = tr_.find_all('td', recursive=False)
            if len(o_tds) < 2:
                continue
            news_a = o_tds[1].find('a')
            news_from = o_tds[1].find('span')
            date_ = o_tds[0].text.strip()
            if '-' not in date_:
                date_ = prev_date.split(' ')[0] + ' ' + date_

            prev_date = date_
            info_ = [date_, news_a.text, news_a['href'], news_from.text]
            outer_news_info.append({tags__[i]: info_[i] for i in range(0, len(tags__))})

        return pd.DataFrame(outer_news_info)

    @staticmethod
    def __get_XHR_requests__(ticker):
        s_u = 'https://finviz.com/api/statement.ashx?t=' + ticker + '&s='
        statement_dicts = {'income_statement': WebScraper.get_json(s_u + 'IA'),
                           'balance_sheet': WebScraper.get_json(s_u + 'BA'),
                           'cash_flow': WebScraper.get_json(s_u + 'CA')}

        # convert dict to dataframes
        # issue 2: KeyError: 'data'
        # solution: some tickers dont have XHR_request data, return None
        income_statement_df = None
        balance_sheet_df = None
        cash_flow_df = None

        if 'data' in statement_dicts['income_statement']:
            income_statement_df = pd.DataFrame.from_dict(statement_dicts['income_statement']['data'])

        if 'data' in statement_dicts['balance_sheet']:
            balance_sheet_df = pd.DataFrame.from_dict(statement_dicts['balance_sheet']['data'])

        if 'data' in statement_dicts['cash_flow']:
            cash_flow_df = pd.DataFrame.from_dict(statement_dicts['cash_flow']['data'])

        return income_statement_df, balance_sheet_df, cash_flow_df

    @staticmethod
    def __get_insider_trading_df__(soup):
        insider_trading_table = soup.find('table', class_="body-table")
        # might not have insider trading
        if insider_trading_table is None:
            return None

        insider_trading_trs = insider_trading_table.find_all('tr', recursive=False)
        insider_trading_info = []

        tags__ = [td.text.strip() for td in insider_trading_trs[0].find_all('td')]
        tags__.insert(1, 'Insider History URL')
        tags__.append('SEC Form 4 URL')
        for tr in insider_trading_trs[1:]:
            tds = tr.find_all('td', recursive=False)
            info_ = [td.text.strip() for td in tds]
            info_.insert(1, 'https://finviz.com/' + tds[0].find('a')['href'])
            info_.append(tds[len(tds) - 1].find('a')['href'])

            insider_trading_info.append({tags__[i]: info_[i] for i in range(0, len(tags__))})

        return pd.DataFrame(insider_trading_info)

    def __init__(self, ticker="FB"):
        main_url = 'https://finviz.com/quote.ashx?t=' + ticker
        self.soup = WebScraper.get_soup(main_url)

        # base info
        full_title = self.soup.find('table', class_='fullview-title')  
        self.exists = full_title is not None

        if full_title is None:
            # Doesn't exist
            return

        trs = full_title.find_all('tr', recursive=False)
        self.ticker = trs[0].find(id="ticker").text
        self.exchange = trs[0].find('span').text
        self.company_name = trs[1].text
        self.sectors = [x.strip() for x in trs[2].text.split('|')]

        # fundament table (the table with index, market cap, etc.)
        fundamental_tds = self.soup.find('table', class_='snapshot-table2').find_all('td')

        self.fundamental_df = pd.DataFrame.from_dict(
            {fundamental_tds[index].text: [fundamental_tds[index + 1].text] for index in
             range(0, len(fundamental_tds) - 1, 2)})
        print(self.fundamental_df[3])

        self.outer_ratings_df = Quote.__get_outer_ratings_df__(self.soup)
        #self.outer_news_df = Quote.__get_outer_news_df__(self.soup)
        self.income_statement_df, self.balance_sheet_df, self.cash_flow_df = Quote.__get_XHR_requests__(ticker)
        self.insider_trading_df = Quote.__get_insider_trading_df__(self.soup)

I added

amzn = Quote('AMZN')

at the end and named it Quote.py to make it my own function. Running this(python Quote.py) gave me an error that traced back to self.outer_news_df = Quote.__get_outer_news_df__(self.soup) so I commented that out, and it works!

You should see something like

C:\Users\oazubuko\AppData\Local\Programs\Python\Python39>python Quote.py Index P/E EPS (ttm) Insider Own Shs Outstand Perf Week ... Recom SMA20 SMA50 SMA200 Volume Change 0 S&P 500 108.18 1.11 9.80% 10.18B 4.79% ... 1.70 -3.93% -10.18% -14.55% 10,742,238 -3.18%

I don't care about outer news so I stopped there but hopefully this gets fixed eventually.

Hope this helps

Oazubuko commented 1 year ago

print(self.fundamental_df[3]) should have been commented out sorry

oscar0812 commented 1 year ago

Fixed. Please update and let me know if you encounter any issues. I will close this issue with your confirmation.

oscar0812 commented 1 year ago

Good observation @Oazubuko, Finviz.com was failing to load some outer news, hence the issue.

oscar0812 commented 1 year ago

Closing this issue. Please re-open if the problem persists.