Call(ticker) fails after multiple runs

[KanikaBatra]

In my application, I want to extract options data per minute for one of the tickers. What my python code does as of now is following:

_g = Call(Ticker) exp_dates = g.expirationdates() for each expiration date: g = Call(Ticker, d, m, y) dat = g.calls() g = Put(Ticker, d, m, y) dat = g.calls() store cumulative data to database

However, after running this code successfully 3 times at 2 minute interval, the 4th time code fails with following error: "raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)" at g = Call(Ticker)

After an hour or so, the code starts working again. And again after 3 runs, it fails.

Please let me know if there is some limitation with wallstreet or it is something I am missing out.

Thanks in advance :)

[mcdallas]

Hi @KanikaBatra . I cannot recreate the error you are getting. Could you paste a code fragment that I can run as well as the full traceback?

[KanikaBatra]

Hi @mcdallas, Here is the code I wrote:

 # Import Libraries
from wallstreet import Stock, Call, Put
from datetime import datetime
import json
from pandas.io.json import json_normalize
import pandas as pd
import numpy as np
import timeit
import threading

def OptionsData(expiration_dates, stockPrice):
    ticker = 'googl'
    datC = pd.DataFrame()
    datP = pd.DataFrame()
    for exp_date in expiration_dates:
        print(exp_date)
        timestamp = int((datetime.utcnow() - datetime(1970, 1, 1, 0, 0, 0, 0)).total_seconds())
        datC = datC.append(getOptionsDF(ticker, optionType="Call",expiration_date=exp_date,stockPrice=stockPrice,
                           lowerThresh=0.75,upperThresh=1.25), ignore_index=True)
        datP = datP.append(getOptionsDF(ticker, optionType="Put",expiration_date=exp_date,stockPrice=stockPrice,
                           lowerThresh=0.75,upperThresh=1.25))

    # Write data to Database
    print('Data written to database at'+str(datetime.utcnow()))

    # Execute every 2 minutes
    threading.Timer(120.0, OptionsData(expiration_dates, stockPrice)).start()

 # Wrapper for Calls and Puts
def getOptionsDF(ticker, expiration_date, stockPrice, lowerThresh, upperThresh, optionType):

    # Collect the current timestamp and split it into date (dd), month (mm) and year (yyyy)
    timestamp = int((datetime.utcnow() - datetime(1970, 1, 1, 0, 0, 0, 0)).total_seconds())
    dmy = expiration_date.split("-")

    # If option type is Call, create Call object else if it is Put, create Put object
    if(optionType=="Call"):
        g = Call(ticker, d=int(dmy[0]) , m=int(dmy[1]), y=int(dmy[2]))
    elif(optionType=="Put"):
        g = Put(ticker, d=int(dmy[0]) , m=int(dmy[1]), y=int(dmy[2]))

    # Filter out the strike prices of interest    
    interested_strike = list(filter(lambda  x: x>lowerThresh*stockPrice.price,
        list(filter(lambda x: x<upperThresh*stockPrice.price, g.strikes))))
    interested_strike = [ '%.2f' % strike for strike in interested_strike ]

    # Get entire data from wallstreet API as json, normalize it and convert to a dataframe  
    json_list = g.calls
    options_df = json_normalize(json_list)
    options_df = options_df[options_df['strike'].isin(interested_strike)]
    options_df['timestamp'] = timestamp
    return(options_df)

 # Get all valid expiration dates
def getExpirationDates(ticker):
    g = Call(ticker)
    expiration_date_list = g.expirations
    return(expiration_date_list)

Now when I execute the code as follows:

expiration_dates = getExpirationDates('googl')
stockPrice=Stock('googl')
OptionsData(expiration_dates, stockPrice)

Data is written to DB once however, next time it fails and post which I can't even do g=Call('googl')

Thanks!

[mcdallas]

Could you also post the full traceback when the error occurs?

[KanikaBatra]

Here is the full traceback:

20-01-2017
17-03-2017
16-06-2017
19-01-2018
---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
<ipython-input-2-81abb571cab6> in <module>()
      1 expiration_dates = getExpirationDates('googl')
      2 stockPrice=Stock('googl')
----> 3 OptionsData(expiration_dates, stockPrice)

<ipython-input-1-d50d17d8532e> in OptionsData(expiration_dates, stockPrice)
     19                            lowerThresh=0.75,upperThresh=1.25), ignore_index=True)
     20         datP = datP.append(getOptionsDF(ticker, optionType="Put",expiration_date=exp_date,stockPrice=stockPrice,
---> 21                            lowerThresh=0.75,upperThresh=1.25))
     22 
     23     # Write data to Database

<ipython-input-1-d50d17d8532e> in getOptionsDF(ticker, expiration_date, stockPrice, lowerThresh, upperThresh, optionType)
     42     # Filter out the strike prices of interest
     43     interested_strike = list(filter(lambda  x: x>lowerThresh*stockPrice.price,
---> 44         list(filter(lambda x: x<upperThresh*stockPrice.price, g.strikes))))
     45     interested_strike = [ '%.2f' % strike for strike in interested_strike ]
     46 

<ipython-input-1-d50d17d8532e> in <lambda>(x)
     41 
     42     # Filter out the strike prices of interest
---> 43     interested_strike = list(filter(lambda  x: x>lowerThresh*stockPrice.price,
     44         list(filter(lambda x: x<upperThresh*stockPrice.price, g.strikes))))
     45     interested_strike = [ '%.2f' % strike for strike in interested_strike ]

/home/kanika_batra/anaconda3/lib/python3.5/site-packages/wallstreet/wallstreet.py in price(self)
     74     @property
     75     def price(self):
---> 76         self.update()
     77         return self._price
     78 

/home/kanika_batra/anaconda3/lib/python3.5/site-packages/wallstreet/wallstreet.py in update(self)
     67 
     68     def update(self):
---> 69         self.__init__(self.ticker)
     70 
     71     def __repr__(self):

/home/kanika_batra/anaconda3/lib/python3.5/site-packages/wallstreet/wallstreet.py in __init__(self, quote, exchange)
     45 
     46         jayson = r.text.replace('\n','')
---> 47         jayson = json.loads(jayson[2:])[0]
     48 
     49         try:

/home/kanika_batra/anaconda3/lib/python3.5/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    317             parse_int is None and parse_float is None and
    318             parse_constant is None and object_pairs_hook is None and not kw):
--> 319         return _default_decoder.decode(s)
    320     if cls is None:
    321         cls = JSONDecoder

/home/kanika_batra/anaconda3/lib/python3.5/json/decoder.py in decode(self, s, _w)
    337 
    338         """
--> 339         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    340         end = _w(s, end).end()
    341         if end != len(s):

/home/kanika_batra/anaconda3/lib/python3.5/json/decoder.py in raw_decode(self, s, idx)
    355             obj, end = self.scan_once(s, idx)
    356         except StopIteration as err:
--> 357             raise JSONDecodeError("Expecting value", s, err.value) from None
    358         return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Thanks!

[mcdallas]

Well the .price attribute is dynamic i.e each time you access it, it connects to google, fetches the current price and recreates the object. I am not sure why it fails to parse the response in your case but you can try to replace the stockPrice.price in your OptionsData function with stockPrice._price (which is static)

[mcdallas]

btw if you are using the version on pip, you might want to install directly from this repo because there is a fix for your problem in the newer version.

[KanikaBatra]

Oh yes, I installed using pip. Let me try the two changes you have suggested and get back. Thanks for quick replies :)

[KanikaBatra]

Hey, I tried the changes you mentioned in above trails. Instead of the error I was getting earlier, now I am getting the below error:

  File "/usr/local/lib/python3.6/site-packages/wallstreet/wallstreet.py", line 221, in __init__
    super().__init__(quote, **kw)
  File "/usr/local/lib/python3.6/site-packages/wallstreet/wallstreet.py", line 130, in __init__
    self.underlying = Stock(quote, source=self.source)
  File "/usr/local/lib/python3.6/site-packages/wallstreet/wallstreet.py", line 51, in __init__
    self._google(query)
  File "/usr/local/lib/python3.6/site-packages/wallstreet/wallstreet.py", line 91, in _google
    raise LookupError('Ticker symbol not found. Try adding the exchange parameter')
LookupError: Ticker symbol not found. Try adding the exchange parameter```

Looking forward to the resolution of the problem. Thanks in advance. 

[mcdallas]

@KanikaBatra I managed to replicate your issue after several tries (it seems to happen randomly) and I run it through the pycharm debugger. The reason is that it gets a 503 response from Google Finance which translates to service unavailable (The server is currently unable to handle the request due to a temporary overloading or maintenance of the server).

My suggestion is to either add some exception handling to catch the error and retry after a few seconds or alternatively use the source='yahoo' keyword to grab the data from Yahoo finance althought it will be delayed by 15 mins.