CodeInFilth
commented
2 years ago I have also recieved this error on 5+ version commit #89 #90 #91 #92 and #93
I am getting a 403 error when trying to pull the data from retail.... if i go to the address in a browser it seems to be fine so I will test on a different IP shortly
Traceback (most recent call last):
File "scheduled_tasks/economy/get_retail_sales.py", line 44, in <module>
retail_sales()
File "scheduled_tasks/economy/get_retail_sales.py", line 12, in retail_sales
df = pd.read_html("https://ycharts.com/indicators/us_retail_and_food_services_sales")
File "/usr/local/lib/python3.7/dist-packages/pandas/util/_decorators.py", line 299, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/pandas/io/html.py", line 1100, in read_html
displayed_only=displayed_only,
File "/usr/local/lib/python3.7/dist-packages/pandas/io/html.py", line 893, in _parse
tables = p.parse_tables()
File "/usr/local/lib/python3.7/dist-packages/pandas/io/html.py", line 213, in parse_tables
tables = self._parse_tables(self._build_doc(), self.match, self.attrs)
File "/usr/local/lib/python3.7/dist-packages/pandas/io/html.py", line 732, in _build_doc
raise e
File "/usr/local/lib/python3.7/dist-packages/pandas/io/html.py", line 713, in _build_doc
with urlopen(self.io) as f:
File "/usr/local/lib/python3.7/dist-packages/pandas/io/common.py", line 195, in urlopen
return urllib.request.urlopen(*args, **kwargs)
File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
I have had a fix for the earnings on one of my. versions but I would have to dig it up, whichi I will try to do but I have work and projects due this upcoming week so you may get to it faster.
also not sure what version of python you are on but the way you format the get earnings calander is breaking it on most versions. When it trys to insert into a database it fails due to the variables in the query not formatting correctly (i believe)
https://finance.yahoo.com/calendar/earnings?day=2021-10-29&size=100&offset=0
https://finance.yahoo.com/calendar/earnings?day=2021-10-29&size=100&offset=100
https://finance.yahoo.com/calendar/earnings?day=2021-10-29&size=100&offset=200
https://finance.yahoo.com/calendar/earnings?day=2021-10-29&size=100&offset=300
https://finance.yahoo.com/calendar/earnings?day=2021-10-29&size=100&offset=400
https://finance.yahoo.com/calendar/earnings?day=2021-10-30&size=100&offset=0
https://finance.yahoo.com/calendar/earnings?day=2021-10-30&size=100&offset=100
https://finance.yahoo.com/calendar/earnings?day=2021-10-30&size=100&offset=200
https://finance.yahoo.com/calendar/earnings?day=2021-10-30&size=100&offset=300
https://finance.yahoo.com/calendar/earnings?day=2021-10-30&size=100&offset=400
Traceback (most recent call last):
File "scheduled_tasks/get_earnings_calendar.py", line 156, in <module>
insert_earnings_into_db(get_earnings(1, forward=True))
File "scheduled_tasks/get_earnings_calendar.py", line 112, in insert_earnings_into_db
mkt_cap, eps_est, eps_act, surprise, earning_date, earning_time))
sqlite3.OperationalError: near "ON": syntax errorall of these errors have been here excet the first one fore multiple commits. I will try an patch for now just bringing to your attention.
guanquann
I have been trying to debug this one myself but im lost now I think it has to do with cache but not sure.
Apple's Options load 10% of the time and and will never load the default date url on APPL options for whatever reason. If I select an older date its random if it is working and future dates same thing for mmost tickers.
Now I noticed there is an option to claer oold option data in the tasks to run I havnt run that yet as I am unsure of the complete reason to do so.
I also notice at the bottom of some scripts like get earnings, there is a 'delete_old_earnings("2021-10-01")', Is there an area in update ticker_info that has something similar?
I am testing on ubuntu 18.04 and will be setting up windows machine shortly.
P.S. Now I have noticed a few things in the code the one that stands out the most is within the get_financial.py more than half the document is commented out. What was the reason you left this code here? Is it for error testing or fixing like in the helpers.py?