Newbie to pandas here, tried to import the data using your bs4 script. Is it a problem on my end or yours?

kylebruder / metal_prices

Scrapes gold and silver prices and saves them into a csv file. You can also run an API to return date range specific results in JSON format.

1 stars 0 forks source link

Newbie to pandas here, tried to import the data using your bs4 script. Is it a problem on my end or yours? #9

Open DSforSC opened 2 years ago

DSforSC commented 2 years ago

Traceback (most recent call last): File "c:\Users\itcwork\Documents\Python Scripts\Python_projects\Crypto_vs_Metals\importing_metal_btc_prices.py", line 60, in fetch_prices('https://www.investing.com/commodities/gold-historical-data', 'gold_prices.csv') File "c:\Users\itcwork\Documents\Python Scripts\Python_projects\Crypto_vs_Metals\importing_metal_btc_prices.py", line 36, in fetch_prices rows = table.find_all('tr')[1:-1] AttributeError: 'NoneType' object has no attribute 'find_all'

kylebruder commented 2 years ago

Thanks for posting the traceback. I will look into the cause :)

kylebruder commented 2 years ago

The URL still works in the browser, but the connection is refused when using the script. It looks like investing.com got wise about web scraping. I think the request header needs to be modified to appear to be coming from a web browser rather than a script. This repo was created as a proof of concept a few years back. Unfortunately, I am committing my time to an different project ATM. If you would like to contribute and post a merge request I will be happy to review it and merge it. Thanks so much for the feed back!