While using the pinterest scraper, I came across the following error:
Obtained 550 results after 9 scrolls
Scraping: http://www.pinterest.com/nealthegr8/detailed/
URL failed: http://www.pinterest.com/nealthegr8/detailed/
connections attempted: 1
exception message: 'NoneType' object has no attribute 'get'
Traceback (most recent call last):
File "pinterest.py", line 136, in process_whole_page
results = process(soup)
File "pinterest.py", line 420, in
for pin in soup.select('div.item')
File "pinterest.py", line 388, in parse_pin
pin.find('a', {'class': 'pinImageWrapper'}).get('href')),
AttributeError: 'NoneType' object has no attribute 'get'
I am not sure if the BeautifulSoup find() function has changed since this code was written, but using the current way of selecting class in a soup i.e. class_='myclass' as an argument to the find() function solved the problem for me.
While using the pinterest scraper, I came across the following error:
Obtained 550 results after 9 scrolls Scraping: http://www.pinterest.com/nealthegr8/detailed/ URL failed: http://www.pinterest.com/nealthegr8/detailed/ connections attempted: 1 exception message: 'NoneType' object has no attribute 'get' Traceback (most recent call last): File "pinterest.py", line 136, in process_whole_page results = process(soup) File "pinterest.py", line 420, in
for pin in soup.select('div.item')
File "pinterest.py", line 388, in parse_pin
pin.find('a', {'class': 'pinImageWrapper'}).get('href')),
AttributeError: 'NoneType' object has no attribute 'get'
I am not sure if the BeautifulSoup find() function has changed since this code was written, but using the current way of selecting class in a soup i.e. class_='myclass' as an argument to the find() function solved the problem for me.