Closed Casyfill closed 8 years ago
Hi. I was using scraperwiki for a while. However, today my new scraper "ScrapeDeputies" don't want to write data to sql: here is a snippet
datum = [] for link in GetDeputiesLinks(): d = getData(link) name = d.cssselect(" div.hc-r > h1")[0].text info = d.cssselect("div.deputat-info")[0] PARTY text = info.cssselect("p.deputat-info-intro")[0].text try: party = text[text.index(u'«')+1:text.index(u'»')] except: party = "?" EDUCATION if 'Образование' in stringify_children(info): education = None else: education = '|'.join([x.text for x in info.cssselect('ul.list-ul1')[-1].cssselect('li') if x.text!=None]) print text datum.append( {'name':name, 'education':education, 'text':text, 'party':party} scraperwiki.sql.save(unique_keys, datum)
datum = [] for link in GetDeputiesLinks(): d = getData(link) name = d.cssselect(" div.hc-r > h1")[0].text info = d.cssselect("div.deputat-info")[0]
text = info.cssselect("p.deputat-info-intro")[0].text
try: party = text[text.index(u'«')+1:text.index(u'»')] except: party = "?"
if 'Образование' in stringify_children(info): education = None else:
education = '|'.join([x.text for x in info.cssselect('ul.list-ul1')[-1].cssselect('li') if x.text!=None])
print text
datum.append( {'name':name, 'education':education, 'text':text, 'party':party} scraperwiki.sql.save(unique_keys, datum)
i have text printed over all links i've expected. However, my sql looks line empty
Oops. It looks like it just took a bit more time to appear, then i've expected
Hi. I was using scraperwiki for a while. However, today my new scraper "ScrapeDeputies" don't want to write data to sql: here is a snippet
i have text printed over all links i've expected. However, my sql looks line empty