scraperwiki / code-scraper-in-browser-tool

Just like on ScraperWiki Classic; now a part of QuickCode.
https://quickcode.io
Other
38 stars 8 forks source link

cant write to sql #138

Closed Casyfill closed 8 years ago

Casyfill commented 9 years ago

Hi. I was using scraperwiki for a while. However, today my new scraper "ScrapeDeputies" don't want to write data to sql: here is a snippet

datum = [] for link in GetDeputiesLinks(): d = getData(link) name = d.cssselect(" div.hc-r > h1")[0].text info = d.cssselect("div.deputat-info")[0]

PARTY

text = info.cssselect("p.deputat-info-intro")[0].text

try: party = text[text.index(u'«')+1:text.index(u'»')] except: party = "?"

EDUCATION

if 'Образование' in stringify_children(info): education = None else:

   education = '|'.join([x.text for x in info.cssselect('ul.list-ul1')[-1].cssselect('li') if x.text!=None])

print text

datum.append( {'name':name, 'education':education, 'text':text, 'party':party}
scraperwiki.sql.save(unique_keys, datum)

i have text printed over all links i've expected. However, my sql looks line empty

Casyfill commented 9 years ago

Oops. It looks like it just took a bit more time to appear, then i've expected