Closed drj11 closed 9 years ago
I found one!
q = lxml.html.parse('http://lxml.de/lxmlhtml.html').xpath('//td')[0].text_content()
So.. what's the failure? That looks an awful lot like the phrase "Ian Bicking"
to me.
I just dropped that comment in before I caught the train, so at least I wouldn't forget it. It's not actually a string:
>>> scraperwiki.sql.save([], dict(t=q))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "scraperwiki/sql.py", line 187, in save
fit_row(connection, row, unique_keys)
File "scraperwiki/sql.py", line 314, in fit_row
get_column_type(column_value))
File "scraperwiki/sql.py", line 353, in get_column_type
return PYTHON_SQLITE_TYPE_MAP[type(column_value)]
KeyError: <class 'lxml.etree._ElementStringResult'>
dumptruck extends the typemap if lxml is available: https://github.com/scraperwiki/dumptruck/blob/master/dumptruck/dumptruck.py#L49
[edited on 2014-09-22]
scraperwiki.sql.save()
can't save values that are instances oflxml.etree._ElementStringResult
(see example below).Discovered when running the
archinterface
scraper: it crashes because it was trying to save some sort of lxml string object.