mysterious "no such table" crash

[drj11]

Traceback (most recent call last):
  File "tool/hooks/refresh", line 131, in 
    main()
  File "tool/hooks/refresh", line 25, in main
    return convert_one(url)
  File "tool/hooks/refresh", line 72, in convert_one
    scraperwiki.sql.save([], features, table_name="feature")
  File "/usr/local/lib/python2.7/dist-packages/scraperwiki/sql.py", line 187, in save
    fit_row(connection, row, unique_keys)
  File "/usr/local/lib/python2.7/dist-packages/scraperwiki/sql.py", line 325, in fit_row
    add_column(connection, new_column)
  File "/usr/local/lib/python2.7/dist-packages/scraperwiki/sql.py", line 345, in add_column
    connection.execute(stmt)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 729, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/ddl.py", line 69, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 783, in _execute_ddl
    compiled
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 958, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1160, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 951, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 436, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (OperationalError) no such table: feature u'ALTER TABLE feature ADD COLUMN x FLOAT' ()

What I did to get this is not so clear. I had a pre-existing GeoJSON tool into which I had copied a sqlite file, then I changed the code to the shapefile branch of geojson-tool and then poked it with the UI.

Investigating.

[drj11]

You can provoke essentially the same bug (I think) by trying the same GeoJSON tool twice.

[frabcus]

I get this one, which I have a hunch is related.

e22n7kq@cobalt-p:~$ ./tool/do.py 
Traceback (most recent call last):
  File "./tool/do.py", line 161, in <module>
    do_spreadsheet(spreadsheet_csv_url)
  File "./tool/do.py", line 101, in do_spreadsheet
    process_person_cols(person)
  File "./tool/do.py", line 72, in process_person_cols
    scraperwiki.sqlite.save(['name', 'start'], data, table_name="holidays")
  File "/usr/local/lib/python2.7/dist-packages/scraperwiki/sql.py", line 188, in save
    connection.execute(insert.values(row))
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 729, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 321, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 958, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1160, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 951, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 436, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (OperationalError) no such table: holidays u'INSERT OR REPLACE INTO holidays (name, start, "end", days, status) VALUES (?, ?, ?, ?, ?)' 

[drj11]

Now that I've pretty much fixed it, I can see that it's cause by a DROP in .execute().

[drj11]

Fixed by https://github.com/scraperwiki/scraperwiki-python/pull/71