simonw / sqlite-utils

Python CLI utility and library for manipulating SQLite databases
https://sqlite-utils.datasette.io
Apache License 2.0
1.67k stars 111 forks source link

Feature request: trim all leading and trailing white space for all columns for all tables in a database #523

Open fgregg opened 1 year ago

fgregg commented 1 year ago

It's pretty common that i need to trim leading or trailing white space from lots of columns in a database a part of an initial ETL.

I use the following recipe a lot, and it would be great to include this functionality into sqlite-utils

trimify.sql

select 'select group_concat(''update [' || name || '] set ['' || name || ''] = trim(['' || name || ''])'', '';
'') || '';
'' as sql_to_run from pragma_table_info('''||name||''');' from sqlite_schema;

then something like:

    sqlite3 example.db < scripts/trimify.sql > table_trim.sql && \
             sqlite3 $example.db < table_trim.sql > trim.sql && \
             sqlite3 $example.db < trim.sql
fgregg commented 1 year ago

I also often then run another little script to cast all empty strings to null, but i save that for another issue if this gets accepted.