simonw / sqlite-utils

Python CLI utility and library for manipulating SQLite databases
https://sqlite-utils.datasette.io
Apache License 2.0
1.58k stars 106 forks source link

sqlite-utils command fails when used with xargs #614

Closed danielhoherd closed 5 months ago

danielhoherd commented 5 months ago

Description

When using xargs, the sqlite-utils command fails. If we copy the exact command that xargs -t showed and run it manually, the command succeeds.

Example

$ ls
foo.csv
$ ls *.csv | tail -n1 | xargs -t sqlite-utils insert --csv --pk "User ID" --replace --alter db.sqlite3
sqlite-utils insert --csv --pk User ID --replace --alter db.sqlite3 foo.csv
Usage: sqlite-utils insert [OPTIONS] PATH TABLE FILE
Try 'sqlite-utils insert -h' for help.

Error: Missing argument 'FILE'.
$ sqlite-utils insert --csv --pk User ID --replace --alter db.sqlite3 foo.csv
  [###################################-]   99%

Tested environments

$ sw_vers
ProductName:        macOS
ProductVersion:     14.2.1
BuildVersion:       23C71
$ python --version --version
Python 3.11.7 (main, Dec  4 2023, 18:10:11) [Clang 15.0.0 (clang-1500.1.0.2.5)]
$ sqlite-utils --version
sqlite-utils, version 3.36
$ grep PRETTY_NAME /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
$ python3 --version --version
Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0]
$ sqlite-utils --version
sqlite-utils, version 3.36
danielhoherd commented 5 months ago

sqlite-utils also does not work with a while loop.

$ ls *.csv | tail -n1 | while read -r X ; do sqlite-utils insert --csv --pk "User ID" --replace --alter db.sqlite3 "$X" ; done ;
Usage: sqlite-utils insert [OPTIONS] PATH TABLE FILE
Try 'sqlite-utils insert -h' for help.

Error: Missing argument 'FILE'.

Or subshells

$ sqlite-utils insert --csv --pk "User ID" --replace --alter db.sqlite3 $(ls *.csv | tail -n1)
Usage: sqlite-utils insert [OPTIONS] PATH TABLE FILE
Try 'sqlite-utils insert -h' for help.

Error: Missing argument 'FILE'.
$ sqlite-utils insert --csv --pk "User ID" --replace --alter db.sqlite3 `ls *.csv | tail -n1`
Usage: sqlite-utils insert [OPTIONS] PATH TABLE FILE
Try 'sqlite-utils insert -h' for help.

Error: Missing argument 'FILE'.
danielhoherd commented 5 months ago

After drinking some coffee, and reading the help text, and thinking "surely it can't fail in all these ways", I see I have a syntax error in my examples. 🤦‍♂️