Need to create Databases directory

[shaunanoordin]

Scripting Issue

Issue: If I try to run docker-compose build, I will receive the error: "sqlite3.OperationalError: unable to open database file. ERROR: Service 'datasette' failed to build: The command '/bin/sh -c /usr/local/bin/labs-import-csv-files-to-sqlite.sh' returned a non-zero code: 1"

Analysis:

• Full error:

  ---
  Importing gorongosa/subjects.csv to table: subjects in db: ./databases/gorongosa.db
  Traceback (most recent call last):
  File "/usr/local/bin/csvs-to-sqlite", line 10, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/csvs_to_sqlite/cli.py", line 164, in cli
    conn = sqlite3.connect(dbname)
  sqlite3.OperationalError: unable to open database file
  ERROR: Service 'datasette' failed to build: The command '/bin/sh -c /usr/local/bin/labs-import-csv-files-to-sqlite.sh' returned a non-zero code: 1

• It appears that in the scripting line, csvs-to-sqlite --replace-tables $input_csv_file $db_name_path, the csvs-to-sqlite doesn't recursively create folders for $db_path_name if the folders don't exist.

Solution

Run mkdir ./databases if the folder doesn't yet exist.

In the file labs-import-csv-files-to-sqlite.sh, add the following line of code:

...
03  # remove any existing (ephemeral) db as we're going to rebuild it
04  rm -f "./databases/*.db"
05  
++  # create a new database folder
++  [ ! -d "./databases" ] && mkdir "./databases"
06
07  # find all the dirs in data tha have csv, these will be the db names
08  for db_name in $(find -type f -name *.csv -exec dirname {} \; | uniq | sed 's/.\///');
...

Or! Easier yet, give me write access to this repo and I'll push my branch. 👍

[shaunanoordin]

Butts! I can't even assign this issue. Tagging @camallen

[camallen]

closed by f8e8eab58e829feeaab63c8cc338b84ec81d05e2

Thanks @shaunanoordin - the above commit on main should have fixed this (building locally for me now), can you please pull the latest change and try again?

Also you should have access to maintain this repo now, let me know if you don't.

[shaunanoordin]

I pulled a fresh git clone on this repo and can confirm that the .gitkeep fix in f8e8eab works! 👍 Datasette was built and is running fine on my machine at the moment.

[shaunanoordin]

Also updated the PR's title because for some reason I didn't complete the whole sentence.