`*.db` database being created

[simonw]

Running this command:

datasette publish fly \                        
--app my-new-fly-application-2 \
--volume-name vol1 \
--create-volume 1 \
--create-db tiddlywiki \
--install datasette-auth-passwords \
--install datasette-tiddlywiki \
--plugin-secret datasette-auth-passwords root_password_hash $ROOT_PW

Resulted in a database called * being created. This is using the released versions of everything.

[simonw]

I added --generate-dir /tmp/out to the above command. Here's the generated Dockerfile:

FROM python:3.8
COPY . /app
WORKDIR /app

ENV DATASETTE_SECRET '0c835f3596c3d20eb47fbbf759a51afd813b492a5402c73353b5640f7f109ca9'
RUN pip install -U datasette datasette-auth-passwords datasette-tiddlywiki
RUN datasette inspect  --inspect-file inspect-data.json
ENV PORT 8080
EXPOSE 8080
CMD datasette serve --host 0.0.0.0 --cors --inspect-file inspect-data.json --metadata metadata.json /data/tiddlywiki.db --create --port $PORT /data/*.db

Last line is:

CMD datasette serve --host 0.0.0.0 --cors --inspect-file inspect-data.json --metadata metadata.json /data/tiddlywiki.db --create --port $PORT /data/*.db

[simonw]

This is odd. https://stackoverflow.com/questions/41428013/why-does-wildcard-for-jar-execution-not-work-in-docker-cmd suggests that CMD ... should support wildcards like this.

It does offer this as an alternative though:

CMD ["/bin/sh", "-c", "java -jar /app/file*.jar"]

[simonw]

Lots of examples of CMD used with a * pattern here too: https://cs.github.com/?scopeName=All+repos&scope=&q=%2FCMD.*%5C*%5C.%2F+path%3ADockerfile

[simonw]

Could it be that a *.db file is created if you run datasette serve ... /blah/*.db --create and there are no files that match the pattern?

[simonw]

zsh does this:

datasette /tmp/star-dbs/*.db --create -p 8263
zsh: no matches found: /tmp/star-dbs/*.db

sh does this:

/tmp % sh
sh-3.2$ datasette /tmp/star-dbs/*.db --create -p 8263
INFO:     Started server process [51125]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8263 (Press CTRL+C to quit)
^CINFO:     Shutting down
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
INFO:     Finished server process [51125]
sh-3.2$ ls -lah /tmp/star-dbs
total 8
-rw-r--r--   1 simon  wheel   4.0K Feb 14 14:11 *.db

So sh does indeed create a *.db file if one does not yet exist.

bash does this:

bash-3.2$ datasette /tmp/star-dbs/*.db --create -p 8263
INFO:     Started server process [51138]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8263 (Press CTRL+C to quit)
^CINFO:     Shutting down
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
INFO:     Finished server process [51138]
bash-3.2$ ls -lah /tmp/star-dbs/
total 8
-rw-r--r--   1 simon  wheel   4.0K Feb 14 14:12 *.db

So bash also creates the *.db file if there are no matches.

[simonw]

Options to fix this:

• Teach Datasette itself to ignore *.db when used in conjunction with --create. Very easy fix to apply, but I'd need to ship a Datasette dot-release for it.
• Figure out a way to get the CMD line in the Dockerfile to do what's intended here - which is to include all files matching /data/*.db when the command runs, but NOT create a *.db file otherwise

[simonw]

Bash has a nullglob option, not sure how to use that in the context of a CMD line in a Dockerfile though:

• https://unix.stackexchange.com/questions/216222/handle-wildcards-matching-no-file-in-bash
• https://superuser.com/questions/1310211/sh-syntax-to-handle-zero-files-matching-a-wildcard-as-well-as-more

[simonw]

https://cs.github.com/?scopeName=All+repos&scope=&q=%2FCMD.*nullglob%2F+path%3ADockerfile search has a couple of examples of nullglob in Dockerfile:

SHELL ["/bin/bash", "-c"]
CMD flake8; shopt -s globstar nullglob dotglob; shellcheck **/*.{sh,bash,ksh,bashrc,bash_profile,bash_login,bash_logout}

And:

CMD ["/bin/bash","-c","shopt -s globstar nullglob && /usr/local/bin/kube-score score --output-format ci /workspace/**/*.{yaml,yml}"]

I'm going to try that second one.

[simonw]

Manually tested it and it works.