Volume support plus integration tests

[simonw]

Refs #10. Still needed:

• [x] --generate-dir option
• [x] Rename --generate-dir to just --generate (and back again, changed my mind)
• [x] --create-volume working
• [x] --create-db working
• [x] --create-volume tests
• [x] Implement thing where it checks to see if previous deploy had volumes and re-mounts them
• [x] --volume-name x working
• [x] --volume-name x tests
• [x] Manual testing script, which will inform...
• [x] ... integration tests for the new functionality
• [x] Updated documentation

[simonw]

Second part of that test script:

echo '{"name": "Barry"}' | sqlite-utils insert other.db docs -
datasette publish fly fixtures.db other.db \
--app test-script-1 \
--plugin-secret datasette-auth-passwords root_password_hash 'pbkdf2_sha256$260000$35a5c9d08a2116aa91e6012474df127b$Zgt5b9fDkc2ivG9syQq296rZ7VmF37MfHfOKUKfOLrQ=' \
--install datasette-auth-passwords \
--install datasette-tiddlywiki \
--install datasette-graphql

After running this there should be a new other.db database but my tiddlywiki.db database should remain, with its data. And the datasette-graphql plugin should be installed.

[simonw]

Something went wrong - https://test-script-1.fly.dev/ lists the fixtures.db and other.db but not tiddlywiki.db.

https://test-script-1.fly.dev/-/plugins shows the right plugins.

The volume seems to still be attached:

% flyctl volumes list -a test-script-1
ID                      NAME        SIZE    REGION  ZONE    ATTACHED VM CREATED AT  
vol_ke628r6pwdpvwmnp    datasette   1GB     sjc     c0a5    ea325cfb    3 hours ago 

So why isn't the tiddlywiki.db database visible any more?

[simonw]

I added debug code and the Dockerfile ends in this:

CMD datasette serve --host 0.0.0.0 -i fixtures.db -i other.db --cors --inspect-file inspect-data.json --metadata metadata.json --port $PORT

Which is wrong, it should also serve /data/*.db.

[simonw]

Here's why:

        extra_options = extra_options or ""
        if create_db:
            for database_name in create_db:
                if not database_name.endswith(".db"):
                    database_name += ".db"
                extra_options += " /data/{}".format(database_name)
            extra_options += " --create"

[simonw]

I tried doing this:

extra_options += " --create /data/*.db"

And now I have a database called *!

Because of this: https://github.com/simonw/datasette/blob/7d24fd405f3c60e4c852c5d746c91aa2ba23cf5b/datasette/utils/__init__.py#L367-L370

    if extra_options:
        for opt in extra_options.split():
            cmd.append(f"{opt}")
    cmd = [shlex.quote(part) for part in cmd]

[simonw]

Solution is pretty messy: I'm going to read the generated Dockerfile and re-write the last line of it.

[simonw]

Got this error from flyctl list volumes -a x

Could not resolve App

Because the app had not yet been created.

[simonw]

Got the basic integration test running again. Now I just need an integration test that creates a volume, then deploys again without explicitly listing that volume.

[simonw]

Just got this from the integration test I'm writing:

E           AssertionError: Error: Error calling 'flyctl volumes create':
E             
E             Error Could not resolve App

Which makes sense, because I was attempting to create the volume before I created the app!

[simonw]

I'm not going to write an integration test for the re-deploying case, I'll trust my existing non-integration tests for that.

[jamesmstone]

Champion! thanks