search

simonw / datasette-publish-fly

Datasette plugin for publishing data using Fly
Apache License 2.0
20 stars 7 forks source link

Large sqlite db, 10min threshold guess #19

Closed justmars closed 2 years ago

justmars commented 2 years ago

Tried:

datasette publish fly x.db \
--app "test" \
--region "sin" \
--extra-options="--setting sql_time_limit_ms 15000" \
--metadata metadata.yml \
--install pysqlite3-binary \
--install datasette-auth-tokens

The ph.db sqlite3 file is 2gb.

After ~10 minutes it stops with an error.

The deployment used to be work well. I'm guessing it was because it hit the threshold of inactivity for the remote builder, i.e. as a feature: "They turn off automatically after 10 minutes of inactivity"

==> Verifying app config → Verified app config ==> Building image Remote builder fly-builder-white-wood-1586 ready ==> Creating build context → Creating build context done ==> Building image with Docker → docker host: 20.10.12 linux x86_64 [+] Building 596.1s (0/1) => [internal] load remote build context 596.1s ERRO[0622] Can’t add file /private/var/folders/7g/qc77bj6j63q0gr0mrzvbmxkc0000gn/T/tmph1n8z2gw/test/x.db to tar: io: read/write on closed pipe ERRO[0622] Can’t close tar writer: io: read/write on closed pipe Error failed to fetch an image or build from source: error building: unexpected EOF

Assuming the guess is correct, is any any workaround possible? Will building the docker image locally help in not consuming the full 10 minutes?

justmars commented 2 years ago

It appears to be related to the 10 minute threshold. Was able to deploy using more manual means thanks to the answers provided by community members in fly.io.