0.1.0 release fails to launch on a MacBook Air (macOS Mojave 10.14.6)

[simonw]

The icon bounces once and then the app quits.

[simonw]

On further investigation it looks to be a problem with the ~/.datasette-app/venv virtual environment.

Once installed, /Applications/Datasette.app bundles a working Python 3.9:

/Applications/Datasette.app/Contents/Resources/python/bin/python3.9 --version
Python 3.9.6

It also manages to run the full installation into the virtual environment - ls ~/.datasette-app/venv/bin returned the expected list of binaries, including python3.9 and datasette.

But... those binaries do not run correctly:

$ ~/.datasette-app/venv/bin/python3.9
-bash: /Users/nataliedowne/.datasette-app/venv/bin/python3.9: No such file or directory
$ ~/.datasette-app/venv/bin/datasette
-bash: /Users/nataliedowne/.datasette-app/venv/bin/datasette: /Users/nataliedowne/.datasette-app/venv/bin/python3.9: bad interpreter: No such file or directory

[simonw]

Here's the relevant code: https://github.com/simonw/datasette-app/blob/a862d74672bf4979272ed423c44b241d36bb501c/main.js#L181-L202

[simonw]

Maybe findPython() found the wrong Python? https://github.com/simonw/datasette-app/blob/a862d74672bf4979272ed423c44b241d36bb501c/main.js#L246-L260

[simonw]

This is very weird... I used the Python embedded in /Applications/Datasette.app to create a new virtual environment and install Datasette and that worked fine!

cd /tmp
/Applications/Datasette.app/Contents/Resources/python/bin/python3.9 -m venv venv
venv/bin/pip install datasette
venv/bin/datasette

This started the Datasette server running locally.

So for some reason the virtual environment created by the Electron app ends up broken, but the same environment created by running the same commands in a shell works as expected.

[simonw]

Does this bug affect any other systems on more recent operating systems? Need to get some people to install it to find out!

[simonw]

Here's a workaround for anyone who runs into the same bug. Launch the app for the first time, then paste the following into a terminal window:

rm -rf ~/.datasette-app/venv/
/Applications/Datasette.app/Contents/Resources/python/bin/python3.9 -m venv ~/.datasette-app/venv
~/.datasette-app/venv/bin/pip install datasette==0.59a2 datasette-app-support

Launch the app a second time and it should run correctly. This worked on the MacBook Air running Mojave where I first saw this bug.

[ageitgey]

I am also having this (or a very similar) issue. The app launched and ran correctly the first time. The second time, the app wouldn't run again and I have a broken venv where the python binary is missing.

I think my issue is that if you launch the app initially outside of /Applications/, macOS applies app translation to the app as a security measure. This results in the initial Python install getting mistakenly getting installed to a temporary random folder instead of to the actual folder you want.

I think the simplest workaround might be to refuse to run / refuse to install Python if the user isn't running from /Applications/ and throw up an error message. But maybe there is a more complex way to work around the issue.

Here's some debugging that might be helpful from how I saw the issue:

% pwd
/Users/ageitgey/Downloads/Datasette.app/Contents

Run the app manually to get a stack trace:

% ./MacOS/Datasette
Failed to start datasette Error: spawn /Users/ageitgey/.datasette-app/venv/bin/datasette ENOENT
    at Process.ChildProcess._handle.onexit (internal/child_process.js:269:19)
    at onErrorNT (internal/child_process.js:465:16)
    at processTicksAndRejections (internal/process/task_queues.js:80:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn /Users/ageitgey/.datasette-app/venv/bin/datasette',
  path: '/Users/ageitgey/.datasette-app/venv/bin/datasette',
  spawnargs: [
    '--port',
    8001,
    '--version-note',
    'xyz-for-datasette-app',
    '--setting',
    'sql_time_limit_ms',
    '10000',
    '--setting',
    'max_returned_rows',
    '2000',
    '--setting',
    'facet_time_limit_ms',
    '1000',
    '--setting',
    'max_csv_mb',
    '0'
  ]
}

Ok, so the datasette server won't run. Try to invoke it manually:

% /Users/ageitgey/.datasette-app/venv/bin/datasette

zsh: /Users/ageitgey/.datasette-app/venv/bin/datasette: bad interpreter: /Users/ageitgey/.datasette-app/venv/bin/python3.9: no such file or directory

Datasette thinks Python doesn't exist, so it can't run. See if Python is really there:

% ls -la  /Users/ageitgey/.datasette-app/venv/bin/
total 152
drwxr-xr-x  21 ageitgey  staff   672 Sep  9 09:29 .
drwxr-xr-x   6 ageitgey  staff   192 Sep  9 09:29 ..
-rw-r--r--   1 ageitgey  staff  8834 Sep  9 09:29 Activate.ps1
-rw-r--r--   1 ageitgey  staff  1910 Sep  9 09:29 activate
-rw-r--r--   1 ageitgey  staff   859 Sep  9 09:29 activate.csh
-rw-r--r--   1 ageitgey  staff  1999 Sep  9 09:29 activate.fish
-rwxr-xr-x   1 ageitgey  staff   243 Sep  9 09:29 datasette
-rwxr-xr-x   1 ageitgey  staff   249 Sep  9 09:29 dateadd
-rwxr-xr-x   1 ageitgey  staff   250 Sep  9 09:29 datediff
-rwxr-xr-x   1 ageitgey  staff   242 Sep  9 09:29 hupper
-rwxr-xr-x   1 ageitgey  staff   277 Sep  9 09:29 normalizer
-rwxr-xr-x   1 ageitgey  staff  5386 Sep  9 09:29 pint-convert
-rwxr-xr-x   1 ageitgey  staff   254 Sep  9 09:29 pip
-rwxr-xr-x   1 ageitgey  staff   254 Sep  9 09:29 pip3
-rwxr-xr-x   1 ageitgey  staff   254 Sep  9 09:29 pip3.9
lrwxr-xr-x   1 ageitgey  staff     9 Sep  9 09:29 python -> python3.9
lrwxr-xr-x   1 ageitgey  staff     9 Sep  9 09:29 python3 -> python3.9
lrwxr-xr-x   1 ageitgey  staff   166 Sep  9 09:29 python3.9 -> /private/var/folders/nn/t5x7tzq140b0g7qldcxgcnrm0000gn/T/AppTranslocation/A314CF18-67D2-4D08-BFD9-41B0A91BDF54/d/Datasette.app/Contents/Resources/python/bin/python3.9
-rwxr-xr-x   1 ageitgey  staff   246 Sep  9 09:29 sqlite-utils
-rwxr-xr-x   1 ageitgey  staff   242 Sep  9 09:29 tabulate
-rwxr-xr-x   1 ageitgey  staff   244 Sep  9 09:29 uvicorn

Ok, python was set up as a link to /private/var/folders/nn/t5x7tzq140b0g7qldcxgcnrm0000gn/T/AppTranslocation/A314CF18-67D2-4D08-BFD9-41B0A91BDF54/d/Datasette.app/Contents/Resources/python/bin/python3.9.

Spoiler: That temporary folder no longer exists. That's an App Translation folder that gets created when you run an app outside of a trusted area. By linking to a temporary location, datasette will fail to run the next time you try it. It will look like Python didn't get installed. It did, but it will gone then.

Now, let's try the initial run again, but instead do the initial run from within /Applications/:

% rm -rf ~/.datasette-app

# <move Datasette.app to /Applications/ and re-run>

Result: The app runs again as expected. In addition, you end up with a non-broken python install:

% ls -la  /Users/ageitgey/.datasette-app/venv/bin/
total 152
drwxr-xr-x  21 ageitgey  staff   672 Sep  9 10:01 .
drwxr-xr-x   6 ageitgey  staff   192 Sep  9 10:01 ..
-rw-r--r--   1 ageitgey  staff  8834 Sep  9 10:01 Activate.ps1
-rw-r--r--   1 ageitgey  staff  1910 Sep  9 10:01 activate
-rw-r--r--   1 ageitgey  staff   859 Sep  9 10:01 activate.csh
-rw-r--r--   1 ageitgey  staff  1999 Sep  9 10:01 activate.fish
-rwxr-xr-x   1 ageitgey  staff   243 Sep  9 10:01 datasette
-rwxr-xr-x   1 ageitgey  staff   249 Sep  9 10:01 dateadd
-rwxr-xr-x   1 ageitgey  staff   250 Sep  9 10:01 datediff
-rwxr-xr-x   1 ageitgey  staff   242 Sep  9 10:01 hupper
-rwxr-xr-x   1 ageitgey  staff   277 Sep  9 10:01 normalizer
-rwxr-xr-x   1 ageitgey  staff  5386 Sep  9 10:01 pint-convert
-rwxr-xr-x   1 ageitgey  staff   254 Sep  9 10:01 pip
-rwxr-xr-x   1 ageitgey  staff   254 Sep  9 10:01 pip3
-rwxr-xr-x   1 ageitgey  staff   254 Sep  9 10:01 pip3.9
lrwxr-xr-x   1 ageitgey  staff     9 Sep  9 10:01 python -> python3.9
lrwxr-xr-x   1 ageitgey  staff     9 Sep  9 10:01 python3 -> python3.9
lrwxr-xr-x   1 ageitgey  staff    67 Sep  9 10:01 python3.9 -> /Applications/Datasette.app/Contents/Resources/python/bin/python3.9
-rwxr-xr-x   1 ageitgey  staff   246 Sep  9 10:01 sqlite-utils
-rwxr-xr-x   1 ageitgey  staff   242 Sep  9 10:01 tabulate
-rwxr-xr-x   1 ageitgey  staff   244 Sep  9 10:01 uvicorn

[simonw]

@ageitgey this is fantastic information, thank you - really helpful. I hadn't heard of app translation before.

[simonw]

This is working on that laptop now - I've successfully installed several builds.