Automated test for Pyodide compatibility

[simonw]

Refs:

• 1733

Need something in the test suite such that if Datasette breaks against Pyodide in the future we hear about it.

I'm thinking this is an opportunity to use shot-scraper javascript.

[simonw]

Here's a script that seems to work. It builds the wheel, starts a Python web server that serves the wheel, runs a test with shot-scraper and then shuts down the server again.

#!/bin/bash

# Build the wheel
python3 -m build

# Find name of wheel
wheel=$(basename $(ls dist/*.whl))
# strip off the dist/

# Create a blank index page
echo '
<script src="https://cdn.jsdelivr.net/pyodide/v0.20.0/full/pyodide.js"></script>
' > dist/index.html

# Run a server for that dist/ folder
cd dist
python3 -m http.server 8529 &
cd ..

shot-scraper javascript http://localhost:8529/ "
async () => {
  let pyodide = await loadPyodide();
  await pyodide.loadPackage(['micropip', 'ssl', 'setuptools']);
  let output = await pyodide.runPythonAsync(\`
    import micropip
    await micropip.install('h11==0.12.0')
    await micropip.install('http://localhost:8529/$wheel')
    import ssl
    import setuptools
    from datasette.app import Datasette
    ds = Datasette(memory=True, settings={'num_sql_threads': 0})
    (await ds.client.get('/_memory.json?sql=select+55+as+itworks&_shape=array')).text
  \`);
  if (JSON.parse(output)[0].itworks != 55) {
    throw 'Got ' + output + ', expected itworks: 55';
  }
  return 'Test passed!';
}
"

# Shut down the server
pkill -f 'http.server 8529'

[simonw]

I'm going to start off by running this manually - I may run it on every commit once this is all a little bit more stable.

I can base the workflow on https://github.com/simonw/scrape-hacker-news-by-domain/blob/main/.github/workflows/scrape.yml

[simonw]

https://github.com/simonw/datasette/runs/6265915080?check_suite_focus=true failed but looks like it passed because I forgot to use set -e at the start of the bash script.

It failed because it didn't have build available.

[simonw]

Test ran in 38 seconds and passed! https://github.com/simonw/datasette/runs/6265954274?check_suite_focus=true

I'm going to have it run on every commit and PR.