uwdata / arquero

Query processing and transformation of array-backed data tables.
https://idl.uw.edu/arquero
BSD 3-Clause "New" or "Revised" License
1.22k stars 64 forks source link

RFC: Add Deno Support #280

Open lsh opened 2 years ago

lsh commented 2 years ago

Problem

The Arquero build distribution does not support the Deno runtime (even with the --compat flag).

Proposed Solution

Since the main dependency that causes issues for a Deno build is apache-arrow installed from Node, replacing it with a CDN link for the Deno distribution would allow a Deno build target.

Considerations

Another build target means series of tests for platform specific errors.

Benefits

Access to the Deno runtime and ecosystem.

domoritz commented 2 years ago

Skypack (which works for arrow) seems to have an issue with arquero.

> import arquero from "https://cdn.skypack.dev/arquero"
Uncaught TypeError: Relative import path "acorn" not prefixed with / or ./ or ../
    at https://cdn.skypack.dev/-/arquero@v4.8.8-dWXL1bFb2EbzIQrDNAMi/dist=es2019,mode=imports/optimized/arquero.js:2:34
    at async <anonymous>:2:33

That would probably be the first thing to fix.

domoritz commented 2 years ago

import * as arquero from "https://cdn.jsdelivr.net/npm/arquero@4.8.8/+esm"; works actually. 🎉

lsh commented 2 years ago

The acorn issue might just be that there's no ESM target in general?

lsh commented 2 years ago

Note that it seems any of the load{CSV,JSON,etc} operations seem to not work in Deno (using the jsdelivr URL).

Node output:

> const df = await aq.loadJSON("/Users/lukas/Downloads/cars.json");
undefined
> df
ColumnTable [Table: 9 cols x 406 rows] {
  _names: [
    'Name',
    'Miles_per_Gallon',
    'Cylinders',
    'Displacement',
    'Horsepower',
    'Weight_in_lbs',
    'Acceleration',
    'Year',
    'Origin'
  ],
  _data: {
    Name: Column$1 { data: [Array] },
    Miles_per_Gallon: Column$1 { data: [Array] },
    Cylinders: Column$1 { data: [Array] },
    Displacement: Column$1 { data: [Array] },
    Horsepower: Column$1 { data: [Array] },
    Weight_in_lbs: Column$1 { data: [Array] },
    Acceleration: Column$1 { data: [Array] },
    Year: Column$1 { data: [Array] },
    Origin: Column$1 { data: [Array] }
  },
  _total: 406,
  _nrows: 406,
  _mask: null,
  _group: null,
  _order: null
}

Deno output:

> const df = await aq.loadJSON("/Users/lukas/Downloads/cars.json");
Uncaught TypeError: Invalid URL
    at Object.opSync (deno:core/01_core.js:172:12)
    at opUrlParse (deno:ext/url/00_url.js:48:27)
    at new URL (deno:ext/url/00_url.js:322:20)
    at new Request (deno:ext/fetch/23_request.js:212:27)
    at deno:ext/fetch/26_fetch.js:429:29
    at new Promise (<anonymous>)
    at fetch (deno:ext/fetch/26_fetch.js:425:20)
    at us (https://cdn.jsdelivr.net/npm/arquero@4.8.8/+esm:7:86377)
    at Module.ps (https://cdn.jsdelivr.net/npm/arquero@4.8.8/+esm:7:86640)
    at <anonymous>:2:21