nshiab / simple-data-analysis

Easy-to-use and high-performance JavaScript library for data analysis. Works with tabular and geospatial data.
https://nshiab.github.io/simple-data-analysis/
MIT License
195 stars 11 forks source link

Not working with Deno #439

Open drewbitt opened 2 months ago

drewbitt commented 2 months ago

Loosely related to #430 as well

import { SimpleNodeDB } from "npm:simple-data-analysis";

const sdb = new SimpleNodeDB();

deno run -A filename.ts

error: Uncaught (in promise) Error: Cannot find module '/Users/drewbitt/Repos/x/scripts/find-cadences-imports/node_modules/.deno/duckdb@0.10.1/node_modules/duckdb/lib/binding/duckdb.node'
Require stack:
- /Users/drewbitt/Repos/x/scripts/find-cadences-imports/node_modules/.deno/duckdb@0.10.1/node_modules/duckdb/lib/duckdb-binding.js
- /Users/drewbitt/Repos/x/scripts/find-cadences-imports/node_modules/.deno/duckdb@0.10.1/node_modules/duckdb/lib/duckdb.js
- /Users/drewbitt/Repos/x/scripts/find-cadences-imports/node_modules/.deno/duckdb@0.10.1/node_modules/duckdb/lib/duckdb.js
    at Function.Module._resolveFilename (node:module:566:15)
    at Function.Module._load (node:module:453:27)
    at Module.require (node:module:603:19)
    at require (node:module:709:16)
    at Object.<anonymous> (file:///Users/drewbitt/Repos/x/scripts/find-cadences-imports/node_modules/.deno/duckdb@0.10.1/node_modules/duckdb/lib/duckdb-binding.js:4:15)
    at Object.<anonymous> (file:///Users/drewbitt/Repos/x/scripts/find-cadences-imports/node_modules/.deno/duckdb@0.10.1/node_modules/duckdb/lib/duckdb-binding.js:7:4)
    at Module._compile (node:module:653:34)
    at Object.Module._extensions..js (node:module:667:10)
    at Module.load (node:module:591:32)
    at Function.Module._load (node:module:486:12)

Possibly fixed if packaged via jsr and it could resolve duckdb in https://github.com/nshiab/simple-data-analysis/issues/431 , but the current implementation in https://jsr.io/@nshiab/simple-data-analysis has the same problem.

Summary

It is looking for lib/binding/duckdb.node' but I don't have that

74987
drewbitt commented 2 months ago

Likely related to https://github.com/denoland/deno/issues/16164. There is a way around it (doing the postinstall manually) in https://github.com/denoland/deno/issues/15611#issuecomment-1368519481

nshiab commented 2 months ago

Thank you @drewbitt! Busy week for me. I'll check this soon. I would love the library to work with Deno, too. I quickly tried publishing on jsr.io (as you found out), but it wasn't a success.

drewbitt commented 3 weeks ago

You can now run simple-data-analysis in Deno using DENO_FUTURE=1 and installing packages with npm install (until Deno supports postinstall). Simple example:

❯ npm install duckdb simple-data-analysis
npm WARN deprecated inflight@1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
npm WARN deprecated @npmcli/move-file@2.0.1: This functionality has been moved to @npmcli/fs
npm WARN deprecated npmlog@5.0.1: This package is no longer supported.
npm WARN deprecated npmlog@6.0.2: This package is no longer supported.
npm WARN deprecated rimraf@3.0.2: Rimraf versions prior to v4 are no longer supported
npm WARN deprecated are-we-there-yet@2.0.0: This package is no longer supported.
npm WARN deprecated are-we-there-yet@3.0.1: This package is no longer supported.
npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
npm WARN deprecated glob@8.1.0: Glob versions prior to v9 are no longer supported
npm WARN deprecated gauge@3.0.2: This package is no longer supported.
npm WARN deprecated gauge@4.0.4: This package is no longer supported.

added 153 packages in 3s

9 packages are looking for funding
  run `npm fund` for details

~/Downloads via  v20.12.2 took 2s
❯ DENO_FUTURE=1 deno
Deno 1.44.2
exit using ctrl+d, ctrl+c, or close()
REPL is running with all permissions allowed.
To specify permissions, run `deno repl` with allow flags.
> import { SimpleDB } from "npm:simple-data-analysis"; const sdb = new SimpleDB();
undefined
> const employees = sdb.newTable("employees")
undefined
> await employees.logTable()

table employees: no data
undefined
nshiab commented 3 weeks ago

I just tested the example from the readme and it works well!

Do you know what DENO_FUTURE=1 stands for?

And I asked about the postinstall in this issue: https://github.com/denoland/deno/issues/23656#issuecomment-2165969214

drewbitt commented 3 weeks ago

DENO_FUTURE=1 allows it to use node_modules installed by npm install which is currently the only way postinstall could ever run (through npm)

nshiab commented 3 weeks ago

Hmmmm. I see. Thank you! Why are you interested in simple-data-analysis, if I may ask? 🤓

drewbitt commented 3 weeks ago

Seems like a great way of interacting with DuckDB and replace some of our Deno scripts for data analysis and reporting with something more performant with easy to use features.

nshiab commented 3 weeks ago

Nice!