observablehq / framework

A static site generator for data apps, dashboards, reports, and more. Observable Framework combines JavaScript on the front-end for interactive graphics with any language on the back-end for data analysis.
https://observablehq.com/framework/
ISC License
2.58k stars 125 forks source link

import from "npm:" is not allowed in Data Loaders #1623

Closed ManuelR-D closed 2 months ago

ManuelR-D commented 2 months ago

I'm trying to use a data loader with the following code

import utils from './utils.js';
import { FileAttachment } from "npm:@observablehq/stdlib";

const zipFile = FileAttachment("data/compras2.zip").zip();
const compras = zipFile.file("compras2.json").json("utf8",{typed: false});
process.stdout.write(utils.getDistinctEmpresasFromCompras(compras));

and I get the following error during build time

node:internal/modules/esm/load:228
    throw new ERR_UNSUPPORTED_ESM_URL_SCHEME(parsed, schemes);
          ^

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. Received protocol 'npm:'
    at throwIfUnsupportedURLScheme (node:internal/modules/esm/load:228:11)
    at defaultLoad (node:internal/modules/esm/load:110:3)
    at ModuleLoader.load (node:internal/modules/esm/loader:567:13)
    at ModuleLoader.moduleProvider (node:internal/modules/esm/loader:442:56)
    at new ModuleJob (node:internal/modules/esm/module_job:76:27)
    at #createModuleJob (node:internal/modules/esm/loader:455:17)
    at ModuleLoader.getJobFromResolveResult (node:internal/modules/esm/loader:266:34)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:247:17) {
  code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}

Am I doing something wrong? Is FileAttachment supposed to work within a data loader?

Edit: tried this with Node v20.17.0 and v22.7.0

mbostock commented 2 months ago

JavaScript data loaders are simply normal Node.js programs. You can’t use npm: imports as these are not supported natively by Node.js. You should use the normal Node.js APIs to read files (typically importing from node:fs/promises).

muziejus commented 2 weeks ago

I just wanted to clarify that npm: is not allowed in data loaders. I understand that the OP was misusing FileAttachment, but I get the same error above when I write a data loader like:

import { readParquet } from "npm:parquet-wasm";

...

In other words, as a "simply normal Node.js program," a data loader uses normal ESM loaders import modules and does not have access to npm:?

mbostock commented 2 weeks ago

@muziejus Correct.

muziejus commented 2 weeks ago

Thanks!