Closed Zamiell closed 3 years ago
Some broader context, unified
, remark
, retext
, and rehype
are moving to ESM https://github.com/unifiedjs/unified/issues/121
The error you are seeing:
Only file and data URLs are supported by the default ESM loader
is raised because one or more of the plugins or presets in your configuration uses ESM, but the configuration file itself is in CJS.
In your specific case this package is retext-english
, which you updated from version 3 to version 4 which is ESM only https://github.com/retextjs/retext/releases/tag/8.0.0
There are two ways you can resolve this:
First rename .remarkrc.js to .remarkrc.mjs
Then update the config from CJS:
module.exports = {
"plugins": [
[
require("remark-retext"),
require("unified")()
.use(require("retext-english")),
],
],
};
to ESM:
import remarkRetext from "remark-retext";
import unified from "unified";
import retextEnglish from "retext-english";
const config = {
plugins: [[remarkRetext, unified().use(retextEnglish)]],
};
export default config;
Doing this will allow you to use the latest versions of remark and remark plugins.
lock the plugins to the last major version that was CJS based in package.json.
For retext-english
that would be the ^3.0.0
version range.
Choosing this option, will keep the existing config working, but will not get new features and fixes added in new releases of plugins.
Thanks for the quick reply Christian. I converted my config to the mjs format and I am good to go now.
I experienced some additional pain getting the file to lint though, which I will document here:
"!.remarkrc.mjs"
to the ignorePatterns
section of my .eslintrc.js
file, because ESLint will explicitly ignore hidden files by default. (This is necessary even if you put .remarkrc.mjs
in the include
section of your tsconfig.eslint.json
file.)extraFileExtensions: [".mjs"],
to the parserOptions
section of my .eslintrc.js
file, because ESLint refuses to parse mjs files otherwise.Actually, I'm not all set.
Remark does not seem to be automatically finding my ".remarkrc.mjs" file. So simply renaming it from ".remarkrc.js" to ".remarkrc.mjs" as Christian describes above does not seem to be sufficient. Instead, Remark is just using no configuration at all, and reporting that there are 0 linter errors, which is why I was initially mislead into thinking that everything was working.
Furthermore, when I attempt to manually specify my config to Remark, I get an error:
$ npx remark test.md -r .remarkrc.mjs
test.md
1:1 error Error: Cannot parse given file `.remarkrc.mjs`
Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only file and data URLs are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'd:'
at new NodeError (node:internal/errors:363:5)
at Loader.defaultResolve [as _resolve] (node:internal/modules/esm/resolve:825:11)
at Loader.resolve (node:internal/modules/esm/loader:89:40)
at Loader.getModuleJob (node:internal/modules/esm/loader:242:28)
at Loader.import (node:internal/modules/esm/loader:177:28)
at importModuleDynamically (node:internal/modules/cjs/loader:1049:29)
at importModuleDynamicallyWrapper (node:internal/vm/module:437:21)
at importModuleDynamically (node:vm:382:43)
at exports.importModuleDynamicallyCallback (node:internal/process/esm_loader:30:14)
at loadFromAbsolutePath (D:\Repositories\hanabi.github.io\node_modules\unified-engine\lib\configuration.js:305:5)
at FormattedError (D:\Repositories\hanabi.github.io\node_modules\fault\index.js:29:12)
at onparse (D:\Repositories\hanabi.github.io\node_modules\unified-engine\lib\find-up.js:81:11)
at done (D:\Repositories\hanabi.github.io\node_modules\trough\wrap.js:55:16)
× 1 error
Some questions:
1) Why isn't Remark automatically picking up my ".remarkrc.mjs" file?
2) Is it possible to manually specify the location of a ".remarkrc.mjs" file?
3) Where is the MJS file format documented?
4) Is there a --verbose
flag or something along these lines so that I can get Remark to tell me the file path of the configuration that it is using?
$ DEBUG="*" npx remark test.md
2021-07-23T21:47:33.931Z unified-engine:configuration Looking for `[ '.remarkrc', '.remarkrc.js', '.remarkrc.yml', '.remarkrc.yaml' ]` configuration files
2021-07-23T21:47:33.932Z unified-engine:configuration Looking for `remarkConfig` fields in `package.json` files
2021-07-23T21:47:33.936Z unified-engine:find-up No files found for `D:\Repositories\hanabi.github.io\test.md`
In the first "Looking for" line, it doesn't mention that it is looking for a ".remarkrc.mjs" file. Does that imply a bug in Remark?
For reference:
$ npx remark --version
remark: 13.0.0, remark-cli: 9.0.0
More info: Even when I manually specify the config file, it says "no files found" for some reason, which also seems to be a bug:
$ DEBUG="*" npx remark misc/FAQ.md --config .remarkrc.mjs
2021-07-23T22:13:38.020Z unified-engine:configuration Looking for `[ '.remarkrc', '.remarkrc.js', '.remarkrc.yml', '.remarkrc.yaml' ]` configuration files
2021-07-23T22:13:38.021Z unified-engine:configuration Looking for `remarkConfig` fields in `package.json` files
2021-07-23T22:13:38.025Z unified-engine:find-up No files found for `D:\Repositories\hanabi.github.io\.remarkrc.mjs`
2021-07-23T22:13:38.026Z unified-engine:file-set-pipeline:stdin Ignoring `streamIn`
2021-07-23T22:13:38.026Z unified-engine:file-pipeline:read Reading `D:\Repositories\hanabi.github.io\.remarkrc.mjs` in `utf8`
2021-07-23T22:13:38.026Z unified-engine:file-pipeline:read Reading `D:\Repositories\hanabi.github.io\misc\FAQ.md` in `utf8`
2021-07-23T22:13:38.027Z unified-engine:file-pipeline:read Read `D:\Repositories\hanabi.github.io\.remarkrc.mjs` (error: null)
2021-07-23T22:13:38.027Z unified-engine:file-pipeline:read Read `D:\Repositories\hanabi.github.io\misc\FAQ.md` (error: null)
2021-07-23T22:13:38.028Z unified-engine:find-up No files found for `D:\Repositories\hanabi.github.io\.remarkrc.mjs`
2021-07-23T22:13:38.028Z unified-engine:file-pipeline:configure Using settings `{}`
2021-07-23T22:13:38.028Z unified-engine:file-pipeline:configure Using `0` plugins
2021-07-23T22:13:38.028Z unified-engine:file-pipeline:parse Parsing `misc\FAQ.md`
2021-07-23T22:13:38.064Z unified-engine:file-pipeline:parse Parsed document
2021-07-23T22:13:38.064Z unified-engine:file-pipeline:transform Transforming document `misc\FAQ.md`
2021-07-23T22:13:38.064Z unified-engine:file-pipeline:transform Transformed document (error: null)
2021-07-23T22:13:38.064Z unified-engine:file-pipeline:queue Queueing `misc\FAQ.md`
2021-07-23T22:13:38.064Z unified-engine:file-pipeline:queue Interupting flush: `.remarkrc.mjs` is not finished
2021-07-23T22:13:38.064Z unified-engine:file-pipeline:queue `misc\FAQ.md` can be flushed
2021-07-23T22:13:38.064Z unified-engine:file-pipeline:queue Not flushing: some files cannot be flushed
2021-07-23T22:13:38.064Z unified-engine:file-pipeline:configure Using settings `{}`
2021-07-23T22:13:38.064Z unified-engine:file-pipeline:configure Using `0` plugins
2021-07-23T22:13:38.065Z unified-engine:file-pipeline:parse Parsing `.remarkrc.mjs`
2021-07-23T22:13:38.068Z unified-engine:file-pipeline:parse Parsed document
2021-07-23T22:13:38.068Z unified-engine:file-pipeline:transform Transforming document `.remarkrc.mjs`
2021-07-23T22:13:38.068Z unified-engine:file-pipeline:transform Transformed document (error: null)
2021-07-23T22:13:38.068Z unified-engine:file-pipeline:queue Queueing `.remarkrc.mjs`
2021-07-23T22:13:38.068Z unified-engine:file-pipeline:queue `.remarkrc.mjs` can be flushed
2021-07-23T22:13:38.068Z unified-engine:file-pipeline:queue `misc\FAQ.md` can be flushed
2021-07-23T22:13:38.068Z unified-engine:file-pipeline:queue Flushing: all files can be flushed
2021-07-23T22:13:38.069Z unified-engine:file-pipeline:stringify Not compiling document without output settings
2021-07-23T22:13:38.069Z unified-engine:file-pipeline:copy Not copying
2021-07-23T22:13:38.069Z unified-engine:file-pipeline:stdout Ignoring writing to `streamOut`
2021-07-23T22:13:38.069Z unified-engine:file-pipeline:file-system Ignoring writing to file-system
2021-07-23T22:13:38.069Z unified-engine:file-pipeline:stringify Not compiling document without output settings
2021-07-23T22:13:38.069Z unified-engine:file-pipeline:copy Not copying
2021-07-23T22:13:38.069Z unified-engine:file-pipeline:stdout Ignoring writing to `streamOut`
2021-07-23T22:13:38.069Z unified-engine:file-pipeline:file-system Ignoring writing to file-system
.remarkrc.mjs: no issues found
misc\FAQ.md: no issues found
(I've manually set up the FAQ.md file to include errors that Remark should catch.)
In the first "Looking for" line, it doesn't mention that it is looking for a ".remarkrc.mjs" file. Does that imply a bug in Remark?
Most likely you locked yourself on an older version of unified engine (a transitive dependency brought in by remark), version 8.2.0 or higher of unified engine is needed https://github.com/unifiedjs/unified-engine/releases/tag/8.2.0
Try running npm upgrade
or clearing your lock file and installing again.
Hey Chris,
Thanks for the help. I've deleted my package-json
file and nuked my node_modules
directory.
But after doing an npm install
, the same problem still persists.
Is there a flag in remark-cli to print out what version of the unified-engine
that the remark-cli is using?
Below, I run a find
command to print out the version of the unified-engine that I have installed:
$ find . -wholename "*unified-engine/package.json" | xargs grep version
"version": "8.2.0",
This matches the latest version that you describe in your last post. Here's unified itself:
$ find . -wholename "*unified/package.json" | xargs grep version
./node_modules/remark-admonitions/node_modules/unified/package.json: "version": "8.4.2",
./node_modules/retext-diacritics/node_modules/unified/package.json: "version": "10.0.1",
./node_modules/retext-english/node_modules/unified/package.json: "version": "10.0.1",
./node_modules/retext-indefinite-article/node_modules/unified/package.json: "version": "10.0.1",
./node_modules/retext-redundant-acronyms/node_modules/unified/package.json: "version": "10.0.1",
./node_modules/retext-repeated-words/node_modules/unified/package.json: "version": "10.0.1",
./node_modules/retext-sentence-spacing/node_modules/unified/package.json: "version": "10.0.1",
./node_modules/retext-syntax-mentions/node_modules/unified/package.json: "version": "10.0.1",
./node_modules/retext-syntax-urls/node_modules/unified/package.json: "version": "10.0.1",
./node_modules/unified/package.json: "version": "9.2.0",
Not sure what else to try. Do you see anything else wrong? Or is this a bug in remark itself?
@Zamiell Could you update your reproduction repo with the error you’re seeing now so I can ivnestigate?
@wooorm I have updated the repo. Here are the new steps to reproduce:
git clone https://github.com/Zamiell/remark-test.git
cd remark-test
npm ci
DEBUG="*" npx remark README.md
The first line of the output will be as follows:
2021-07-24T16:29:13.746Z unified-engine:configuration Looking for `[ '.remarkrc', '.remarkrc.js', '.remarkrc.yml', '.remarkrc.yaml' ]` configuration files
This appears to be a bug, because there should be an entry in the array with a string of ".remarkrc.mjs", but there isn't.
Also, can you reopen this issue?
So running this by adding the config file explicitly does work for me:
I believe you were seeing an issue with that? Or not?
This appears to be a bug, because there should be an entry in the array with a string of ".remarkrc.mjs", but there isn't.
We could add it. It’s more of a feature though. The more files we add, the slower it gets for everyone.
I ran the same command, and get different results:
$ DEBUG="*" npx remark README.md --rc-path .remarkrc.mjs
2021-07-24T17:27:01.383Z unified-engine:configuration Looking for `[ '.remarkrc', '.remarkrc.js', '.remarkrc.yml', '.remarkrc.yaml' ]` configuration files
2021-07-24T17:27:01.385Z unified-engine:configuration Looking for `remarkConfig` fields in `package.json` files
2021-07-24T17:27:01.388Z unified-engine:find-up No files found for `D:\Repositories\remark-test\README.md`
2021-07-24T17:27:01.389Z unified-engine:file-set-pipeline:stdin Ignoring `streamIn`
2021-07-24T17:27:01.390Z unified-engine:file-pipeline:read Reading `D:\Repositories\remark-test\README.md` in `utf8`
2021-07-24T17:27:01.390Z unified-engine:file-pipeline:read Read `D:\Repositories\remark-test\README.md` (error: null)
2021-07-24T17:27:01.390Z unified-engine:find-up Checking given file `D:\Repositories\remark-test\.remarkrc.mjs`
2021-07-24T17:27:01.392Z unified-engine:find-up Only file and data URLs are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'd:'
2021-07-24T17:27:01.392Z unified-engine:file-pipeline:queue Queueing `README.md`
2021-07-24T17:27:01.392Z unified-engine:file-pipeline:queue Flushing: all files can be flushed
2021-07-24T17:27:01.392Z unified-engine:file-pipeline:stringify Not compiling failed document
2021-07-24T17:27:01.392Z unified-engine:file-pipeline:copy Not copying
2021-07-24T17:27:01.392Z unified-engine:file-pipeline:stdout Ignoring writing to `streamOut`
2021-07-24T17:27:01.392Z unified-engine:file-pipeline:file-system Ignoring writing to file-system
README.md
1:1 error Error: Cannot parse given file `.remarkrc.mjs`
Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only file and data URLs are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'd:'
at new NodeError (node:internal/errors:363:5)
at Loader.defaultResolve [as _resolve] (node:internal/modules/esm/resolve:825:11)
at Loader.resolve (node:internal/modules/esm/loader:89:40)
at Loader.getModuleJob (node:internal/modules/esm/loader:242:28)
at Loader.import (node:internal/modules/esm/loader:177:28)
at importModuleDynamically (node:internal/modules/cjs/loader:1049:29)
at importModuleDynamicallyWrapper (node:internal/vm/module:437:21)
at importModuleDynamically (node:vm:382:43)
at exports.importModuleDynamicallyCallback (node:internal/process/esm_loader:30:14)
at loadFromAbsolutePath (D:\Repositories\remark-test\node_modules\unified-engine\lib\configuration.js:305:5)
at FormattedError (D:\Repositories\remark-test\node_modules\fault\index.js:29:12)
at onparse (D:\Repositories\remark-test\node_modules\unified-engine\lib\find-up.js:81:11)
at done (D:\Repositories\remark-test\node_modules\trough\wrap.js:55:16)
× 1 error
Maybe it is because I am on Windows 10? I don't know.
We could add it. It’s more of a feature though. The more files we add, the slower it gets for everyone.
Perhaps I am confused, but I don't see how this is a feature. Let me ask it this way: Why is remark-cli
looking for a '.remarkrc.js'
file by default at all when the program crashes and burns when a '.remarkrc.js'
file is loaded? That is simply a bug in the program, right?
Also, I will ask once again, can we reopen this issue?
Maybe it is because I am on Windows 10? I don't know.
Somewhat, I think this is a Windows-specific bug. https://github.com/unifiedjs/unified-engine/commit/3857a54ef2e502dd6f0e6fdca6a497b7945b40d4 would fix it if something like it would land on the v8 branch
Why is remark-cli looking for a '.remarkrc.js' file by default at all when the program crashes and burns when a '.remarkrc.js' file is loaded?
The reason it looks for a .js
file, is because JavaScript is supported. Either CommonMark or modules. However, it is only as such supported when you use CJS normally, or when you use ESM and add a type: module
.
It does not crash. Node forbids using static imports statements in what is otherwise a CommonJS script.
More on ESM: https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c.
unified-engine also looks for a couple of other files, like YAML and JSON. But it’s not scalable to support every config format or every extension.
Also, I will ask once again, can we reopen this issue?
This is not an issue in this project, and is still being triaged. I’d rather open an issue in the correct place about the actual problem (which I’m trying to figure out)
Thanks for the quick reply woorm.
The reason it looks for a .js file, is because JavaScript is supported. Either CommonMark or modules. However, it is only as such supported when you use CJS normally, or when you use ESM and add a type: module. It does not crash. Node forbids using static imports statements in what is otherwise a CommonJS script. More on ESM: https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c. unified-engine also looks for a couple of other files, like YAML and JSON. But it’s not scalable to support every config format or every extension.
The two options that Christian outlined in his original post are to: 1) switch to ESM 2) stay on old versions forever
I think the obvious choice for end users is to pick option 1. But if end-users are 1) forced to switch to ESM when using the latest version, 2) but remark no longer automatically picks up their remarkrc files
then that is a regression, right?
Maybe I should ask the question in a different way: Is there a way for end-users to: 1) use the latest version of remark 2) and simultaneously have remark automatically pick up a remarkrc file?
If so, I don't think it has been discussed in this thread thus far. I think that in general, end-users expect a tool to automatically pick up a rc file without having to manually specify it, like Bash does, like ESLint does, like Prettier does, etc.
An alternative would be to encourage adding "type": "module"
to package.json which allows ESM to be used in plain .js files.
Either would be fine by me, thoughts @wooorm?
Any updates on this issue? Can we reopen this issue, or get an issue opened in the right place?
Hi! Thanks for the ping and your patience.
I’ve had some time to land a fix to include .remarkrc.json
, .remarkrc.cjs
, and .remarkrc.mjs
in the latest unified-engine
. Back when this issue was opened, remark-cli
was not yet released with that version of the engine, and the ecosystem was in flux.
Now things have stabilized, and you can update remark-cli
(or anything) to the latest version for them to work.
Thus, updating remark-cli
in your test repo, will result in the lines you wanted:
unified-engine:configuration Looking for `[
'.remarkrc',
'.remarkrc.json',
'.remarkrc.cjs',
'.remarkrc.mjs',
'.remarkrc.js',
'.remarkrc.yaml',
'.remarkrc.yml'
]` configuration files +0ms
Assuming that you also install unified
, use its latest version (which was not a dependency in your package.json
yet), and change:
-import unified from "unified";
+import {unified} from "unified";
...then everything works.
It’s been a bit of a long thread and I’m not sure if there are other things you wanted to have differently?
Yes, this issue appears to be resolved now in the latest versions of everything, thanks again wooorm.
Error Message
On the latest version, remark no longer works properly on Windows, resulting in the following error:
This is a regression because it worked just fine in the past on older versions.
Test Repo
You can easily reproduce this bug by following the steps in my test repository.
Environment