After upgrading v1.19.0->1.20.0: failed to load plugin...please provide a valid path

el2iot2 commented 3 weeks ago

After upgrading to v1.20.0, the JavaScript plugin that I wrote was no longer found and prevented commands from working:

Works in v1.19.0

> npx @redocly/cli@1.19.0 build-docs myapi -o yada.html

    ╔═══════════════════════════════════════════════════════╗
    ║                                                       ║
    ║  A new version of Redocly CLI (1.20.0) is available.  ║
    ║  Update now: `npm i -g @redocly/cli@latest`.          ║
    ║  Changelog: https://redocly.com/docs/cli/changelog/   ║
    ║                                                       ║
    ╚═══════════════════════════════════════════════════════╝

Found .redocly.yaml and using theme.openapi options
Prerendering docs
Non-existing tag "tenant-provisioning" is added to the group "Administration / Operation"
Non-existing tag "trait-async-event" is added to the group "Traits"
Non-existing tag "trait-relo-distribution" is added to the group "Traits"

🎉 bundled successfully in: yada.html (1787 KiB) [⏱ 3ms].

> npx @redocly/cli@1.19.0 check-config

    ╔═══════════════════════════════════════════════════════╗
    ║                                                       ║
    ║  A new version of Redocly CLI (1.20.0) is available.  ║
    ║  Update now: `npm i -g @redocly/cli@latest`.          ║
    ║  Changelog: https://redocly.com/docs/cli/changelog/   ║
    ║                                                       ║
    ╚═══════════════════════════════════════════════════════╝

Properties 'enumSkipQuotes', 'sortOperationsAlphabetically', 'sortTagsAlphabetically', 'htmlTemplate', 'theme' are only used in API Reference Docs and Redoc version 2.x or earlier.

✅  Your config is valid.

Fails in v1.20.0

> npx @redocly/cli@1.20.0 build-docs myapi -o yada.html
Something went wrong when processing :

  - Failed to load plugin "plugins/myplugin.js". Please provide a valid path

> npx @redocly/cli@1.20.0 check-config
Something went wrong when processing :

  - Failed to load plugin "plugins/myplugin.js". Please provide a valid path

OpenAPI description

OpenApi: 3.1

.redocly.yaml

# See https://redoc.ly/docs/cli/configuration/ for more information.
apis:
  myapi:
    root: yada/openapi/openapi_myapi.yaml
    rules:
      no-server-example.com:
        severity: off
      tag-description:
        severity: off
      paths-kebab-case:
        severity: off
      operation-4xx-response:
        severity: off
      no-invalid-media-type-examples:
        severity: off
        allowAdditionalProperties: true
  #  snip other APIs
plugins:
  - plugins/myplugin.js
extends:
  - recommended
rules:
  no-unused-components:
    severity: warn
  no-invalid-media-type-examples:
    severity: warn
    allowAdditionalProperties: true
theme:
  openapi:
    codeSamples: 
      languages: 
        - lang: C#
        - lang: curl

    enumSkipQuotes: true
    sortOperationsAlphabetically: true
    sortTagsAlphabetically: true
    htmlTemplate: ./docs/index.html
    theme:
      logo:
        gutter: 3em
      rightPanel:
        backgroundColor: 'rgb(0, 91, 119)'

      colors:
        error:
          main: rgb(203, 44, 48)
        accent:
          main: '#ffc845'
        primary:
          main: 'rgb(0, 91, 119)'

plugins/myplugin.js

const XAudience = require('./decorators/x-audience.js');
//Other plugins, very similar pattern
const X2 = require('./decorators/x2.js');
const X3 = require('./decorators/x3.js');

module.exports = {
  id: 'myplugin',
  decorators: {
    oas3: {
      'x-audience': XAudience,
      'x-2': X2,
      'x-3': X3
    }
  }
};

plugins/decorators/x-audience.js

const utils = require('./utils.js');

module.exports = XAudience;

function XAudience({ value }) {
    console.log("x-audience: " + value);
    let usedTags = {};
    return {
        Root: {
            leave: (node) => {
                utils.DropUnusedTags(node, usedTags);
            }
        },
        Operation: {
            enter: (node, ctx) => {
                let found = node['x-audience'];
                if (found != value) {
                    delete ctx.parent[ctx.key];
                }
                else {
                    utils.MarkTagUsed(node, usedTags);
                }
            }
        },
        PathItem: {
            leave: (node, ctx) => {
                utils.DeletePathItemIfEmpty(node, ctx);
            }
        }
    }
};

Node.js Version(s)

> node --version
v20.7.0

Additional context

Running via NPX.

RomanHotsiy commented 3 weeks ago

Hey @el2iot2. Thanks for reporting. We are looking into it ASAP.

RomanHotsiy commented 3 weeks ago

@el2iot2 the error message is most likely misleading and there may be something related to your plugin.

We are improving the error message in the next release but meanwhile would you be able to share the minimal repro as a zip archive? As i was not able to reproduce it with the details you provided above (I had to comment out utils.js and x2/x3 requires).

el2iot2 commented 3 weeks ago

Yeah, I am sorry. I have been attempting to remove client-specific/internal details to varying success.

I have tried to pare back internals to a simple repro, and after stripping out all but the most simple, it worked fine. So that doesn't help me (or you) understand what happened.

However, running with the new plugin import error version (1.20.1), I am now getting the following on the full (massive) project:

Something went wrong when processing :                                                                                  
  - Failed to load plugin "plugins/myapi.js": Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 's:'

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 's:'
    at new NodeError (node:internal/errors:405:5)
    at throwIfUnsupportedURLScheme (node:internal/modules/esm/load:216:11)
    at defaultLoad (node:internal/modules/esm/load:118:3)
    at ModuleLoader.load (node:internal/modules/esm/loader:388:13)
    at ModuleLoader.moduleProvider (node:internal/modules/esm/loader:270:56)
    at new ModuleJob (node:internal/modules/esm/module_job:65:26)
    at #createModuleJob (node:internal/modules/esm/loader:282:17)
    at ModuleLoader.getJobFromResolveResult (node:internal/modules/esm/loader:240:34)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:221:17)
    at async ModuleLoader.import (node:internal/modules/esm/loader:307:23)

I think this is notable as my plugin path has simply been plugins/myapi.js because I want a relative path. There is no URI scheme, and yet one is reported. Suspiciously, this scheme matches that of my source code drive name on windows (s:). So I am betting that something that resolves that relative path plugins/myapi.js with a .redocly.yaml file located at s:\specs\.redocly.yaml is arriving at s:\specs\plugins\myapi.js and then assuming this is a URI scheme of s: and not a windows drive letter. I suspect that if the absolute path we arrived at was file://s:\specs\plugins\myapi.js (or some other variant), then we may be fine.

el2iot2 commented 3 weeks ago

Aha. As soon as I posted, i realized that my repro was running on my desktop, so would be c: drive and is somehow accounted for. Moving repro to s: made it reproduce:

me@machine  S:\repro\specs 
> npx @redocly/cli@1.20.0 check-config

    ╔═══════════════════════════════════════════════════════╗
    ║                                                       ║
    ║  A new version of Redocly CLI (1.20.1) is available.  ║
    ║  Update now: `npm i -g @redocly/cli@latest`.          ║
    ║  Changelog: https://redocly.com/docs/cli/changelog/   ║
    ║                                                       ║
    ╚═══════════════════════════════════════════════════════╝

Something went wrong when processing :

  - Failed to load plugin "plugins/repro.js". Please provide a valid path

el2iot2 commented 3 weeks ago

> npx @redocly/cli@1.20.1 check-config
Something went wrong when processing :

  - Failed to load plugin "plugins/repro.js": Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 's:'

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 's:'
    at new NodeError (node:internal/errors:405:5)
    at throwIfUnsupportedURLScheme (node:internal/modules/esm/load:216:11)
    at defaultLoad (node:internal/modules/esm/load:118:3)
    at ModuleLoader.load (node:internal/modules/esm/loader:388:13)
    at ModuleLoader.moduleProvider (node:internal/modules/esm/loader:270:56)
    at new ModuleJob (node:internal/modules/esm/module_job:65:26)
    at #createModuleJob (node:internal/modules/esm/loader:282:17)
    at ModuleLoader.getJobFromResolveResult (node:internal/modules/esm/loader:240:34)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:221:17)
    at async ModuleLoader.import (node:internal/modules/esm/loader:307:23)

I have attached the repro zip, but I suspect just running in windows on a drive other than c: with a relative plugin file reference may reproduce this.

repro-bug1673.zip

RomanHotsiy commented 3 weeks ago

Thanks @el2iot2! It explains it. We're working on a fix.

jeremyfiel commented 2 weeks ago

This is also breaking the vs code extension.

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, node, and electron are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'
    at throwIfUnsupportedURLScheme (node:internal/modules/esm/load:239:11)
    at defaultLoad (node:internal/modules/esm/load:129:3)
    at ModuleLoader.load (node:internal/modules/esm/loader:396:13)
    at ModuleLoader.moduleProvider (node:internal/modules/esm/loader:278:56)
    at new ModuleJob (node:internal/modules/esm/module_job:67:26)
    at #createModuleJob (node:internal/modules/esm/loader:290:17)
    at ModuleLoader.getJobFromResolveResult (node:internal/modules/esm/loader:247:34)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:228:17)
    at async ModuleLoader.import (node:internal/modules/esm/loader:315:23)
    at r (c:\Users\fieljere\.vscode\extensions\redocly.openapi-vs-code-0.4.8\out\server\src\server.js:126:1212)
    at async c:\Users\fieljere\.vscode\extensions\redocly.openapi-vs-code-0.4.8\out\server\src\server.js:128:268
    at async Promise.all (index 0)
    at async XM (c:\Users\fieljere\.vscode\extensions\redocly.openapi-vs-code-0.4.8\out\server\src\server.js:128:77)
    at async ZM (c:\Users\fieljere\.vscode\extensions\redocly.openapi-vs-code-0.4.8\out\server\src\server.js:129:3073)
    at async Z8 (c:\Users\fieljere\.vscode\extensions\redocly.openapi-vs-code-0.4.8\out\server\src\server.js:129:3897)
    at async Iye (c:\Users\fieljere\.vscode\extensions\redocly.openapi-vs-code-0.4.8\out\server\src\server.js:126:576)
    at async u3e (c:\Users\fieljere\.vscode\extensions\redocly.openapi-vs-code-0.4.8\out\server\src\server.js:130:6374)
    at async j_ (c:\Users\fieljere\.vscode\extensions\redocly.openapi-vs-code-0.4.8\out\server\src\server.js:247:18024)
    at async _a (c:\Users\fieljere\.vscode\extensions\redocly.openapi-vs-code-0.4.8\out\server\src\server.js:247:18171)
This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason:
Error: Failed to load plugin "./linting/plugins/index.js": Only URLs with a scheme in: file, data, node, and electron are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'

commenting out my plugins definition in config fixes it but now my custom rules and custom types are ignored

jeremyfiel commented 2 weeks ago

just fyi.. vs code has plans to release with full ESM support very soon. (rollout begins tomorrow 8/30) https://github.com/microsoft/vscode/issues/226260

tatomyr commented 1 week ago

Hi @el2iot2, could you check if it works for you in v1.23.0?

el2iot2 commented 1 week ago

Yep. Will report back once i have checked.

el2iot2 commented 6 days ago

Yes, it appears to avoid the issue when running from the s: drive:

> npx @redocly/cli@1.23.0 check-config

    ╔═══════════════════════════════════════════════════════╗
    ║                                                       ║
    ║  A new version of Redocly CLI (1.23.1) is available.  ║
    ║  Update now: `npm i -g @redocly/cli@latest`.          ║
    ║  Changelog: https://redocly.com/docs/cli/changelog/   ║
    ║                                                       ║
    ╚═══════════════════════════════════════════════════════╝

Deprecated plugin format detected: myapi
Deprecated plugin format detected: myapi
Deprecated plugin format detected: myapi
Deprecated plugin format detected: myapi
Deprecated plugin format detected: myapi
Deprecated plugin format detected: myapi
Deprecated plugin format detected: myapi
Deprecated plugin format detected: myapi
Deprecated plugin format detected: myapi
Deprecated plugin format detected: myapi
Deprecated plugin format detected: myapi
Deprecated plugin format detected: myapi
Deprecated plugin format detected: myapi

Properties 'enumSkipQuotes', 'sortOperationsAlphabetically', 'sortTagsAlphabetically', 'htmlTemplate', 'theme' are only used in API Reference Docs and Redoc version 2.x or earlier.

✅  Your config is valid.

I also see that 1.23.1 also appears to work without error. Is there an example I should be looking at to understand how to do a plugin now? I cobbled together the current plugin from examples that I saw here and in the redocly docs site.

What would I be doing that makes this "deprecated" and how can I fix it?

tatomyr commented 4 days ago

@el2iot2 thank you very much for the feedback!

Is there an example I should be looking at to understand how to do a plugin now?

The docs have been updated recently to reflect the change: https://redocly.com/docs/cli/custom-plugins#plugin-structure. In short, the new syntax requires wrapping the plugin object with an IIFE function. The changes were applied since this PR. Hope that helps.

Meanwhile, I'm closing the issue since it appears to be resolved. Please let me know if you think we should do anything else here.

el2iot2 commented 4 days ago

The docs have been updated recently to reflect the change: https://redocly.com/docs/cli/custom-plugins#plugin-structure. In short, the new syntax requires wrapping the plugin object with an IIFE function.

That did the trick. I am back to a valid plugin. Thanks!

Redocly / redocly-cli