"Cannot find ./maps/entities.json in plugin entities" when requiring Cheerio

[cubehouse]

Hi,

I am using Windows, tried with both native nodejs on Windows and using Bash for Windows. When trying to use cheerio with the live-plugin-manager, I get this error:

    at PluginVm.sandboxResolve (/mnt/g/Code/live-plugin-manager/src/PluginVm.js:205:19)
    at PluginVm.resolve (/mnt/g/Code/live-plugin-manager/src/PluginVm.js:57:21)
    at PluginManager.load (/mnt/g/Code/live-plugin-manager/src/PluginManager.js:520:38)
    at PluginManager.require (/mnt/g/Code/live-plugin-manager/src/PluginManager.js:162:21)
    at PluginVm.sandboxRequire (/mnt/g/Code/live-plugin-manager/src/PluginVm.js:239:33)
    at Object.assign (/mnt/g/Code/live-plugin-manager/src/PluginVm.js:155:25)
    at /mnt/g/Code/live-plugin-manager/plugin_packages/htmlparser2/lib/Tokenizer.js:5:17
    at /mnt/g/Code/live-plugin-manager/plugin_packages/htmlparser2/lib/Tokenizer.js:973:2
    at Script.runInContext (vm.js:133:20)
    at PluginVm.vmRunScriptInSandbox (/mnt/g/Code/live-plugin-manager/src/PluginVm.js:113:16)

This is my basic test case to reproduce the issue:

const PluginManager = require("live-plugin-manager").PluginManager;

const manager = new PluginManager();

async function run() {
  await manager.install("cheerio");

  const cheerio = manager.require("cheerio");
  const $ = cheerio.load('<h2 class="title">Hello world</h2>')

  $('h2.title').text('Hello there!')
  $('h2').addClass('welcome')

  console.log($.html());

  await manager.uninstall("cheerio");
}

run();

I've tracked this down to cheerio's dependency on the "entities" module. Entities does a local include of a JSON file that fails because live-plugin-manager is trying to load "./maps/entities.json" relative to the root of the plugin path, rather than where the require was requested from.

I've had some luck by adding the following to PluginVM.sandboxResolve, which basically looks for the include where the main entrypoint file is, rather than the module base.

// check relative path to main entry point if plugin root fails
const subDirPath = path.resolve(path.dirname(pluginContext.mainFile), requiredName);
if (!subDirPath.startsWith(pluginContext.location)) {
    throw new Error("Cannot require a module outside a plugin");
}
const subDirFile = this.tryResolveAsFile(subDirPath);
if (subDirFile) {
    return subDirFile;
}
const isSubdirectory = this.tryResolveAsDirectory(subDirPath);
if (isSubdirectory) {
    return isSubdirectory;
}

... but am not confident in this fix as I believe pluginContext.mainFile isn't the file requesting the require, but the entrypoint to the whole plugin?

[davideicardi]

I think that you are right. The bug is related probably to the fact that the main file is inside a subdirectory, lib. Strange that I never see this problem in other modules... seems to be quite common.

Regarding the fix I'm now sure. I need some more time to investigate ...

If you are able to publish a pull request I appreciate. Probably we need to try to reproduce it in one of the unit test, it should be quite easy I hope.

Thank you for the bug report

[cubehouse]

Sorry for the delay, I've added a pull request with my fix. I don't think it's particularly robust though, but it does fix requiring Cheerio from within the VM.

[davideicardi]

I have done some investigations. From my test the problem is inside htmlparser2 module, only in version 3.10.1, the version used by cheerio. It contains the following code: require("entities/maps/entities.json"). But the correct path is require("entities/lib/maps/entities.json"); (with lib).

Here a simple test.

• create a new node project
• install entities: npm i entities
• create a new index.js file with: require("entities/maps/entities.json")
• run it

I get an error module not found. Like with live-plugin-manager.

If you look at the latest version of htmlparser2, https://github.com/fb55/htmlparser2/blob/master/src/Tokenizer.ts, the code is now correct.

A quick workaround, until cheerio is update htmlparser2 version, is to just install the latest version (4.0.0) of htmlparser2 after cheerio.

Here an unit test:

        it("issue 14: installing cheerio", async function() {
            await manager.installFromNpm("cheerio", "1.0.0-rc.3");
            await manager.installFromNpm("htmlparser2", "4.0.0");

            const cheerio = manager.require("cheerio");
            assert.isDefined(cheerio, "Plugin is not loaded");

            // try to use the plugin
            const $ = cheerio.load('<h2 class="title">Hello world</h2>')

            $('h2.title').text('Hello there!')
            $('h2').addClass('welcome')

            assert.equal($.html(), '<html><head></head><body><h2 class="title welcome">Hello there!</h2></body></html>');
        });

See also: https://github.com/cheeriojs/cheerio/pull/1339#issuecomment-557819894

[davideicardi]

Let me know if you are able to use the proposed workaround (install version 4 of htmlparser2)