Unable to import module in runtime after programmatically installation

[leungkinghin-ct]

Details

Description For the sake of genericness, our nodejs application is designed to allow customer to choose their preferred library, therefore we are working on PoC to see how to import and install node module programmatically.

We can see the installed module in node_module folder by running npm i install ${module_name} programmatically, however afterwards we still get error when executing import(${module_name}) as the next step in the same runtime.

Errors

Cannot find package '/Projects/node-playground/node_modules/node-fetch/' imported from /Projects/node-playground/index.js

Tested scenarios

1. Tried to test this approach with both node-fetch and axios and neither works, so we conclude that it is not about the module to be imported.
2. Tried to clean up the npm cache, remove node_modules folder and execute npm i after installing the desired module, but no luck.
3. When we stop the runtime and restart the app, everything works fine as module has been installed into node_module folder and added to package.json

Node.js version

v16.14.0

Example code

import { execSync } from 'child_process'

async function printJoke(module) {
    const url = 'https://api.chucknorris.io/jokes/random'
    const request = {}
    const response = await module.default(url, request)
    const responseJson = await response.json()
    console.log(responseJson?.value)
}

async function printJokesWithoutNodeFetch() {
    const dynamicModule = process.env.DYNAMIC_MODULE
    import(dynamicModule).then( async (module) => {
        await printJoke(module)
    }).catch(async (err) =>  {
        console.error(err.message)
        loadNodeFetch(dynamicModule)
        await printJokesWithoutNodeFetch()
    })
}

function loadNodeFetch(dynamicModule) {

    try {
        return execSync(`npm i ${dynamicModule} `).toString();

    }
    catch (error) {
        console.log('========================')
        console.log(`status : ${error.status}`)
        console.log(`message : ${error.message}`)
        console.log(`stdout : ${error.stdout.toString()}`)
    }
}

(async () => {
   await printJokesWithoutNodeFetch()

})()

Operating system

macOS

Scope

Runtime

Module and version

Not applicable.

[preveen-stack]

cc @nodejs/loaders

[aduh95]

async function printJokesWithoutNodeFetch() {
    const dynamicModule = process.env.DYNAMIC_MODULE
    import(dynamicModule).then( async (module) => {
        await printJoke(module)
    }).catch(async (err) =>  {
        console.error(err.message)
        loadNodeFetch(dynamicModule)
        await printJokesWithoutNodeFetch()
    })
}

It looks like you forgot an await here. Also, if printJoke fails, you probably want to call loadNodeFetch only if the import failed, not if printJoke throws:

async function printJokesWithoutNodeFetch() {
    const dynamicModule = process.env.DYNAMIC_MODULE
    await import(dynamicModule).then( async (module) => {
        await printJoke(module)
    }, async (err) =>  {
        console.error(err.message)
        loadNodeFetch(dynamicModule)
        await printJokesWithoutNodeFetch()
    })
}

(async () => {
   await printJokesWithoutNodeFetch()

})()

You should use top-level await:

await printJokesWithoutNodeFetch()

or

await (async () => {
    await printJokesWithoutNodeFetch()
})()

[leungkinghin-ct]

Thank you @aduh95

I have changed the code as below, however it still doesn't work.

async function printJoke(module) {
    const url = 'https://api.chucknorris.io/jokes/random'
    const request = {}
    const response = await module.default(url, request)
    const responseJson = await response.json()
    console.log(responseJson?.value)
}

async function printJokesWithoutNodeFetch() {
    const dynamicModule = process.env.DYNAMIC_MODULE
    await import(dynamicModule).then( async (module) => {
        await printJoke(module)
    }, async (err) =>  {
        console.error(err.message)
        await loadNodeFetch(dynamicModule)
        await printJokesWithoutNodeFetch()
    })
}

async function loadNodeFetch(dynamicModule) {

    try {
        return execSync(`npm i ${dynamicModule} `).toString();

    }
    catch (error) {
        console.log('========================')
        console.log(`status : ${error.status}`)
        console.log(`message : ${error.message}`)
        console.log(`stdout : ${error.stdout.toString()}`)
    }
}

await printJokesWithoutNodeFetch()

It looks like you forgot an await here.

Are you talking about adding await before loadNodeFetch()?

[aduh95]

Are you talking about adding await before loadNodeFetch()?

No, I meant the await before the import(dynamicModule). Using async for loadNodeFetch() won't help, because you are not using anything async in it – and if it's not async, there's no point to use await either. Maybe you need to specify the cwd so npm installs at the right place?

function loadNodeFetch(dynamicModule) {
    try {
        return spawnSync('npm', ['install', dynamicModule], { cwd: new URL('./', import.meta.url) });
    }
    catch (error) {
        console.log('========================')
        console.log(`status : ${error.status}`)
        console.log(`message : ${error.message}`)
        console.log(`stdout : ${error.stdout.toString()}`)
    }
}

[leungkinghin-ct]

Thank you @aduh95 I tried spawnSync but no luck. Actually module is installed into same place inside node_modules folder either by using spawnSync or execSync

[github-actions[bot]]

It seems there has been no activity on this issue for a while, and it is being closed in 30 days. If you believe this issue should remain open, please leave a comment. If you need further assistance or have questions, you can also search for similar issues on Stack Overflow. Make sure to look at the README file for the most updated links.

[github-actions[bot]]

It seems there has been no activity on this issue for a while, and it is being closed. If you believe this issue should remain open, please leave a comment. If you need further assistance or have questions, you can also search for similar issues on Stack Overflow. Make sure to look at the README file for the most updated links.