Modular imports causing error in API npm package

[hassannaftabb]

Hi, I have installed the "echogarden": "^0.11.14" package and when importing it in my file, on startup it's giving the following error:

const echogarden_1 = require("echogarden"); ^ Error [ERR_REQUIRE_ESM]: require() of ES Module C:\Users\HASSAN\Desktop\PROJECTS\SubtitleO\subtitleo-server\node_modules\echogarden\dist\API\API.js from C:\Users\HASSAN\Desktop\PROJECTS\SubtitleO\subtitleo-server\dist\projects\projects.service.js not supported. Instead change the require of API.js in C:\Users\HASSAN\Desktop\PROJECTS\SubtitleO\subtitleo-server\dist\projects\projects.service.js to a dynamic import() which is available in all CommonJS modules. at Object.<anonymous> (C:\Users\HASSAN\Desktop\PROJECTS\SubtitleO\subtitleo-server\dist\projects\projects.service.js:33:22) at Object.<anonymous> (C:\Users\HASSAN\Desktop\PROJECTS\SubtitleO\subtitleo-server\dist\user\user.module.js:22:28) at Object.<anonymous> (C:\Users\HASSAN\Desktop\PROJECTS\SubtitleO\subtitleo-server\dist\folders\folders.module.js:17:23) at Object.<anonymous> (C:\Users\HASSAN\Desktop\PROJECTS\SubtitleO\subtitleo-server\dist\app.module.js:13:26) at Object.<anonymous> (C:\Users\HASSAN\Desktop\PROJECTS\SubtitleO\subtitleo-server\dist\main.js:6:22)

I am using NestJS, so when it's compiled, the commonjs & module crashes with each other.

This is how I am importing it:

import * as Echogarden from 'echogarden';

Please resolve this ASAP, I am stucked due to this.

My nest versions:

"@nestjs/common": "^8.0.0", "@nestjs/core": "^8.0.0",

@rotemdan

[rotemdan]

I'm not familiar with NestJS. Based on the error message, it seems like it is trying to import the module using require, under the hood? Echogarden is a complex application that is written using ESM modules only. I don't know exactly why import * as Echogarden from 'echogarden' is translated to require("echogarden"). You'll need to check what settings are causing this.

Here is a StackOverflow question related to this. Maybe you can try making your own module as ESM? (adding "type":"module" to package.json). Maybe you can try a dynamic import like const Echogarden = await import('echogarden')? I don't know why it may be necessary, these are some of the suggestions in the page.

Also, I noticed your project name is 'SubtitleO'. If you want to get Echogarden's subtitle outputs via the Node.js API, just know that the API currently doesn't expose any subtitle methods (you can get JSON 'timeline' structures, but not subtitles). The reason is that I haven't fully stabilized the subtitles methods. They went through a partial rewrite several weeks ago, and I still have some small things I'm unsure about.

I can expose it via a method like timelineToSubtitles, or via the main API methods themselves, like synthesize and align. I haven't really made up what is the best choice. I can provide both.

Let me know when you get to the point where the import is working and we'll see what type of functionality I'll need to expose via the Node.js API.

Edit: also, Echogarden is virtually impossible to bundle, since it uses a lot of WASM libraries, some native bindings and binary libraries, and other static content. Any solution involving bundlers would likely fail.

[hassannaftabb]

I'm not familiar with NestJS. Based on the error message, it seems like it is trying to import the module using require, under the hood? Echogarden is a complex application that is written using ESM modules only. I don't know exactly why import * as Echogarden from 'echogarden' is translated to require("echogarden"). You'll need to check what settings are causing this.

Here is a StackOverflow question related to this. Maybe you can try making your own module as ESM? (adding "type":"module" to package.json). Maybe you can try a dynamic import like const Echogarden = await import('echogarden')? I don't know why it may be necessary, these are some of the suggestions in the page.

Also, I noticed your project name is 'SubtitleO'. If you want to get Echogarden's subtitle outputs via the Node.js API, just know that the API currently doesn't expose any subtitle methods (you can get JSON 'timeline' structures, but not subtitles). The reason is that I haven't fully stabilized the subtitles methods. They went through a partial rewrite several weeks ago, and I still have some small things I'm unsure about.

I can expose it via a method like timelineToSubtitles, or via the main API methods themselves, like synthesize and align. I haven't really made up what is the best choice. I can provide both.

Let me know when you get to the point where the import is working and we'll see what type of functionality I'll need to expose via the Node.js API.

Edit: also, Echogarden is virtually impossible to bundle, since it uses a lot of WASM libraries, some native bindings and binary libraries, and other static content. Any solution involving bundlers would likely fail.

Hi,

Thanks for your reply.

I am not transcribing my audio using Echogarden but aligning them for better results. Yes, I'll try these fixes if they could help e to solve this.

Otherwise is there any other way i can you your align and other functions for better quality of my timeline?

@rotemdan

[rotemdan]

I tried to make the default setting for alignment work well for most types of reasonably clean audio (no background music or chatter, no simultaneous multiple speakers). The options available through the API are the same as the command line interface. You can test through the CLI to see what result you get with various settings.

The reason I mentioned subtitles is that it is common to want to have subtitles along with the timeline structure. It's not exposed right now in the API but it's not difficult to expose it.

Also, the Node.js API is running in the same thread as the thread you call from, which can block your server while it's running. To get it run in the background, you can start an Echogarden WebSocket server (via the API) and use the Client class to communicate with it. Internally, the Client class also supports communicating with a Node worker thread (it's already implemented and used internally) but it's not exposed yet.

Also, the particular output you shown could also be related to TypeScript module resolution settings - in case you are using TypeScript. You might need to set the tsconfig.json to use ESM modules instead of CommonJS modules (I don't know if this is the case, but it's another possibility).

[hassannaftabb]

I tried to make the default setting for alignment work well for most types of reasonably clean audio (no background music or chatter, no simultaneous multiple speakers). The options available through the API are the same as the command line interface. You can test through the CLI to see what result you get with various settings.

The reason I mentioned subtitles is that it is common to want to have subtitles along with the timeline structure. It's not exposed right now in the API but it's not difficult to expose it.

Also, the Node.js API is running in the same thread as the thread you call from, which can block your server while it's running. To get it run in the background, you can start an Echogarden WebSocket server (via the API) and use the Client class to communicate with it. Internally it also support using Node worker threads but it's not exposed yet.

Also, the particular output you shown could also be related to TypeScript module resolution settings - in case you are using TypeScript. You might need to set the tsconfig.json to use ESM modules instead of CommonJS modules (I don't know if this is the case, but it's another possibility).

This is my tsconfig.json:

{ "compilerOptions": { "module": "CommonJS", "declaration": true, "removeComments": true, "emitDecoratorMetadata": true, "experimentalDecorators": true, "allowSyntheticDefaultImports": true, "target": "es2017", "sourceMap": true, "outDir": "./dist", "baseUrl": "./", "incremental": true, "skipLibCheck": true, "strictNullChecks": false, "noImplicitAny": false, "strictBindCallApply": false, "forceConsistentCasingInFileNames": false, "noFallthroughCasesInSwitch": false } }

If i change module type, it's causing problems to other imports, if i solve that, at the end i get the same error as above. Can you suggest something on this?

Thanks.

[rotemdan]

I asked Bing Chat. It said that you can use dynamic imports like const Echogarden = await import('echogarden') even if the project is set to use CommonJS modules ("module": "CommonJS").

I never did that kind of import myself since I usually do it the other way around: I set my TypeScript project to ESM modules ("module": "Node16"), and then import CommonJS modules using const { default: SomeModule } = await import('someModule'). I use this pattern extensively internally in Echogarden and other projects.

Here is the related TypeScript documentation, with more information:

https://www.typescriptlang.org/docs/handbook/esm-node.html

[hassannaftabb]

I asked Bing Chat. It said that you can use dynamic imports like const Echogarden = await import('echogarden') even if the project is set to use CommonJS modules ("module": "CommonJS").

I never did that kind of import myself since I usually do it the other way around: I set my TypeScript project to ESM modules ("module": "Node16"), and then import CommonJS modules using const { default: SomeModule } = await import('someModule'). I use this pattern extensively internally in Echogarden and other projects.

Here is the related TypeScript documentation, with more information:

https://www.typescriptlang.org/docs/handbook/esm-node.html

Thank you so much for your cooperation. I will try this shortly. Is there any place where I can connect with you actively like Discord or something?

Would be really helpful.

Thank you once again.

[rotemdan]

I don't really use Discord or chat platforms in general. I can try to provide some assistance here if needed. This can be useful for other people who are trying to use the API. Also it's easier to provide code and well-edited, multiline answers via GitHub, compared to chat.

(You can also try asking ChatGPT or Bing AI if the question is not specific to the library - I myself use Bing AI (in 'precise' mode) all the time lately and it usually gives very good answers - I probably asked it about 100 questions today - though that's probably a record).

Anyway, currently the API is still not as complete as the CLI. There are some missing pieces, and it hasn't been tested that much on its own.

[hassannaftabb]

I don't really use Discord or chat platforms in general. I can try to provide some assistance here if needed. This can be useful for other people who are trying to use the API. Also it's easier to provide code and well-edited, multiline answers via GitHub, compared to chat.

(You can also try asking ChatGPT or Bing AI if the question is not specific to the library - I myself use Bing AI (in 'precise' mode) all the time lately and it usually gives very good answers - I probably asked it about 100 questions today - though that's probably a record).

Anyway, currently the API is still not as complete as the CLI. There are some missing pieces, and it hasn't been tested that much on its own.

Unfortunately , I am still getting the same error. Is there any chance we can connect for 5 minutes and see this issue live? I'll put the solution here for users. But I am stuck here so desperately need to solve this.

Would be appreciatable.

Thanks you.

@rotemdan

[rotemdan]

It's not clear if the error you're describing is particular to the library, or a general issue about importing an ESM module from a CommonJS TypeScript project.

I can try to help with things that are specific to Echogarden, but not just any general programming issues.

You can just paste the code and error messages here and I'll try to see what they mean. You can even upload the entire project as a zip file. I don't think that chat will make any difference. I actually think it's easier here (at least for me, so I don't have to answer immediately and can do other things in the same time).

[hassannaftabb]

It's not clear if the error you're describing is particular to the library, or a general issue about importing an ESM module from a CommonJS TypeScript project.

I can try to help with things that are specific to Echogarden, but not just any general programming issues.

You can just paste the code and error messages here and I'll try to see what they mean. You can even upload the entire project as a zip file. I don't think that chat will make any difference. I actually think it's easier here (at least for me, so I don't have to answer immediately and can do other things in the same time).

Here is the error (same as before):

ERROR [ExceptionsHandler] require() of ES Module C:\Users\HASSAN\Desktop\PROJECTS\SubtitleO\subtitleo-server\node_modules\echogarden\dist\API\API.js from C:\Users\HASSAN\Desktop\PROJECTS\SubtitleO\subtitleo-server\dist\projects\projects.service.js not supported. Instead change the require of API.js in C:\Users\HASSAN\Desktop\PROJECTS\SubtitleO\subtitleo-server\dist\projects\projects.service.js to a dynamic import() which is available in all CommonJS modules. Error [ERR_REQUIRE_ESM]: require() of ES Module C:\Users\HASSAN\Desktop\PROJECTS\SubtitleO\subtitleo-server\node_modules\echogarden\dist\API\API.js from C:\Users\HASSAN\Desktop\PROJECTS\SubtitleO\subtitleo-server\dist\projects\projects.service.js not supported. Instead change the require of API.js in C:\Users\HASSAN\Desktop\PROJECTS\SubtitleO\subtitleo-server\dist\projects\projects.service.js to a dynamic import() which is available in all CommonJS modules. at C:\Users\HASSAN\Desktop\PROJECTS\SubtitleO\subtitleo-server\dist\projects\projects.service.js:94:64 at async ProjectsService.transcribe (C:\Users\HASSAN\Desktop\PROJECTS\SubtitleO\subtitleo-server\dist\projects\projects.service.js:94:29)

and here is my code:

Here I am importing it like this:

async transcribe( videoPath, userId, folderId, videoName, language, originalName, ) { const Echogarden = (await import('echogarden')).default;

and here i am using it:

const aligned_transcription = await Echogarden.align( tempAudioPath, data.result.text, { subtitles: { minWordsInLine: 3, }, }, ); console.log(aligned_transcription);

this is the compiled version of the function where it's causing error:

async transcribe(videoPath, userId, folderId, videoName, language, originalName) { const Echogarden = (await Promise.resolve().then(() => require('echogarden'))).default;

I have multiple other modules. this is the first time i encountered this issue.

[rotemdan]

Although I develop with Node.js a lot. The issue that you are presenting is not something that I come across often since I always set TypeScript to use ESM modules ("module": "Node16"), and import CommonJS modules without special problems, both statically and dynamically.

I don't have an answer right now. Here is what Bing Chat has to say about this:

It later suggested some really hack-ish way (using eval) to do it that I wouldn't recommend.

As long as TypeScript is emitting require, the problem isn't specific to Echogarden, and I don't know exactly how to help.

[hassannaftabb]

Although I develop with Node.js a lot. The issue that you are presenting is not something that I come across often since I always set TypeScript to use ESM modules ("module": "Node16"), and import CommonJS modules without special problems, both statically and dynamically.

I don't have an answer right now. Here is what Bing Chat has to say about this:

It later suggested some really hack-ish way (using eval) to do it that I wouldn't recommend.

As long as TypeScript is emitting require, the problem isn't specific to Echogarden, and I don't know exactly how to help.

OK Thanks for your support, I'll try to get some solution.