Nitrino / easysubs

Browser extension for learning languages with watching movies and TV shows
https://easysubs.cc
MIT License
244 stars 22 forks source link

init.ts error when trying to build the master branch #221

Closed TasseDeCafe closed 1 year ago

TasseDeCafe commented 1 year ago

Hi!

When trying to run "npm run build" from the cloned repo on the master branch, I get the following error:

Could not resolve './{}/init' from src/models/init.ts error during build: Error: Could not resolve './{}/init' from src/models/init.ts at error (/Users/xxxxxxx/WebstormProjects/easysubs/node_modules/vite/node_modules/rollup/dist/shared/rollup.js:198:30) at ModuleLoader.handleResolveId (/Users/xxxxxxx/WebstormProjects/easysubs/node_modules/vite/node_modules/rollup/dist/shared/rollup.js:22464:24) at /Users/xxxxxxx/WebstormProjects/easysubs/node_modules/vite/node_modules/rollup/dist/shared/rollup.js:22427:26

Any idea what could be going on here? I'm not familiar enough with this stack to figure out the issue.

Thank you!

Nitrino commented 1 year ago

Hi @TasseDeCafe In the master branch there is version v2.0 which is under active development and the build may not work yet. If you want to add something to the stable version, use the v1.0 branch And if you want to add something to the new version, use npm run dev

TasseDeCafe commented 1 year ago

Hi! @Nitrino Thank you for your answer, I'll check out that branch instead. I indeed have an idea for a new feature, but it might outside the scope of your project. I would like to be able to generate subtitles with the Whisper API from OpenAI, because they're of much higher quality than the auto-generated subtitles from YouTube. The idea would be to add an option to the extension to save an API key in the settings, and generate subtitles on videos that don't have subtitles through the click of a button.

Another feature I would like to have is the ability to select one or multiple words in the subtitles and save them. I know that Language Reactor allows you to save words (or the whole subtitle), but it can't save multiple words. I suppose that implementing this feature would require to have a backend to handle accounts, which isn't free... but I'm not satisfied with what Language Reactor can do (and it's only available on Chrome).

I also have some other ideas on how the subtitles could be used to practice languages with the help of LLMs like GPT-4, so I'm quite curious to see if I can understand your library.

Let me know if you have any comments on that.

Nitrino commented 1 year ago

@TasseDeCafe The second version is being designed and developed to support our own server, among other things. I have many ideas for improving the plugin, but they will still require a server. For example:

I was also recently thinking about integration with Whisper, but I was thinking more about using their open source version on our server.

But these are big plans that require a lot of work and will cost a lot of money because servers and integrations with AI are quite expensive and I will have to make the service partially paid. I'm confused about this now, because a paid service is a whole other task.

TasseDeCafe commented 1 year ago

Yes, running your own instance might be a good idea if you can find a service that is as cheap or cheaper than OpenAI. The issue I found with Whisper was that it was quite slow on consumer hardware, especially the large v2 model (the best one). I'd like to be able to use the best model for my transcriptions.

Do you have an idea of which translation model you'd like to use? I'm a fan of DeepL, but it tends to get a bit expensive too. But still, it would be nice to be able to enter a DeepL API key and have access to translations from DeepL instead.

I wonder how an open-source LLM like Alpaca performs with translations. What I like about LLMs is that you can provide them context and ask them to translate the word in that specific context, which is better than providing several translations like Language Reactor does (although it works in most cases). It might be too slow for getting a quick translation while watching the video, however.

It would be really cool to be able to practice the new words/collocations learned in a video on a separate website and have an LLM create more texts or exercises with the context of the video. I think that it would be much more interesting and effective than the usual Anki-style websites.

As for the budget, I'd be ready to invest some of my own money in this to have a development server. Which framework would you recommend using? I'm familiar with Node.js and Flask/FastAPI.

I'm more of a hobby programmer than anything else, but I would have time to work on those ideas as I would like to use those tools myself!

Nitrino commented 1 year ago

Do you have an idea of which translation model you'd like to use?

I was thinking about a self-hosted translation solutions like https://github.com/argosopentech/argos-translate.

Using an LLM for all queries would be too slow and expensive, but. In new implementation, I focus more on translating words and phrases than on translating full subtitles, because my goal is to help you learn the language. If we think in this context, translating words is a fairly finite and solvable task. In this case, we can use the LLM + cache responses to further save resources. So I want to have my database that will solve ~95% (just a number out of my mind) of queries and a model that will solve the rest of the queries and replenish the database.

About server development I think this server will consist of a set of services that perform their tasks, so the programming language is not very critical, it will depend more on ready-made solutions.

I have a server at a very early stage, there I use Elixir programming language https://github.com/Nitrino/easysubs-sever

Also Elixir already has pre-trained NMM such as Whisper https://github.com/elixir-nx/bumblebee

If you would be interested in diving into Elixir and extension development I would be happy to help.

TasseDeCafe commented 1 year ago

Hi @TasseDeCafe In the master branch there is version v2.0 which is under active development and the build may not work yet. If you want to add something to the stable version, use the v1.0 branch And if you want to add something to the new version, use npm run dev

Sorry, but I'm not sure how to test the app with npm run dev. As far as I understand Chrome or Firefox require a built app for testing in the browser. I tried again to solve the issue happening with "npm run build", but I can't figure it out. I tried updating packages, but it seems to create additional problems.

EDIT: okay, I figured out that there is a vite plugin for Chrome extensions that works in the dev environment. I still haven't figured out what's going on with the package conflicts.

Nitrino commented 1 year ago

@TasseDeCafe The command npm run dev build the extension into the dist folder. You need to enable developer mode in your browser and add the extension from this folder

TasseDeCafe commented 1 year ago

Yes, thanks, I figured it out later! Do you know of some workaround to solve the dependency issues?

Nitrino commented 1 year ago

@TasseDeCafe What dependency conflicts do you face?

TasseDeCafe commented 1 year ago

So here's what I did: I cloned the repo, and installed the dependencies with npm install from the main branch (on the last commit). When I ran npm run dev, I got a message that indicated that some dependencies for solid-aria were missing, so I added those to the project:

    "@solid-aria/button": "^0.1.3",
    "@solid-aria/collection": "^0.2.0",
    "@solid-aria/listbox": "^0.2.0",
    "@solid-aria/overlays": "^0.1.3",
    "@solid-aria/select": "^0.1.0",

After that, the dev command ran fine, and it's also running correctly in the browser from the /dist folder. Now, I also tried to build the extension, but I'm running into the following issue:

- vite v2.9.15 building for production...
- ✓ 13 modules transformed.
- Could not resolve './{}/init' from src/models/init.ts
- error during build:
- Error: Could not resolve './{}/init' from src/models/init.ts
-     at error (/Users/xxxx/WebstormProjects/easysubs/node_modules/vite/node_modules/rollup/dist/shared/rollup.js:198:30)
-     at ModuleLoader.handleResolveId (/Users/xxxx/WebstormProjects/easysubs/node_modules/vite/node_modules/rollup/dist/shared/rollup.js:22464:24)
-     at /Users/xxxx/WebstormProjects/easysubs/node_modules/vite/node_modules/rollup/dist/shared/rollup.js:22427:26
- ➜  easysubs git:(master) ✗ 

I realized that vite is now at version 4, so I tried to update it, but this seems creates conflicts with other packages:

➜  easysubs git:(master) ✗  npm install vite@latest --save-dev

npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: vite-template-solid@0.0.0
npm WARN Found: vite@2.9.15
npm WARN node_modules/vite
npm WARN   peer vite@"^2.9.0" from @crxjs/vite-plugin@1.0.14
npm WARN   node_modules/@crxjs/vite-plugin
npm WARN     dev @crxjs/vite-plugin@"^1.0.14" from the root project
npm WARN   3 more (vite-plugin-solid, vite-tsconfig-paths, the root project)
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer vite@"^2.9.0" from @crxjs/vite-plugin@1.0.14
npm WARN node_modules/@crxjs/vite-plugin
npm WARN   dev @crxjs/vite-plugin@"^1.0.14" from the root project

And running npm run build after that throws the same error as before:

➜  easysubs git:(master) ✗ npm run build

> vite-template-solid@0.0.0 build
> vite build

vite v4.3.5 building for production...
✓ 19 modules transformed.
✓ built in 157ms
Could not resolve "./{}/init" from "src/models/init.ts"
file: /Users/xxxx/WebstormProjects/easysubs/src/models/init.ts
error during build:
RollupError: Could not resolve "./{}/init" from "src/models/init.ts"
    at error (file:///Users/xxxx/WebstormProjects/easysubs/node_modules/vite/node_modules/rollup/dist/es/shared/node-entry.js:2125:30)
    at ModuleLoader.handleInvalidResolvedId (file:///Users/xxxx/WebstormProjects/easysubs/node_modules/vite/node_modules/rollup/dist/es/shared/node-entry.js:23938:24)
    at file:///Users/xxxx/WebstormProjects/easysubs/node_modules/vite/node_modules/rollup/dist/es/shared/node-entry.js:23900:26

So I'm not quite sure what to do at this point. The dev environment still works with the latest version of Vite.

Nitrino commented 1 year ago

@TasseDeCafe I pushed a dependency update in the wizard and checked the installation from scratch, everything should be working now.

Execute the following commands:

rm -rf  node_modules
rm -rf dist
npm i
npm run dev

After that the dist folder will appear. Next, you need to go into the browser extensions, turn on "development mode" and load the extension from dist folder using "load unpacked".

rakleed commented 1 year ago

@Nitrino after running npm run dev, I still get the Error: The following dependencies are imported but could not be resolved: with a dependency list like this:

@solid-aria/collection (imported by /*****/easysubs/src/components/ui/Select.tsx)
@solid-aria/button (imported by /*****/easysubs/src/components/ui/Select.tsx)
@solid-aria/listbox (imported by /*****/easysubs/src/components/ui/Select.tsx)
@solid-aria/overlays (imported by /*****/easysubs/src/components/ui/Select.tsx)
@solid-aria/select (imported by /*****/easysubs/src/components/ui/Select.tsx)

Running these commands also does not help:

rm -rf node_modules && rm -rf dist
npm i
npm run dev

So I manually install all the missing dependencies with this command:

npm i -D @solid-aria/collection @solid-aria/button @solid-aria/listbox @solid-aria/overlays @solid-aria/select

And then I run npm run dev. Now there are no errors in the terminal, but instead of a working extension, I get a template of React app when I load the unpacked extension in chrome://extensions with the path easysubs/dist: Screenshot 2023-05-25 at 04 44 17

Tried with Node.js 18 and 20.

Nitrino commented 1 year ago

@rakleed It's okay. Check the video on YouTube, the plugin should work

Nitrino commented 1 year ago

The project has been completely rewritten