abdeladim-s / subsai

🎞️ Subtitles generation tool (Web-UI + CLI + Python package) powered by OpenAI's Whisper and its variants 🎞️
https://abdeladim-s.github.io/subsai/
GNU General Public License v3.0
1.16k stars 96 forks source link

Anyway to increase file limit from 200mb? #54

Closed BRUGraphicDesign closed 11 months ago

BRUGraphicDesign commented 11 months ago

Hello, I love the program it works amazingly well! I just want to find out, if is there an easy way to increase the file limit from 200 MB to say 1 GB or higher.

edit: I found out you can add the config.toml file to bypass the limit, but in doing so it pretty much kills my machine.

abdeladim-s commented 11 months ago

Yes @BRUGraphicDesign, the media file size is limited to 200 mb in the webui. I would suggest to use the command line interface for large files.

bigbeka commented 3 months ago

@abdeladim-s thank you for a great work you are doing here.

Any chance to increase the limit for the WebUI?

abdeladim-s commented 3 months ago

@bigbeka, welcome .. 'Glad you found it useful. I think you can run the webui with --maxsize argument and provide the maximum size you want. Here is the command

subsai-webui --server.maxMessageSize 500 # for 500 mb for example

Keep in mind though that you just cannot view a media file beyond 200 mb (this is a limitation of the underlying library used to view the file) otherwise, you can run the transcription and export the subtitles normally.

bigbeka commented 3 months ago

@abdeladim-s Sorry, perhaps I should have been more specific: I am running SubsAI in a Docker Container with docker compose up -d.

Would I be able to pass through subsai-webui --server.maxMessageSize 500 to the build process?

abdeladim-s commented 3 months ago

@bigbeka, yes I think you can, but not to the build, rather to the run command. Let me know if it didn't work.

bigbeka commented 3 months ago

@abdeladim-s no joy, here are the steps I took. With the command enabled, I can upload smaller <200MB files and transcribe them. But bigger files don't even get uploaded via the UI.

docker-compose.yml file: `version: '3.9' services: subsai-webui: image: subsai-subsai-webui volumes:

Logs output: subsai.log

abdeladim-s commented 3 months ago

@bigbeka, the error in the logs output does not seem it's related to the size of the file! Have you tried other models other than faster-whsiper, do you get the same results ?

Also I changed the docker file a bit so that you can pass arguments to the run command directly without changing docker-compose, so I think the steps may look like this:

I would suggest also not to use the upload option (unless if you are running the docker in a remote environment) and use the local path option instead, you can access your mapped files from within the container using the /media_files folder.

bigbeka commented 3 months ago

@abdeladim-s Yes, using the docker compose up -d with the command you suggested and bigger file size, I tried different models and resulted in the same error.

  1. I pulled the latest changes
  2. docker compose build
  3. This resulted in GPU missing error, as I am running it on CPU: docker compose run -p 8501:8501 -v /path/to/your/media_files/folder:/media_files subsai-webui --server.maxMessageSize 10000
  4. I commended out GPU passthrough, and still getting the same error.

I think the issue might lie here, smaller files are getting uploaded, bigger files are not getting uploaded:

Screenshot 2024-03-11 at 09 56 18

I am running this in a server, which I need to make available to my team, for this use case, UI upload of bigger file sizes need to work for upload and transcription.

abdeladim-s commented 3 months ago

@bigbeka, Oh .. so you are running it on a server and you are using the upload option! To increase the limit in this case you'll need to use this command: --server.maxUploadSize 10000 instead of --server.maxMessageSize, please try it as usual and let me know how it goes: docker compose run -p 8501:8501 -v /path/to/your/media_files/folder:/media_files subsai-webui --server.maxUploadSize 10000

bigbeka commented 3 months ago

@abdeladim-s Apologies, again I should have been clear on the setup from the offset.

Please let me know if I should create a new issue for this. So I'm running it in a server, with docker compose up -d and --server.maxUploadSize 10000 worked. GUI shows max file size as 10GB. I now am able to upload larger files.

But, when the upload finishes (1.78GB), I get an error: AxiosError: Request failed with status code 502

Screenshot 2024-03-12 at 09 00 17

If I disregard the error message and click on Transcribe, there is error with these logs: subsai.log

Update:

This looks like an issue with RAM, once the file upload finishes, RAM usage shoots up even before starting the transcription. Smaller (<300mb) files were fine using anything under 4GB RAM, for a file that is 1.78GB big, 12GB RAM was sufficient. But during transcription, only 6GB RAM is used.

This looks like it is related to #45

abdeladim-s commented 3 months ago

@bigbeka, Yes please if you can share your setup so I can understand the situation correctly, I indeed believe the AxiosError is related to the web host you are using or maybe you are using a Proxy of some sort?

Can you re-upload the subsai.log file, because the link returns 404! And yeah I think we can move this to a new issue since the file limit problem is solved!


If you are experiencing high RAM usage, I think you are still using --server.maxMessageSize, because otherwise I remember I fixed this issue by removing the video viewer if the file exceeds the limit, and you should get a warning message instead. Can you please check your docker compose file ?