search

saharmor / whisper-playground

Build real time speech2text web apps using OpenAI's Whisper https://openai.com/blog/whisper/
MIT License
777 stars 140 forks source link

Multiple errors: missing python dev, portaudio and venv #1

Closed Manamama closed 1 year ago

Manamama commented 1 year ago

My box:

OS: Ubuntu 20.04.5 LTS x86_64 Kernel: 5.4.0-125-generic Packages: 3901 (dpkg), 30 (flatpak), 37 (snap) Shell: bash 5.0.17 Python 3.8.10

Your instructions throw many compilation errors:

  1. The python3-dev package is missing: 
    
30    | #include "Python.h"
    |          ^~~~~~~~~~
compilation terminated.
error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
-> 
`sudo apt-get install python3-dev  # for python3.x installs`

2. PortAudio does not compile:

31 | #include "portaudio.h" | ^~~~~ compilation terminated. error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1


Solved only with manual compilation of http://files.portaudio.com/download.html, i.e.:  https://github.com/ProgrammingHero1/romantic-alexa/issues/88 

3. Virtual env is playing up:
`cd backend && source venv/bin/activate && flask run --port 8000`
throws 
`bash: venv/bin/activate: No such file or directory`

I have tried playing with this tip: https://trendoceans.com/how-to-resolve-venv-bin-activate-is-not-executable-by-this-user/ but then gave up.
hursh-desai commented 1 year ago

try looking into the install_playground.sh file and seeing if you can manually run each one of the lines. For me I had to manually run python -m venv venv instead of python3 -m venv venv in order to create a venv folder and then I could go on and activate the environment. However, this could only possibly help with #3.

Manamama commented 1 year ago

A good tip, in fact.

Bug 3, i.e. 

The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.
    apt install python3.8-venv

has quickly solved thereby, but now we have something more serious with flask:

Traceback (most recent call last):
  File "/home/.../.local/bin/flask", line 5, in <module>
    from flask.cli import main
  File "/home/.../.local/lib/python3.8/site-packages/flask/__init__.py", line 5, in <module>
    from .app import Flask as Flask
  File "/home/.../.local/lib/python3.8/site-packages/flask/app.py", line 34, in <module>
    from . import cli
  File "/home/.../.local/lib/python3.8/site-packages/flask/cli.py", line 15, in <module>
    from click.core import ParameterSource
ImportError: cannot import name 'ParameterSource' from 'click.core' (/usr/lib/python3/dist-packages/click/core.py)

After some research, and futher trial plus error, I have found that only this tip https://www.pythonpool.com/solved-importerror-cannot-import-name/ did help: apt-get install python3-flask -> finally:

 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:8000/

A tip to the author: do test these instructions on a vanilla, clean linux box, to check for such missing dev etc. modules, as many of the end-users are not experts (myself included).

Manamama commented 1 year ago

The front-end still throws up new errors:

  1. yarn start -> 00h00m00s 0/0: : ERROR: [Errno 2] No such file or directory: 'start'

solved only by https://stackoverflow.com/questions/53471063/yarn-error-there-are-no-scenarios-must-have-at-least-one

You've got the wrong yarn. The yarn you're executing comes from the cmdtest package. Uninstalling cmdtest first should fix this:

sudo apt remove cmdtest

Once you've uninstalled it, run the commands below to install yarn properly:

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt update
sudo apt install yarn

but then:

  1. React something: 
    
yarn run v1.22.19
$ react-scripts start
/bin/sh: 1: react-scripts: not found
error Command failed with exit code 127.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

solved by:
`sudo yarn install
`

Have pity on us, non-techies! Until now I have never even heard of yarn and such.
Manamama commented 1 year ago

Let me close it by now, as I had suggested the solutions above.

saharmor commented 1 year ago

@Manamama sorry you had to go through all this trouble. Did you manage to run it? Feel free to document the steps you went through for running this playground on a clean linux in a new Readme PR and I'll merge it.

Manamama commented 1 year ago

Dear @saharmor 👍🏽 Yes, it compiled and ran, after all these hoops above. I do not have time for sprucing it up for the Readme: just try to install on a clean Ubuntu and look up the similar errors to update the requirements.txt etc. (Especially these two yarns were tricky to me: a non-Dev.) Have a fine day!

Manamama commented 1 year ago

A quick FYI, and sorry for necrobumping this thread, @saharmor - I have run a quick sanity check today. Here is what I am seeing now, after some days toying with other python projects and thus updating some python modules:

my_account@my_box:~/Downloads/whisper-playground$ sh install_playground.sh
yarn install v1.22.19
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
warning " > @material-ui/core@4.12.4" has incorrect peer dependency "react@^16.8.0 || ^17.0.0".
warning " > @material-ui/core@4.12.4" has incorrect peer dependency "react-dom@^16.8.0 || ^17.0.0".
warning "@material-ui/core > @material-ui/styles@4.11.5" has incorrect peer dependency "react@^16.8.0 || ^17.0.0".
warning "@material-ui/core > @material-ui/styles@4.11.5" has incorrect peer dependency "react-dom@^16.8.0 || ^17.0.0".
warning "@material-ui/core > @material-ui/system@4.12.2" has incorrect peer dependency "react@^16.8.0 || ^17.0.0".
warning "@material-ui/core > @material-ui/system@4.12.2" has incorrect peer dependency "react-dom@^16.8.0 || ^17.0.0".
warning "@material-ui/core > @material-ui/utils@4.11.3" has incorrect peer dependency "react@^16.8.0 || ^17.0.0".
warning "@material-ui/core > @material-ui/utils@4.11.3" has incorrect peer dependency "react-dom@^16.8.0 || ^17.0.0".
warning " > @material-ui/lab@4.0.0-alpha.61" has incorrect peer dependency "react@^16.8.0 || ^17.0.0".
warning " > @material-ui/lab@4.0.0-alpha.61" has incorrect peer dependency "react-dom@^16.8.0 || ^17.0.0".
warning " > @testing-library/user-event@7.2.1" has unmet peer dependency "@testing-library/dom@>=5".
warning " > bootstrap@5.2.1" has unmet peer dependency "@popperjs/core@^2.11.6".
warning " > react-mic@12.4.6" has unmet peer dependency "prop-types@^15.5.10".
warning " > react-mic@12.4.6" has incorrect peer dependency "react@16.x".
warning "react-mic > react-ga@2.7.0" has incorrect peer dependency "react@^15.6.2 || ^16.0".
warning "react-scripts > @typescript-eslint/eslint-plugin > tsutils@3.21.0" has unmet peer dependency "typescript@>=2.8.0 || >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta".
warning " > react-spinners@0.10.6" has incorrect peer dependency "react@^16.0.0 || ^17.0.0".
warning " > react-spinners@0.10.6" has incorrect peer dependency "react-dom@^16.0.0 || ^17.0.0".
[4/4] Building fresh packages...
Done in 42.26s.
install_playground.sh: 7: source: not found
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: wheel in /usr/lib/python3/dist-packages (0.34.2)
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu113
Collecting git+https://github.com/openai/whisper.git (from -r requirements.txt (line 8))
  Cloning https://github.com/openai/whisper.git to /tmp/pip-req-build-f3rpb53k
  Running command git clone --filter=blob:none --quiet https://github.com/openai/whisper.git /tmp/pip-req-build-f3rpb53k
  Resolved https://github.com/openai/whisper.git to commit d18e9ea5dd2ca57c697e8e55f9e654f06ede25d0
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy in /home/my_account/.local/lib/python3.8/site-packages (from -r requirements.txt (line 1)) (1.21.0)
Requirement already satisfied: tqdm in /home/my_account/.local/lib/python3.8/site-packages (from -r requirements.txt (line 2)) (4.62.3)
Requirement already satisfied: transformers>=4.19.0 in /home/my_account/.local/lib/python3.8/site-packages (from -r requirements.txt (line 3)) (4.22.2)
Requirement already satisfied: ffmpeg-python==0.2.0 in /home/my_account/.local/lib/python3.8/site-packages (from -r requirements.txt (line 4)) (0.2.0)
Requirement already satisfied: pyaudio in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 5)) (0.2.12)
Requirement already satisfied: SpeechRecognition in /home/my_account/.local/lib/python3.8/site-packages (from -r requirements.txt (line 6)) (3.8.1)
Requirement already satisfied: pydub in /home/my_account/.local/lib/python3.8/site-packages (from -r requirements.txt (line 7)) (0.25.1)
Requirement already satisfied: torch in /home/my_account/.local/lib/python3.8/site-packages (from -r requirements.txt (line 10)) (1.12.1)
Requirement already satisfied: flask in /usr/lib/python3/dist-packages (from -r requirements.txt (line 11)) (1.1.1)
Requirement already satisfied: flask_cors in /home/my_account/.local/lib/python3.8/site-packages (from -r requirements.txt (line 12)) (3.0.10)
Requirement already satisfied: future in /usr/lib/python3/dist-packages (from ffmpeg-python==0.2.0->-r requirements.txt (line 4)) (0.18.2)
Requirement already satisfied: pyyaml>=5.1 in /home/my_account/.local/lib/python3.8/site-packages (from transformers>=4.19.0->-r requirements.txt (line 3)) (6.0)
Requirement already satisfied: requests in /home/my_account/.local/lib/python3.8/site-packages (from transformers>=4.19.0->-r requirements.txt (line 3)) (2.28.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.9.0 in /home/my_account/.local/lib/python3.8/site-packages (from transformers>=4.19.0->-r requirements.txt (line 3)) (0.10.0)
Requirement already satisfied: filelock in /home/my_account/.local/lib/python3.8/site-packages (from transformers>=4.19.0->-r requirements.txt (line 3)) (3.8.0)
Requirement already satisfied: tokenizers!=0.11.3,<0.13,>=0.11.1 in /home/my_account/.local/lib/python3.8/site-packages (from transformers>=4.19.0->-r requirements.txt (line 3)) (0.12.1)
Requirement already satisfied: packaging>=20.0 in /home/my_account/.local/lib/python3.8/site-packages (from transformers>=4.19.0->-r requirements.txt (line 3)) (21.3)
Requirement already satisfied: regex!=2019.12.17 in /home/my_account/.local/lib/python3.8/site-packages (from transformers>=4.19.0->-r requirements.txt (line 3)) (2022.3.2)
Requirement already satisfied: more-itertools in /home/my_account/.local/lib/python3.8/site-packages (from whisper==1.0->-r requirements.txt (line 8)) (8.14.0)
Requirement already satisfied: typing-extensions in /home/my_account/.local/lib/python3.8/site-packages (from torch->-r requirements.txt (line 10)) (4.3.0)
Requirement already satisfied: Six in /usr/lib/python3/dist-packages (from flask_cors->-r requirements.txt (line 12)) (1.14.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/lib/python3/dist-packages (from packaging>=20.0->transformers>=4.19.0->-r requirements.txt (line 3)) (2.4.6)
Requirement already satisfied: charset-normalizer<3,>=2 in /home/my_account/.local/lib/python3.8/site-packages (from requests->transformers>=4.19.0->-r requirements.txt (line 3)) (2.1.1)
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests->transformers>=4.19.0->-r requirements.txt (line 3)) (2019.11.28)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3/dist-packages (from requests->transformers>=4.19.0->-r requirements.txt (line 3)) (1.25.8)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests->transformers>=4.19.0->-r requirements.txt (line 3)) (2.8)
my_account@my_box:~/Downloads/whisper-playground$ cd backend && source venv/bin/activate && flask run --port 8000
Traceback (most recent call last):
  File "/home/my_account/.local/bin/flask", line 5, in <module>
    from flask.cli import main
  File "/usr/lib/python3/dist-packages/flask/__init__.py", line 21, in <module>
    from .app import Flask
  File "/usr/lib/python3/dist-packages/flask/app.py", line 32, in <module>
    from werkzeug.wrappers import BaseResponse
ImportError: cannot import name 'BaseResponse' from 'werkzeug.wrappers' (/home/my_account/.local/lib/python3.8/site-packages/werkzeug/wrappers/__init__.py)

In short, it had worked back then, and it does not work now, at a recompile. No time to troubleshoot it, of course, but here is a suggestion of the root cause: https://github.com/mjmeijer/donald_2021/issues/4