schneiderfelipe commented 2 years ago

I did exactly how explained in the documentation:

$ pip install numpy Cython
Collecting numpy
  Using cached numpy-1.22.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB)
Collecting Cython
  Using cached Cython-0.29.28-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (1.9 MB)
Installing collected packages: numpy, Cython
Successfully installed Cython-0.29.28 numpy-1.22.3

~ took 7s
$ sudo apt install libsndfile-dev fluidsynth ffmpeg
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'libsndfile1-dev' instead of 'libsndfile-dev'
ffmpeg is already the newest version (7:4.4-6ubuntu5).
fluidsynth is already the newest version (2.1.7-1.1).
libsndfile1-dev is already the newest version (1.0.31-2).
0 to upgrade, 0 to newly install, 0 to remove and 0 not to upgrade.

~
$ pip install omnizart
Collecting omnizart
  Using cached omnizart-0.1.0-py3-none-any.whl (3.5 MB)
Collecting numba==0.48
  Using cached numba-0.48.0-cp39-cp39-linux_x86_64.whl
Collecting vamp<2.0.0,>=1.1.0
  Using cached vamp-1.1.0-cp39-cp39-linux_x86_64.whl
Collecting madmom<0.17.0,>=0.16.1
  Using cached madmom-0.16.1-cp39-cp39-linux_x86_64.whl
Collecting opencv-python<5.0.0,>=4.4.0
  Using cached opencv_python-4.5.5.64-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (60.5 MB)
Collecting tensorflow<3.0.0,>=2.3.0
  Using cached tensorflow-2.8.0-cp39-cp39-manylinux2010_x86_64.whl (497.6 MB)
Collecting jsonschema<4.0.0,>=3.2.0
  Using cached jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Collecting pretty_midi<0.3.0,>=0.2.9
  Using cached pretty_midi-0.2.9-py3-none-any.whl
Requirement already satisfied: urllib3<2.0.0,>=1.25.11 in /usr/lib/python3/dist-packages (from omnizart) (1.26.5)
Requirement already satisfied: pyyaml<6.0.0,>=5.3.1 in /usr/lib/python3/dist-packages (from omnizart) (5.3.1)
Collecting librosa<0.9.0,>=0.8.0
  Using cached librosa-0.8.1-py3-none-any.whl (203 kB)
Collecting pyfluidsynth<2.0.0,>=1.2.5
  Using cached pyFluidSynth-1.3.0-py3-none-any.whl (18 kB)
Collecting tqdm<5.0.0,>=4.49.0
  Using cached tqdm-4.64.0-py2.py3-none-any.whl (78 kB)
Requirement already satisfied: click<8.0.0,>=7.1.2 in /usr/lib/python3/dist-packages (from omnizart) (7.1.2)
Collecting llvmlite<0.32.0,>=0.31.0dev0
  Using cached llvmlite-0.31.0.tar.gz (110 kB)
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from numba==0.48->omnizart) (52.0.0)
Requirement already satisfied: numpy>=1.15 in ./.local/lib/python3.9/site-packages (from numba==0.48->omnizart) (1.22.3)
Requirement already satisfied: six>=1.11.0 in /usr/lib/python3/dist-packages (from jsonschema<4.0.0,>=3.2.0->omnizart) (1.16.0)
Collecting attrs>=17.4.0
  Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pyrsistent>=0.14.0
  Using cached pyrsistent-0.18.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (115 kB)
Collecting packaging>=20.0
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting decorator>=3.0.0
  Using cached decorator-5.1.1-py3-none-any.whl (9.1 kB)
Collecting audioread>=2.0.0
  Using cached audioread-2.1.9-py3-none-any.whl
Collecting joblib>=0.14
  Using cached joblib-1.1.0-py2.py3-none-any.whl (306 kB)
Collecting soundfile>=0.10.2
  Using cached SoundFile-0.10.3.post1-py2.py3-none-any.whl (21 kB)
Collecting pooch>=1.0
  Using cached pooch-1.6.0-py3-none-any.whl (56 kB)
Collecting resampy>=0.2.2
  Using cached resampy-0.2.2-py3-none-any.whl
Collecting scikit-learn!=0.19.0,>=0.14.0
  Using cached scikit_learn-1.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (26.4 MB)
Collecting scipy>=1.0.0
  Using cached scipy-1.8.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (42.1 MB)
Collecting mido>=1.2.8
  Using cached mido-1.2.10-py2.py3-none-any.whl (51 kB)
Requirement already satisfied: cython>=0.25 in ./.local/lib/python3.9/site-packages (from madmom<0.17.0,>=0.16.1->omnizart) (0.29.28)
Collecting pyparsing!=3.0.5,>=2.0.2
  Using cached pyparsing-3.0.8-py3-none-any.whl (98 kB)
Requirement already satisfied: requests>=2.19.0 in /usr/lib/python3/dist-packages (from pooch>=1.0->librosa<0.9.0,>=0.8.0->omnizart) (2.25.1)
Collecting appdirs>=1.3.0
  Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting threadpoolctl>=2.0.0
  Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting cffi>=1.0
  Using cached cffi-1.15.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (444 kB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting libclang>=9.0.1
  Using cached libclang-14.0.1-py2.py3-none-manylinux1_x86_64.whl (14.5 MB)
Collecting keras-preprocessing>=1.1.1
  Using cached Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
Collecting tensorflow-io-gcs-filesystem>=0.23.1
  Using cached tensorflow_io_gcs_filesystem-0.25.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting flatbuffers>=1.12
  Using cached flatbuffers-2.0-py2.py3-none-any.whl (26 kB)
Collecting opt-einsum>=2.3.2
  Using cached opt_einsum-3.3.0-py3-none-any.whl (65 kB)
Requirement already satisfied: protobuf>=3.9.2 in /usr/lib/python3/dist-packages (from tensorflow<3.0.0,>=2.3.0->omnizart) (3.12.4)
Collecting absl-py>=0.4.0
  Using cached absl_py-1.0.0-py3-none-any.whl (126 kB)
Collecting h5py>=2.9.0
  Using cached h5py-3.6.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (4.5 MB)
Collecting astunparse>=1.6.0
  Using cached astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting typing-extensions>=3.6.6
  Using cached typing_extensions-4.2.0-py3-none-any.whl (24 kB)
Collecting keras<2.9,>=2.8.0rc0
  Using cached keras-2.8.0-py2.py3-none-any.whl (1.4 MB)
Collecting tf-estimator-nightly==2.8.0.dev2021122109
  Using cached tf_estimator_nightly-2.8.0.dev2021122109-py2.py3-none-any.whl (462 kB)
Collecting termcolor>=1.1.0
  Using cached termcolor-1.1.0-py3-none-any.whl
Collecting google-pasta>=0.1.1
  Using cached google_pasta-0.2.0-py3-none-any.whl (57 kB)
Collecting grpcio<2.0,>=1.24.3
  Using cached grpcio-1.46.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.4 MB)
Collecting wrapt>=1.11.0
  Using cached wrapt-1.14.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (77 kB)
Collecting gast>=0.2.1
  Using cached gast-0.5.3-py3-none-any.whl (19 kB)
Collecting tensorboard<2.9,>=2.8
  Using cached tensorboard-2.8.0-py3-none-any.whl (5.8 MB)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/lib/python3/dist-packages (from astunparse>=1.6.0->tensorflow<3.0.0,>=2.3.0->omnizart) (0.34.2)
Collecting markdown>=2.6.8
  Using cached Markdown-3.3.7-py3-none-any.whl (97 kB)
Collecting tensorboard-data-server<0.7.0,>=0.6.0
  Using cached tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl (4.9 MB)
Collecting google-auth-oauthlib<0.5,>=0.4.1
  Using cached google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting werkzeug>=0.11.15
  Using cached Werkzeug-2.1.2-py3-none-any.whl (224 kB)
Collecting tensorboard-plugin-wit>=1.6.0
  Using cached tensorboard_plugin_wit-1.8.1-py3-none-any.whl (781 kB)
Collecting google-auth<3,>=1.6.3
  Using cached google_auth-2.6.6-py2.py3-none-any.whl (156 kB)
Collecting cachetools<6.0,>=2.0.0
  Using cached cachetools-5.0.0-py3-none-any.whl (9.1 kB)
Collecting rsa<5,>=3.1.4
  Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
  Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting requests-oauthlib>=0.7.0
  Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting importlib-metadata>=4.4
  Using cached importlib_metadata-4.11.3-py3-none-any.whl (18 kB)
Requirement already satisfied: zipp>=0.5 in /usr/lib/python3/dist-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard<2.9,>=2.8->tensorflow<3.0.0,>=2.3.0->omnizart) (1.0.0)
Collecting pyasn1<0.5.0,>=0.4.6
  Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/lib/python3/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.9,>=2.8->tensorflow<3.0.0,>=2.3.0->omnizart) (3.1.0)
Building wheels for collected packages: llvmlite
  Building wheel for llvmlite (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/setup.py'"'"'; __file__='"'"'/tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-b88zt9pn
       cwd: /tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/
  Complete output (15 lines):
  running bdist_wheel
  /usr/bin/python3 /tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/ffi/build.py
  LLVM version... 13.0.0

  Traceback (most recent call last):
    File "/tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/ffi/build.py", line 168, in <module>
      main()
    File "/tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/ffi/build.py", line 158, in main
      main_posix('linux', '.so')
    File "/tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/ffi/build.py", line 120, in main_posix
      raise RuntimeError(msg)
  RuntimeError: Building llvmlite requires LLVM 7.0+ Be sure to set LLVM_CONFIG to the right executable path.
  Read the documentation at http://llvmlite.pydata.org/ for more information about building llvmlite.

  error: command '/usr/bin/python3' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for llvmlite
  Running setup.py clean for llvmlite
Failed to build llvmlite
Installing collected packages: pyasn1, rsa, pyasn1-modules, cachetools, requests-oauthlib, pyparsing, pycparser, llvmlite, importlib-metadata, google-auth, werkzeug, threadpoolctl, tensorboard-plugin-wit, tensorboard-data-server, scipy, packaging, numba, markdown, joblib, grpcio, google-auth-oauthlib, cffi, appdirs, absl-py, wrapt, typing-extensions, tf-estimator-nightly, termcolor, tensorflow-io-gcs-filesystem, tensorboard, soundfile, scikit-learn, resampy, pyrsistent, pooch, opt-einsum, mido, libclang, keras-preprocessing, keras, h5py, google-pasta, gast, flatbuffers, decorator, audioread, attrs, astunparse, vamp, tqdm, tensorflow, pyfluidsynth, pretty-midi, opencv-python, madmom, librosa, jsonschema, omnizart
    Running setup.py install for llvmlite ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/setup.py'"'"'; __file__='"'"'/tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-wnrsu259/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/schneider/.local/include/python3.9/llvmlite
         cwd: /tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/
    Complete output (18 lines):
    running install
    running build
    got version from file /tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/llvmlite/_version.py {'version': '0.31.0', 'full': 'fe7d985f6421d87f613bd414479d29d912771562'}
    running build_ext
    /usr/bin/python3 /tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/ffi/build.py
    LLVM version... 13.0.0

    Traceback (most recent call last):
      File "/tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/ffi/build.py", line 168, in <module>
        main()
      File "/tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/ffi/build.py", line 158, in main
        main_posix('linux', '.so')
      File "/tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/ffi/build.py", line 120, in main_posix
        raise RuntimeError(msg)
    RuntimeError: Building llvmlite requires LLVM 7.0+ Be sure to set LLVM_CONFIG to the right executable path.
    Read the documentation at http://llvmlite.pydata.org/ for more information about building llvmlite.

    error: command '/usr/bin/python3' failed with exit code 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/setup.py'"'"'; __file__='"'"'/tmp/pip-install-h5twqgap/llvmlite_69ce54ce335e452e9d0440e39568c630/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-wnrsu259/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/schneider/.local/include/python3.9/llvmlite Check the logs for full command output.

This seems to be related to #26 and #29.

Some information about the environment:

$ neofetch
            .-/+oossssoo+/-.               schneider@lenovo
        `:+ssssssssssssssssss+:`           ----------------
      -+ssssssssssssssssssyyssss+-         OS: Ubuntu 21.10 x86_64
    .ossssssssssssssssssdMMMNysssso.       Host: 80R0 Lenovo G50-8
   /ssssssssssshdmmNNmmyNMMMMhssssss/      Kernel: 5.13.0-40-gener
  +ssssssssshmydMMMMMMMNddddyssssssss+     Uptime: 4 hours, 51 min
 /sssssssshNMMMyhhyyyyhmNMMMNhssssssss/    Packages: 2739 (dpkg),
.ssssssssdMMMNhsssssssssshNMMMdssssssss.   Shell: zsh 5.8
+sssshhhyNMMNyssssssssssssyNMMMysssssss+   Resolution: 1440x900
ossyNMMMNyMMhsssssssssssssshmmmhssssssso   DE: GNOME 40.5
ossyNMMMNyMMhsssssssssssssshmmmhssssssso   WM: Mutter
+sssshhhyNMMNyssssssssssssyNMMMysssssss+   WM Theme: Adwaita
.ssssssssdMMMNhsssssssssshNMMMdssssssss.   Theme: Yaru-dark [GTK2/
 /sssssssshNMMMyhhyyyyhdNMMMNhssssssss/    Icons: Yaru [GTK2/3]
  +sssssssssdmydMMMMMMMMddddyssssssss+     CPU: Intel i7-5500U (4)
   /ssssssssssshdmNNNNmyNMMMMhssssss/      GPU: Intel HD Graphics
    .ossssssssssssssssssdMMMNysssso.       Memory: 3708MiB / 7866M
      -+sssssssssssssssssyyyssss+-
        `:+ssssssssssssssssss+:`
            .-/+oossssoo+/-.

~
$ python3 --version                                                Python 3.9.7

~
$ pip --version
pip 20.3.4 from /usr/lib/python3/dist-packages/pip (python 3.9)

schneiderfelipe commented 2 years ago

Follow-up

The development installation instructions work fine (both make install and make install-dev; poetry install works fine as well). But the following fails:

$ pip install .
Defaulting to user installation because normal site-packages is not writeable
Processing /home/schneider/Dropbox/omnizart
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting urllib3==1.26.4
  Downloading urllib3-1.26.4-py2.py3-none-any.whl (153 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━ 153.3/153.3 KB 10.8 MB/s eta 0:00:00
Collecting mir_eval<0.7,>=0.6
  Downloading mir_eval-0.6.tar.gz (87 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━ 87.9/87.9 KB 25.6 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting pyfluidsynth<2.0.0,>=1.2.5
  Using cached pyFluidSynth-1.3.0-py3-none-any.whl (18 kB)
Collecting pretty_midi<0.3.0,>=0.2.9
  Using cached pretty_midi-0.2.9-py3-none-any.whl
Collecting tqdm<5.0.0,>=4.49.0
  Using cached tqdm-4.64.0-py2.py3-none-any.whl (78 kB)
Collecting spleeter<3.0.0,>=2.3.0
  Downloading spleeter-2.3.0-py3-none-any.whl (51 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.1/51.1 KB 16.3 MB/s eta 0:00:00
Collecting vamp<2.0.0,>=1.1.0
  Using cached vamp-1.1.0-cp39-cp39-linux_x86_64.whl
Requirement already satisfied: click<8.0.0,>=7.1.2 in /usr/lib/python3/dist-packages (from omnizart==0.5.0) (7.1.2)
Collecting madmom<0.17.0,>=0.16.1
  Using cached madmom-0.16.1-cp39-cp39-linux_x86_64.whl
Collecting pillow<9.0.0,>=8.3.2
  Downloading Pillow-8.4.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.1/3.1 MB 63.8 MB/s eta 0:00:00
Collecting jsonschema<4.0.0,>=3.2.0
  Using cached jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Requirement already satisfied: pyyaml<6.0.0,>=5.3.1 in /usr/lib/python3/dist-packages (from omnizart==0.5.0) (5.3.1)
Collecting librosa<0.9.0,>=0.8.0
  Using cached librosa-0.8.1-py3-none-any.whl (203 kB)
Collecting tensorflow==2.5.0
  Downloading tensorflow-2.5.0-cp39-cp39-manylinux2010_x86_64.whl (454.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━ 454.4/454.4 MB 5.4 MB/s eta 0:00:00
ERROR: Package 'omnizart' requires a different Python: 3.9.7 not in '<3.9,>=3.6.1'

which is an error that's not shown when installing from PyPI. It installs fine with pip install -U . with the following modification:

$ git diff
diff --git a/pyproject.toml b/pyproject.toml
index c65b1ab..5a23966 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -23,7 +23,7 @@ classifiers = [
 exclude = ["weights*"]

 [tool.poetry.dependencies]
-python = ">=3.6.1,<3.9"
+python = ">=3.6.1,<=3.9.7"
 click = "^7.1.2"
 pretty_midi = "^0.2.9"
 librosa = "^0.8.0"
@@ -62,4 +62,3 @@ target-version = ['py36']
 [build-system]
 requires = ["poetry>=0.12"]
 build-backend = "poetry.masonry.api"

BreezeWhite commented 2 years ago

Hi, thanks for the extensive information! It's really inspiring to see many like you interested in this project. As for current, I am not sure whether there is any better workaround to the pip install approach without so many bug, neither do I want to force users to go with the development installation approach. That's to scary for most non-geeks. After all, the best I can do is to reference the issue in the document.

schneiderfelipe commented 2 years ago

After I understood the issue with Python 3.9+ in https://github.com/Music-and-Culture-Technology-Lab/omnizart/issues/70#issuecomment-1120142854, I kind of get it. Maybe there will be a solution after tensorflow v2.9.0 is released (which will support Python 3.9)?

BreezeWhite commented 2 years ago

If that doesn't take too much efforts, I would consider it, since I am not in the lab and having another job now.

Music-and-Culture-Technology-Lab / omnizart

Installation instructions fail under Ubuntu 21.10 #65

Follow-up