search

Selur / hybrid-vapoursynth-addon

Adding Vapoursynth and plugins to Hybrid
11 stars 4 forks source link

vspipe is nowhere to be found #1

Closed Extarys closed 4 years ago

Extarys commented 4 years ago

I installed Hybrid today and found out I needed VapourSynth for some deinterlacing. Good I found this repo :D But I can't figure out where the bin files are.

The steps I did:

  1. git clone the repo to /media/hdd/git/hybrid-vapoursynth-addon-master
  2. make 'build-vapoursynth.sh' executable and execute
  3. everything ran fine
  4. cannot find vspipe executable in the home dir, temp dir or any system folders

The script copied some files from /media/hdd/git/hybrid-vapoursynth-addon-master to /home/me/opt/vapoursynth.

I tried running install-vs.sh that was also copied in my home dir, but it says the folder in the home directory already exists.

I also cannot find the ffmpeg bin that was compiled with this script. Or is this used only to get the nasm bin?

OS Linux Mint 20 Python 3.8

image

Selur commented 4 years ago

I also cannot find the ffmpeg bin that was compiled with this script. Or is this used only to get the nasm bin?

FFmpeg is only compiled as part as dependency of some of the vapoursynth plugins and not as standalone, so no surprise you can't find it.

vspipe, should be under /usr/local/bin/vspipe ('which vsipe' should find it)

Extarys commented 4 years ago

For ffmpeg I wasn't sure, thanks!

I have no vspipe at /usr/local/bin :thinking:

I removed the build folder on my temp dir and the opt folder in my home directory and recompiled. Here is the output: https://pastebin.com/TGBVa409

Selur commented 4 years ago

basically afk today, but try removing the '--disable-vspipe' in the vapoursynth-build.sh ;)

Extarys commented 4 years ago

ARGH :man_facepalming: why is it there :disappointed: Thanks for dropping by, even if you are afk today buddy!

Hmm, it appears it still didn't create the vspipe file. I think it's the install-vs.sh that is suppose to create it, although I'm no bash expert. And that file refuses to run as the folder in the home directory already exists - created by the build-vapoursynth script.

I would appreciate if you could take a look when you have time, I'll wait - I wanted to rip some DVDs I have here but handbrake deinterlace didn't do a good enough job for me.

:heart:

Selur commented 4 years ago

Okay, I looked at it and you are right vspipe should be build in install-vs.sh and

if [ -e "$vsprefix" ]; then
  echo "Path \`$vsprefix' already exists!"
  exit 1
fi

set -e

cd "$(dirname "$0")"

should be replaced with:

set -e

cd "$sprefix"

problem is with python3.8 there seem to be some missing dependencies. -> I hope I find a fix on the weekend

Extarys commented 4 years ago

I had to change libopenjp2-?-dev to libopenjp2-7-dev in the build-vapoursynth file for it to work (I forgot to mention this earlier).

I'm just leaving the logs here in case it can help you, I won't try too much today as I don't have enough patience to play with this :disappointed: . I think the script miss a python dependency (testresources), like you said yesterday :smile:

Trying to install vspipe:

$ ./install-vs.sh 
Copy files.cp: 'include' and '/home/me/opt/vapoursynth/include' are the same file
cp: 'lib' and '/home/me/opt/vapoursynth/lib' are the same file
cp: cannot stat 'vsplugins': No such file or directory

So I tried to run build-plugins.sh:

$ ./build-plugins.sh 
+ sudo apt update
[sudo] password for me:         
Hit:1 http://mirror.ca-tr.kamatera.com/ubuntu focal InRelease
Get:2 http://mirror.ca-tr.kamatera.com/ubuntu focal-updates InRelease [111 kB] 
Hit:3 http://security.debian.org buster/updates InRelease                      
Get:4 http://mirror.ca-tr.kamatera.com/ubuntu focal-backports InRelease [98.3 kB]
Hit:5 https://deb.debian.org/debian buster InRelease                           
Get:6 https://deb.debian.org/debian buster-updates InRelease [51.9 kB]
Get:7 http://mirror.ca-tr.kamatera.com/ubuntu focal-updates/main amd64 DEP-11 Metadata [196 kB]
Get:8 http://mirror.ca-tr.kamatera.com/ubuntu focal-updates/universe amd64 DEP-11 Metadata [177 kB]
Get:9 http://mirror.ca-tr.kamatera.com/ubuntu focal-updates/multiverse amd64 DEP-11 Metadata [2,468 B]
Hit:10 http://archive.canonical.com/ubuntu focal InRelease                     
Get:11 http://mirror.ca-tr.kamatera.com/ubuntu focal-backports/universe amd64 DEP-11 Metadata [1,972 B]
Hit:12 http://ppa.launchpad.net/font-manager/staging/ubuntu focal InRelease    
Get:13 http://security.ubuntu.com/ubuntu focal-security InRelease [107 kB]
Hit:14 http://ppa.launchpad.net/lutris-team/lutris/ubuntu focal InRelease      
Ign:15 http://mirror.csclub.uwaterloo.ca/linuxmint-packages ulyana InRelease
Get:16 http://security.ubuntu.com/ubuntu focal-security/main amd64 DEP-11 Metadata [21.2 kB]
Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 DEP-11 Metadata [36.0 kB]
Hit:18 http://mirror.csclub.uwaterloo.ca/linuxmint-packages ulyana Release
Fetched 803 kB in 2s (523 kB/s)
Reading package lists... Done
Building dependency tree       
Reading state information... Done
All packages are up to date.
+ sudo apt upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
+ sudo apt install --no-install-recommends build-essential cmake yasm git wget mercurial unzip p7zip-full python3-pip zlib1g-dev libfftw3-dev libopencv-dev ocl-icd-opencl-dev opencl-headers libboost-dev libboost-filesystem-dev libboost-system-dev libbluray-dev libpng-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
build-essential is already the newest version (12.8ubuntu1).
cmake is already the newest version (3.16.3-1ubuntu1).
git is already the newest version (1:2.25.1-1ubuntu3).
libboost-dev is already the newest version (1.71.0.0ubuntu2).
libboost-filesystem-dev is already the newest version (1.71.0.0ubuntu2).
libboost-system-dev is already the newest version (1.71.0.0ubuntu2).
libfftw3-dev is already the newest version (3.3.8-2ubuntu1).
libpng-dev is already the newest version (1.6.37-2).
ocl-icd-opencl-dev is already the newest version (2.2.11-1ubuntu1).
unzip is already the newest version (6.0-25ubuntu1).
wget is already the newest version (1.20.3-1ubuntu1).
zlib1g-dev is already the newest version (1:1.2.11.dfsg-2ubuntu1).
libbluray-dev is already the newest version (1:1.2.0-1).
libopencv-dev is already the newest version (4.2.0+dfsg-5).
mercurial is already the newest version (5.3.1-1ubuntu1).
opencl-headers is already the newest version (2.2~2019.08.06-g0d5f18c-1).
p7zip-full is already the newest version (16.02+dfsg-7build1).
python3-pip is already the newest version (20.0.2-5ubuntu1).
yasm is already the newest version (1.3.0-2ubuntu1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
+ set +x
ERROR: launchpadlib 1.10.13 requires testresources, which is not installed.

Build plugins:
 addgrain (1/62) ... done
 awarpsharp2 (2/62) ... done
 bifrost (3/62) ... done
 bilateral (4/62) ... done
 bm3d (5/62) ... done
 cas (6/62) ... done
 ctmf (7/62) ... done
 d2vsource (8/62) ... done
 dctfilter (9/62) ... done
 deblockpp7 (10/62) ... done
 deblock (11/62) ... done
 decross (12/62) ... done
 dedot (13/62) ... done
 degrainmedian (14/62) ... done
 depan (15/62) ... done
 descale (16/62) ... done
 dfttest (17/62) ... done
 dotkill (18/62) ... done
 eedi2 (19/62) ... done
 eedi3m (20/62) ... done
 f3kdb (21/62) ... done
 ffms2 (22/62) ... done
 fft3dfilter (23/62) ... failed
 fixtelecinedfades (24/62) ... done
 fluxsmooth (25/62) ... done
 fmtconv (26/62) ... done
 hqdn3d (27/62) ... done
 it (28/62) ... done
 knlmeanscl (29/62) ... done
 lghost (30/62) ... done
 minideen (31/62) ... done
 msmoosh (32/62) ... done
 mvtoolssf (33/62) ... done
 mvtools (34/62) ... done
 neofft3d (35/62) ... done
 nnedi3cl (36/62) ... done
 nnedi3 (37/62) ... done
 readmpls (38/62) ... done
 retinex (39/62) ... done
 sangnommod (40/62) ... done
 sangnom (41/62) ... done
 scenechange (42/62) ... done
 ssiq (43/62) ... done
 svpflow (44/62) ... done
 tcanny (45/62) ... done
 tdeintmod (46/62) ... done
 templinearapproximate (47/62) ... done
 temporalmedian (48/62) ... done
 temporalsoften2 (49/62) ... done
 temporalsoften (50/62) ... done
 timecube (51/62) ... done
 tonemap (52/62) ... done
 ttempsmooth (53/62) ... done
 vaguedenoiser (54/62) ... done
 vcmod (55/62) ... done
 vfrtocfr (56/62) ... done
 vsimagereader (57/62) ... done
 vslsmashsource (58/62) ... done
 vsrawsource (59/62) ... done
 vsznedi3 (60/62) ... done
 waifu2x-w2xc (61/62) ... done
 yadifmod (62/62) ... done

ERROR: Exception:
Traceback (most recent call last):
  File "/usr/lib/python3.8/shutil.py", line 788, in move
    os.rename(src, real_dst)
PermissionError: [Errno 13] Permission denied: '/usr/lib/python3/dist-packages/wheel' -> '/tmp/pip-uninstall-b8ucvvxh'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/base_command.py", line 186, in _main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/_internal/commands/uninstall.py", line 78, in run
    uninstall_pathset = req.uninstall(
  File "/usr/lib/python3/dist-packages/pip/_internal/req/req_install.py", line 687, in uninstall
    uninstalled_pathset.remove(auto_confirm, verbose)
  File "/usr/lib/python3/dist-packages/pip/_internal/req/req_uninstall.py", line 394, in remove
    moved.stash(path)
  File "/usr/lib/python3/dist-packages/pip/_internal/req/req_uninstall.py", line 283, in stash
    renames(path, new_path)
  File "/usr/lib/python3/dist-packages/pip/_internal/utils/misc.py", line 334, in renames
    shutil.move(old, new)
  File "/usr/lib/python3.8/shutil.py", line 800, in move
    rmtree(src)
  File "/usr/lib/python3.8/shutil.py", line 715, in rmtree
    _rmtree_safe_fd(fd, path, onerror)
  File "/usr/lib/python3.8/shutil.py", line 672, in _rmtree_safe_fd
    onerror(os.unlink, fullname, sys.exc_info())
  File "/usr/lib/python3.8/shutil.py", line 670, in _rmtree_safe_fd
    os.unlink(entry.name, dir_fd=topfd)
PermissionError: [Errno 13] Permission denied: '_version.py'
Selur commented 4 years ago

Delete the install-vs.sh, delete your ~/opt/vapoursynth folder and try: https://pastebin.com/vBABER0f if that works, run the build-plugins.sh.

Cu Selur

Selur commented 4 years ago

nope, there's still something wrong.

Selur commented 4 years ago

No clue whats all broken, my motivation is after 5 hours at a low and I'm more inclined to drop Linux as a supported platform than looking into this much more,... I simply lack the time and motivation to look into it and from the looks of it there are no simple ways to deploy Vapoursynth,..

alain54 commented 4 years ago

Hi Selur,

I'm ALM, do you remember me ? Today I have coded a lightweight Vapoursynth plugin for my labo, and my team works only in Linux :) So:

  1. Yes, install-vs.sh is required, with some slight refresh. I will try to release a new version tomorrow, or monday. My main problem is: why a compilation in $HOME/opt/vapoursynth and an installation in this same directory ? Why not an install in /usr/local, or in a subdirectory of Hybrid ?

  2. The error at the end of build-plugins.sh, the pip3 uninstall error, comes from the pip3 package installation : the option -I is missing. So some packages could be not locally installed and the next uninstall tries to remove the system packages. export PYTHONUSERBASE="$vsprefix" pip3 install -q --upgrade -I --user setuptools wheel # must be installed first pip3 install -q --upgrade -I --user meson ninja

  3. The error(s) in compiling of vapoursynth.c in install-vs.sh is very simple: you use a C++ compiler, g++, on C code that uses the keyword namespace as a variable name. Really bad luck :) Replace g++ -std=c++11 by gcc in install-vs.sh after the #cython comment to get the vapoursynth.so library.

  4. Another possible error in install-vs.sh, but maybe in my system only: pkg-config --libs python3 [nothing] I'm in ubuntu 20.04, up-to-date; I will search tomorrow in pkg-config why no library is given here. The dependencies of the python3 libraries seems to be : -lpython3.8 -lpthread -ldl -lutil -lm -lm If you have the same problem, replace $( pkg-config --libs python3 ) with these dependencies, with the correct sub-version of python3.

All the best, ALM, a.k.a. Alain Filbois

Selur commented 4 years ago

@alain54: Thanks, any help is welcomed, since due to some personal 'real life' problems I totally lack the time and with everything going on atm also the motivation to really look into it. Thanks!

I have no problem to have vapoursynth somewhere else on the system. I just need a 'fixed' location so I can look for it. In the beginning I hoped that a real portable version would be possible and maintain able, but seeing that Python and lots of other dependencies vary a lot and are not really backward compatible that dream died. (Nowadays I'm happy that the tool build script and the build-plugins script work fine. ;))

alain54 commented 4 years ago

Hi Selur,

In attachement, new_scripts.tar.gz

build-plugins.sh, build-vapoursynth.sh, the new little configuration file config.txt and hybrid-vapoursynth-addon/header.sh.

install-vs.sh is no longer useful.

2 source files, env.sh & env.csh are now provided in $VSPREFIX, for sh/bash/zsh... and tcsh/csh.

Without any variable settings, you might compile vapoursynth & 61/62 plugins: ./build-vapoursynth.sh ./build-plugins.sh

So you need to upgrade README.md :)

The only one broken plugin is fft3dfilter, but its programmer modifies actually the code with #include . It seems to me it's been obsolete for 7 years... That's weird.

Careful: I have not run the plugins one by one to test them!

All comments are welcome!

NB: Be careful, use $VSPREFIX and not $vsprefix in the plugin scripts. I had to do an export vsprefix=$VSPREFIX in ./build-plugins.sh Many scripts in build-plugins directory use $vsprefix instead of $VSPREFIX.

All the best,

Alain

Selur commented 4 years ago

Thanks a lot to groggy atm. but I'll look at it and test it all tomorrow :)

Extarys commented 4 years ago

I was... hm... sleeping for the past few days I didn't follow along I'm sorry.

Selur, sorry for your problems, I hope everything gets back on track soon for you <3.

Alain, I've tried your script, it's working! OMG!

I had to manually do this as I was getting some missing packages:

pip3 install Cython
pip3 install testresources

Plugins are still building. I'll report back, but so far so good - very excited!

Selur commented 4 years ago

Okay, finally I came around to do some testing, here's what I found so far: calling ./build-vapoursynth.py on 18.04, build-vapoursynth aborts with:

Usage:   
  pip install [options] <requirement specifier> [package-index-options] ...
  pip install [options] -r <requirements file> [package-index-options] ...
  pip install [options] [-e] <vcs project url> ...
  pip install [options] [-e] <local project path> ...
  pip install [options] <archive url/path> ...

no such option: --no-warn-script-location

-> removing it from the script helped

After that it stopped at

configure: error: Cython required but not found.

adding 'cython3' to the install list fixes that.

After adding

export LD_LIBRARY_PATH=/usr/local/lib
export PYTHONPATH=/usr/local/lib/python3.6/site-packages

to my ~/.bashrc and ~/.profile calling: vspipe --version fails with Failed to initialize VapourSynth environment

Using the build-plugins.sh and headers.sh from the repository works(but config.txt isn't used. (dotkill, fft3dfilter and dotkill failed, but those probably need updates). Using your build-plugins.sh and headers.sh building fails for most of the plugins since meson can't be found,...

Cu Selur

Selur commented 4 years ago

okay '/usr/local/lib/python3.6/site-packages' doesn't exist any more,.. due to the install.sh being removed,.. using: PYTHONPATH=/home/selur/opt/vapoursynth/lib/python3.6/site-packages seems to work,..

Selur commented 4 years ago

Made a new commit with which building vapoursynth and the plugins worked fine. Would be nice to get some feedback if that fixes the issue.

Cu Selur

alain54 commented 4 years ago

Hi,

Sorry, Selur, my scripts have not been tested in 18.04, only in 20.04!

From your new commit, with /usr/local empty and ~/opt/vapoursynth removed: 18.04.5 LTS: compilation 100% OK, with all plugins; 20.04 LTS: compilation 100% OK, with all plugins.

Well done!

Alain

Selur commented 4 years ago

No, problem. Happy to get help with this at all. :) So: Thank you!

Cu Selur