Add Cython and Password Support

[Theelx]

This combines the password and Cython PRs into one.

[dobrosketchkun]

@Theelgirl you forget about README.md:

Requires installation of FFmpeg and libmagic first, then install using pip3

But one needs a ffmpeg executable for this to work, anyway, so you should explicitly say it in there, may be?

[dobrosketchkun]

So, I tried patch-1 on Win8 machine.

It seems it compiled ok:

C:\Users\user\Documents\dobrosketchkun\python\fvid\mine\test\fvid-patch-1>py setup.py install

Compiling fvid/fvid_cython.pyx because it changed.
[1/1] Cythonizing fvid/fvid_cython.pyx
running install
running build
running build_ext
building 'fvid_cython' extension
creating build
creating build\temp.win32-3.7
creating build\temp.win32-3.7\Release
creating build\temp.win32-3.7\Release\fvid
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\cl.exe /c /nologo /Ox
 /W3 /GL /DNDEBUG /MD -I./fvid -Ifvid/ -IC:\Users\user\AppData\Local\Progr
ams\Python\Python37-32\include -IC:\Users\user\AppData\Local\Programs\Pyth
on\Python37-32\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC
\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" "
-IC:\Program Files (x86)\Windows Kits\8.1\include\shared" "-IC:\Program Files (x
86)\Windows Kits\8.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\incl
ude\winrt" /EHsc /Tpfvid/fvid_cython.cpp /Fobuild\temp.win32-3.7\Release\fvid/fv
id_cython.obj
fvid_cython.cpp
creating C:\Users\user\Documents\dobrosketchkun\python\fvid\mine\test\fvid-patch-
1\build\lib.win32-3.7
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\link.exe /nologo /INC
REMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\Be
lozersky\AppData\Local\Programs\Python\Python37-32\libs /LIBPATH:C:\Users\Beloze
rsky\AppData\Local\Programs\Python\Python37-32\PCbuild\win32 "/LIBPATH:C:\Progra
m Files (x86)\Microsoft Visual Studio 14.0\VC\LIB" "/LIBPATH:C:\Program Files (x
86)\Windows Kits\10\lib\10.0.10240.0\ucrt\x86" "/LIBPATH:C:\Program Files (x86)\
Windows Kits\8.1\lib\winv6.3\um\x86" /EXPORT:PyInit_fvid_cython build\temp.win32
-3.7\Release\fvid/fvid_cython.obj /OUT:build\lib.win32-3.7\fvid_cython.cp37-win3
2.pyd /IMPLIB:build\temp.win32-3.7\Release\fvid\fvid_cython.cp37-win32.lib
   Creating library build\temp.win32-3.7\Release\fvid\fvid_cython.cp37-win32.lib
 and object build\temp.win32-3.7\Release\fvid\fvid_cython.cp37-win32.exp
Generating code
Finished generating code
running install_lib
copying build\lib.win32-3.7\fvid_cython.cp37-win32.pyd -> C:\Users\user\Ap
pData\Local\Programs\Python\Python37-32\Lib\site-packages
running install_egg_info
Writing C:\Users\user\AppData\Local\Programs\Python\Python37-32\Lib\site-p
ackages\UNKNOWN-0.0.0-py3.7.egg-info
C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\distutils\dist
.py:274: UserWarning: Unknown distribution option: 'long_description_content_typ
e'
  warnings.warn(msg)
C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\distutils\dist
.py:274: UserWarning: Unknown distribution option: 'install_requires'
  warnings.warn(msg)
C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\distutils\dist
.py:274: UserWarning: Unknown distribution option: 'python_requires'
  warnings.warn(msg)
C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\distutils\dist
.py:274: UserWarning: Unknown distribution option: 'entry_points'
  warnings.warn(msg)
running install
running build
running build_py
creating build\lib
creating build\lib\fvid
copying fvid\cythonizer.py -> build\lib\fvid
copying fvid\fvid.py -> build\lib\fvid
copying fvid\__init__.py -> build\lib\fvid
copying fvid\__main__.py -> build\lib\fvid
running install_lib
creating C:\Users\user\AppData\Local\Programs\Python\Python37-32\Lib\site-
packages\fvid
copying build\lib\fvid\cythonizer.py -> C:\Users\user\AppData\Local\Progra
ms\Python\Python37-32\Lib\site-packages\fvid
copying build\lib\fvid\fvid.py -> C:\Users\user\AppData\Local\Programs\Pyt
hon\Python37-32\Lib\site-packages\fvid
copying build\lib\fvid\__init__.py -> C:\Users\user\AppData\Local\Programs
\Python\Python37-32\Lib\site-packages\fvid
copying build\lib\fvid\__main__.py -> C:\Users\user\AppData\Local\Programs
\Python\Python37-32\Lib\site-packages\fvid
byte-compiling C:\Users\user\AppData\Local\Programs\Python\Python37-32\Lib
\site-packages\fvid\cythonizer.py to cythonizer.cpython-37.pyc
byte-compiling C:\Users\user\AppData\Local\Programs\Python\Python37-32\Lib
\site-packages\fvid\fvid.py to fvid.cpython-37.pyc
byte-compiling C:\Users\user\AppData\Local\Programs\Python\Python37-32\Lib
\site-packages\fvid\__init__.py to __init__.cpython-37.pyc
byte-compiling C:\Users\user\AppData\Local\Programs\Python\Python37-32\Lib
\site-packages\fvid\__main__.py to __main__.cpython-37.pyc
running install_egg_info
Writing C:\Users\user\AppData\Local\Programs\Python\Python37-32\Lib\site-p
ackages\fvid-0.0.2-py3.7.egg-info

But when I tried to use it:

C:\Users\user\Documents\dobrosketchkun\python\fvid\mine\test>py -m fvid -i Lenna.png -e -o Lenna.mp4
Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\runpy.
py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\runpy.
py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\site-p
ackages\fvid\__main__.py", line 2, in <module>
    from fvid import main
ImportError: cannot import name 'main' from 'fvid' (C:\Users\user\AppData\
Local\Programs\Python\Python37-32\lib\site-packages\fvid\__init__.py)

[dobrosketchkun]

Also, it seems I cannot uninstall it:

C:\Users\Belozersky\Documents\Khromov\python\fvid\mine\test>py -m pip uninstall fvid

Found existing installation: fvid 0.0.2
ERROR: Cannot uninstall 'fvid'. It is a distutils installed project and thus we
cannot accurately determine which files belong to it which would lead to only a
partial uninstall.

[Theelx]

Ok, I'll try and find a way to install cython without distutils. However, the "ImportError: cannot import name 'main' from 'fvid'" shouldn't be coming from cython, cython doesn't change anything there. Also, try compiling the cython directly with py cythonizer.py build_ext --inplace and tell me how that works on windows.

[dobrosketchkun]

Here is a log:

C:\Users\user\Documents\dobrosketchkun\python\fvid\mine\test\fvid-patch-1\fvid>py
 cythonizer.py build_ext --inplace
running build_ext
building 'fvid_cython' extension
creating build
creating build\temp.win32-3.7
creating build\temp.win32-3.7\Release
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\cl.exe /c /nologo /Ox
 /W3 /GL /DNDEBUG /MD -IC:\Users\user\AppData\Local\Programs\Python\Python
37-32\include -IC:\Users\user\AppData\Local\Programs\Python\Python37-32\in
clude "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\P
rogram Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" "-IC:\Program File
s (x86)\Windows Kits\8.1\include\shared" "-IC:\Program Files (x86)\Windows Kits\
8.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\winrt" /EHsc
/Tpfvid_cython.cpp /Fobuild\temp.win32-3.7\Release\fvid_cython.obj
fvid_cython.cpp
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\link.exe /nologo /INC
REMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\Be
lozersky\AppData\Local\Programs\Python\Python37-32\libs /LIBPATH:C:\Users\Beloze
rsky\AppData\Local\Programs\Python\Python37-32\PCbuild\win32 "/LIBPATH:C:\Progra
m Files (x86)\Microsoft Visual Studio 14.0\VC\LIB" "/LIBPATH:C:\Program Files (x
86)\Windows Kits\10\lib\10.0.10240.0\ucrt\x86" "/LIBPATH:C:\Program Files (x86)\
Windows Kits\8.1\lib\winv6.3\um\x86" /EXPORT:PyInit_fvid_cython build\temp.win32
-3.7\Release\fvid_cython.obj /OUT:C:\Users\user\Documents\dobrosketchkun\python\f
vid\mine\test\fvid-patch-1\fvid\fvid_cython.cp37-win32.pyd /IMPLIB:build\temp.wi
n32-3.7\Release\fvid_cython.cp37-win32.lib
   Creating library build\temp.win32-3.7\Release\fvid_cython.cp37-win32.lib and
object build\temp.win32-3.7\Release\fvid_cython.cp37-win32.exp
Generating code
Finished generating code

Archive with built files:

build.zip

[Theelx]

That log looks like it worked. Try building this new version from scratch now and let me know how it works.

Edit: Windows is the root of all evil.

[dobrosketchkun]

Edit: Windows is the root of all evil.

lol

---

You forget about a comma before zip_safe=False

The log:

C:\Users\user\Documents\dobrosketchkun\python\fvid\mine\test\fvid-patch-1>py setup.py install

WARNING: The wheel package is not available.
C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\site-packages\
setuptools\dist.py:679: UserWarning: Module Cython was already imported from C:\
Users\user\AppData\Local\Programs\Python\Python37-32\lib\site-packages\Cyt
hon\__init__.py, but c:\users\belozersky\documents\khromov\python\fvid\mine\test
\fvid-patch-1\.eggs\cython-3.0a6-py3.7-win32.egg is being added to sys.path
  pkg_resources.working_set.add(dist, replace=True)
C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\site-packages\
setuptools\dist.py:679: UserWarning: Module cython was already imported from C:\
Users\user\AppData\Local\Programs\Python\Python37-32\lib\site-packages\cyt
hon.py, but c:\users\belozersky\documents\khromov\python\fvid\mine\test\fvid-pat
ch-1\.eggs\cython-3.0a6-py3.7-win32.egg is being added to sys.path
  pkg_resources.working_set.add(dist, replace=True)
running install
running bdist_egg
running egg_info
creating fvid.egg-info
writing fvid.egg-info\PKG-INFO
writing dependency_links to fvid.egg-info\dependency_links.txt
writing entry points to fvid.egg-info\entry_points.txt
writing requirements to fvid.egg-info\requires.txt
writing top-level names to fvid.egg-info\top_level.txt
writing manifest file 'fvid.egg-info\SOURCES.txt'
reading manifest file 'fvid.egg-info\SOURCES.txt'
writing manifest file 'fvid.egg-info\SOURCES.txt'
installing library code to build\bdist.win32\egg
running install_lib
running build_py
creating build
creating build\lib.win32-3.7
creating build\lib.win32-3.7\fvid
copying fvid\cythonizer.py -> build\lib.win32-3.7\fvid
copying fvid\fvid.py -> build\lib.win32-3.7\fvid
copying fvid\__init__.py -> build\lib.win32-3.7\fvid
copying fvid\__main__.py -> build\lib.win32-3.7\fvid
copying fvid\fvid_cython.cpp -> build\lib.win32-3.7\fvid
running build_ext
building 'fvid.fvid_cython' extension
creating build\temp.win32-3.7
creating build\temp.win32-3.7\Release
creating build\temp.win32-3.7\Release\fvid
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\cl.exe /c /nologo /Ox
 /W3 /GL /DNDEBUG /MD -I./fvid -Ifvid/ -IC:\Users\user\AppData\Local\Progr
ams\Python\Python37-32\include -IC:\Users\user\AppData\Local\Programs\Pyth
on\Python37-32\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC
\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" "
-IC:\Program Files (x86)\Windows Kits\8.1\include\shared" "-IC:\Program Files (x
86)\Windows Kits\8.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\incl
ude\winrt" /EHsc /Tpfvid/fvid_cython.cpp /Fobuild\temp.win32-3.7\Release\fvid/fv
id_cython.obj
fvid_cython.cpp
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\link.exe /nologo /INC
REMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\Be
lozersky\AppData\Local\Programs\Python\Python37-32\libs /LIBPATH:C:\Users\Beloze
rsky\AppData\Local\Programs\Python\Python37-32\PCbuild\win32 "/LIBPATH:C:\Progra
m Files (x86)\Microsoft Visual Studio 14.0\VC\LIB" "/LIBPATH:C:\Program Files (x
86)\Windows Kits\10\lib\10.0.10240.0\ucrt\x86" "/LIBPATH:C:\Program Files (x86)\
Windows Kits\8.1\lib\winv6.3\um\x86" /EXPORT:PyInit_fvid_cython build\temp.win32
-3.7\Release\fvid/fvid_cython.obj /OUT:build\lib.win32-3.7\fvid\fvid_cython.cp37
-win32.pyd /IMPLIB:build\temp.win32-3.7\Release\fvid\fvid_cython.cp37-win32.lib
   Creating library build\temp.win32-3.7\Release\fvid\fvid_cython.cp37-win32.lib
 and object build\temp.win32-3.7\Release\fvid\fvid_cython.cp37-win32.exp
Generating code
Finished generating code
creating build\bdist.win32
creating build\bdist.win32\egg
creating build\bdist.win32\egg\fvid
copying build\lib.win32-3.7\fvid\cythonizer.py -> build\bdist.win32\egg\fvid
copying build\lib.win32-3.7\fvid\fvid.py -> build\bdist.win32\egg\fvid
copying build\lib.win32-3.7\fvid\fvid_cython.cp37-win32.pyd -> build\bdist.win32
\egg\fvid
copying build\lib.win32-3.7\fvid\fvid_cython.cpp -> build\bdist.win32\egg\fvid
copying build\lib.win32-3.7\fvid\__init__.py -> build\bdist.win32\egg\fvid
copying build\lib.win32-3.7\fvid\__main__.py -> build\bdist.win32\egg\fvid
byte-compiling build\bdist.win32\egg\fvid\cythonizer.py to cythonizer.cpython-37
.pyc
byte-compiling build\bdist.win32\egg\fvid\fvid.py to fvid.cpython-37.pyc
byte-compiling build\bdist.win32\egg\fvid\__init__.py to __init__.cpython-37.pyc

byte-compiling build\bdist.win32\egg\fvid\__main__.py to __main__.cpython-37.pyc

creating stub loader for fvid\fvid_cython.cp37-win32.pyd
byte-compiling build\bdist.win32\egg\fvid\fvid_cython.py to fvid_cython.cpython-
37.pyc
creating build\bdist.win32\egg\EGG-INFO
copying fvid.egg-info\PKG-INFO -> build\bdist.win32\egg\EGG-INFO
copying fvid.egg-info\SOURCES.txt -> build\bdist.win32\egg\EGG-INFO
copying fvid.egg-info\dependency_links.txt -> build\bdist.win32\egg\EGG-INFO
copying fvid.egg-info\entry_points.txt -> build\bdist.win32\egg\EGG-INFO
copying fvid.egg-info\not-zip-safe -> build\bdist.win32\egg\EGG-INFO
copying fvid.egg-info\requires.txt -> build\bdist.win32\egg\EGG-INFO
copying fvid.egg-info\top_level.txt -> build\bdist.win32\egg\EGG-INFO
writing build\bdist.win32\egg\EGG-INFO\native_libs.txt
creating dist
creating 'dist\fvid-0.0.2-py3.7-win32.egg' and adding 'build\bdist.win32\egg' to
 it
removing 'build\bdist.win32\egg' (and everything under it)
Processing fvid-0.0.2-py3.7-win32.egg
creating c:\users\belozersky\appdata\local\programs\python\python37-32\lib\site-
packages\fvid-0.0.2-py3.7-win32.egg
Extracting fvid-0.0.2-py3.7-win32.egg to c:\users\belozersky\appdata\local\progr
ams\python\python37-32\lib\site-packages
Adding fvid 0.0.2 to easy-install.pth file
Installing fvid-script.py script to C:\Users\user\AppData\Local\Programs\P
ython\Python37-32\Scripts
Installing fvid.exe script to C:\Users\user\AppData\Local\Programs\Python\
Python37-32\Scripts
Installing fvid.exe.manifest script to C:\Users\user\AppData\Local\Program
s\Python\Python37-32\Scripts

Installed c:\users\belozersky\appdata\local\programs\python\python37-32\lib\site
-packages\fvid-0.0.2-py3.7-win32.egg
Processing dependencies for fvid==0.0.2
Searching for tqdm==4.28.1
Best match: tqdm 4.28.1
Adding tqdm 4.28.1 to easy-install.pth file
Installing tqdm-script.py script to C:\Users\user\AppData\Local\Programs\P
ython\Python37-32\Scripts
Installing tqdm.exe script to C:\Users\user\AppData\Local\Programs\Python\
Python37-32\Scripts
Installing tqdm.exe.manifest script to C:\Users\user\AppData\Local\Program
s\Python\Python37-32\Scripts

Using c:\users\belozersky\appdata\local\programs\python\python37-32\lib\site-pac
kages
Searching for Pillow==5.3.0
Best match: Pillow 5.3.0
Adding Pillow 5.3.0 to easy-install.pth file

Using c:\users\belozersky\appdata\local\programs\python\python37-32\lib\site-pac
kages
Searching for bitstring==3.1.7
Best match: bitstring 3.1.7
Adding bitstring 3.1.7 to easy-install.pth file

Using c:\users\belozersky\appdata\local\programs\python\python37-32\lib\site-pac
kages
Finished processing dependencies for fvid==0.0.2

I don't work either:

C:\Users\user\Documents\dobrosketchkun\python\fvid\mine\test>py -m fvid -i Lenna.
png -e -o Lenna.mp4
Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\runpy.
py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\runpy.
py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\site-p
ackages\fvid\__main__.py", line 2, in <module>
    from fvid import main
ImportError: cannot import name 'main' from 'fvid' (C:\Users\user\AppData\
Local\Programs\Python\Python37-32\lib\site-packages\fvid\__init__.py)

New build files in the directory:

fvid-patch-1.zip

[Theelx]

Try changing main.py to

import sys
from fvid.fvid import main

if __name__ == '__main__':
    sys.exit(main())

Edit: No, that's dumb don't do that.

[Theelx]

@dobrosketchkun Try now? I figure it could have been a conflict with the package name and file name on Windows, as it works fine on Ubuntu.

[dobrosketchkun]

Edit: No, that's dumb don't do that.

Well, actually, I just went to another machine with Win10 and tried as it is and it's doesn't work, so I change main.py in C:\Users\user\AppData\Local\Programs\Python\Python37-32\Lib\site-packages\fvid-0.0.2-py3.7-win32.egg\fvid folder, and it works

[dobrosketchkun]

Should I try your new commits?

[Theelx]

Try the new commits, because the one I told you to do doesn't work on my Ubuntu machine, this might work on both systems.

[dobrosketchkun]

New commits:

• first of all, again, a comma before zip_safe And:

>py -m fvid -i 1.jpg -e -o 1.mp4
Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\site-packages\fvid-0.0.2-py3.7-win32.egg\fvid\__main__.py", line 2, in <module>
    from fvid_python import main
ModuleNotFoundError: No module named 'fvid_python'

doesn't work on my Ubuntu machine

maybe we just should check with the system with os module and import what we need in each case?

[Theelx]

Did you rename fvid.py to fvid_python.py? I'll try and add detection of the os in main.py and adjust accordingly if this doesn't work.

[dobrosketchkun]

Did you rename fvid.py to fvid_python.py?

Ehm, no I did not. I installed it as it is.

Anyway, the result is the same:

ModuleNotFoundError: No module named 'fvid_python'

[Theelx]

Ok, I'll update main.py with detection of os now.

[Theelx]

@dobrosketchkun Try now on windows, it may need to be tested on MacOS also (@AlfredoSequeida)

[dobrosketchkun]

Nice, it works now!

[Theelx]

Great! @AlfredoSequeida If this works on MacOS, I'd say we're good to go! Also, when merging, make sure to adjust the version number to 1.0.0 and release on PyPi, as the current release is 0.0.2 from 5 days ago, and this is a huge improvement in speed, plus it's no longer possible to decrypt old files since they weren't encrypted with the default password, so it's backwards-incompatible.

[zavok]

On Arch Linux, I got similar issue with ImportError: cannot import name 'main' from 'fvid' And solved it in similar way, by modifying from fvid import main to from fvid.fvid import main

[Theelx]

Hm ok, so I guess more granularity is needed on determining the OS.

[Theelx]

@zavok what do you get when running

import platform
import sys

def linux_distribution():
  try:
    return platform.linux_distribution()
  except:
    return "N/A"

print("""Python version: %s
dist: %s
linux_distribution: %s
system: %s
machine: %s
platform: %s
uname: %s
version: %s
mac_ver: %s
""" % (
sys.version.split('\n'),
str(platform.dist()),
linux_distribution(),
platform.system(),
platform.machine(),
platform.platform(),
platform.uname(),
platform.version(),
platform.mac_ver(),
))

on arch linux?

[zavok]

@Theelgirl

Python version: ['3.8.6 (default, Sep 30 2020, 20:59:51) ', '[GCC 10.2.0]']
linux_distribution: N/A
system: Linux
machine: x86_64
platform: Linux-5.8.14-artix1-1-x86_64-with-glibc2.2.5
uname: uname_result(system='Linux', node='trashstation', release='5.8.14-artix1-1', version='#1 SMP PREEMPT Thu, 08 Oct 2020 09:51:02 +0000', machine='x86_64', processor='')
version: #1 SMP PREEMPT Thu, 08 Oct 2020 09:51:02 +0000
mac_ver: ('', ('', '', ''), '')

I removed platform.dist(), because AttributeError: module 'platform' has no attribute 'dist' Also, see https://bugs.python.org/issue1322

someone at stackoverflow recommends using

import distro 
print(distro.linux_distribution())

which on my machine returns ('Artix Linux', 'rolling', 'n/a') (Artix Linux is a flavor of Arch Linux that I'm actually running. Didn't feel the need to specify it in the past comments, sorry)

Anyway, are you planning on putting "if ... else" pair for every possible operating system/distro? This is not a pretty solution...

[Theelx]

@zavok Not an if else pair for everything, as there are only two options. I was going to put the return values of these functions into a tuple and check that. Something like:

if distro.linux_distribution()[0].lower() in ("artix linux", "other linux flavors that require this"):
    from fvid.fvid import main
else:
    from fvid import main

[Theelx]

@dobrosketchkun @zavok If either of you know of a way to easily test if this works on multiple different OSes, please either do it or let me know how, it'd be best if the importing is foolproof, otherwise we all know it won't work properly.

[dobrosketchkun]

Well, as we can see, there only two choices (fvid.fivd and fvid), so you can add these options into setup.py for a person to choose. If the first one doesn't work, reinstall with the second one.

PS there is no NumPy in setup.py btw and just an import of NumPy in fvid.py, so pls delete it. PPS and non of crypto modules

[Theelx]

@dobrosketchkun I don't believe I can make a prompt in setup.py for users that triggers on install, and even if I could, the user can't know which to pick without trying them first. I'll remove the numpy imports and add the crypto modules to setup.py

[Theelx]

Also, what is the name of the module imported here (name on PyPi), and what's the minimum required version for it to work? https://github.com/Theelgirl/fvid/blob/cf93951b2c68095f155b97e0b343aa62825020c1/fvid/fvid.py#L23

[dobrosketchkun]

I don't believe I can make a prompt in setup.py

If that so, we can add another arg to fvid itself, I guess. Can __main__.py catch on of the args, "-v, --version" for example, and according to it import fvid specifically?

[Theelx]

@dobrosketchkun I don't think so, because the argument parsing happens after importing, and the problem is with importing. I can try wrapping the imports as a try/except though.

[Theelx]

@dobrosketchkun I just tried running this on 3.8.6, and I installed pycrypto, but I'm getting this error:

  File "fvid.py", line 309, in main
    bits = get_bits_from_file(args.input, key)
  File "fvid.py", line 63, in get_bits_from_file
    cipher = AES.new(key, AES.MODE_EAX, nonce=SALT)
AttributeError: module 'Crypto.Cipher.AES' has no attribute 'MODE_EAX'

I dont know why it suddenly stopped working, but I checked the github and it hasn't been updated since 2014. Shouldn't we be using a newer module that's actively updated for security and maintainability and such?

Edit: I realized pycryptodome will work drop-in instead, so I'll add that to setup.py

[dobrosketchkun]

@AlfredoSequeida any news?

[AlfredoSequeida]

@dobrosketchkun sorry, I'm really busy lately with school work.

[Theelx]

@AlfredoSequeida Do you think you can give one of us pushing and releasing permissions?

[dobrosketchkun]

For good for all of us, it better be @Theelgirl because I know very little about pulls, pushes, and branches.

[Theelx]

Ah, I'm really bad with branches but I can manage pulls and pushes if needed. However, this would also need to be released to PyPi on important updates, so if he wants to hedge any minute risk of one of us going rogue, he could let me manage PRs/pushes, and you manage PyPi releases.

[AlfredoSequeida]

@Theelgirl Yes! I've been pondering this for a while because I see you guys are so active. And I was actually going to ask you about it. Let me look into how to get that done and I would be glad to see what you do with it.

[Theelx]

Here's a useful link for PyPi: https://pypi.org/help/#collaborator-roles For github, to add a collaborator, you can click Settings (for the repo), then Manage Access, then "Invite Collaborator": https://github.com/AlfredoSequeida/fvid/settings/access

[AlfredoSequeida]

@Theelgirl I sent you a request to add you to the project. If you wish to accept it I would be happy to have you manage it until I have more time to work on it. And I can definitely try to fit some time in when we need to update the PyPi package. Let me know what you think. Also thank you for taking interest in the project, I'm glad you like it enough to contribute the amount you have so far.

[Theelx]

Thanks for the invitation! I actually found this project off a Reddit link from the Python Discord, and I just happened to be using Pillow and Cython extensively, so I figured I'd drop in and help make a few optimizations, but I didn't initially expect to end up contributing as much as I have. However, even though I didn't initially plan for it, I'd love to help continue development and optimize this project, so I accepted the invitation!

[Theelx]

This most recent Cython approach is a whopping 70x faster on a large 26MB jpg of mine than the pure python approach! It takes about 32-33 seconds per frame (each frame is about 3.2 MB, and there are 105 frames) with Python, and about 0.5 seconds per frame with Cython! (running Python 3.8.6 on a 3.8GHz Ryzen processor on Ubuntu 20.04)

I remember saying a while back that I expected something like a 5x speedup at most, but 70x (even though only with large files) is beyond my wildest dreams!

Edit: I did a dumb and measured the pure python performance with my line profiler on, and Cython without it on, Cython actually only gives a 8-9x boost compared to the current python version, and a 3-4x boost compared to a speedier python fvid that I'm about to push :(

[Theelx]

@dobrosketchkun Can you test my most recent changes to fvid on Windows and/or OS X? These work on Ubuntu 20.04, and if they work on both of those platforms, I can merge it.

[dobrosketchkun]

Yeah, it's working quite well and fast.

But.

I tried to check, for once, -f option and used 60. File encoded well, but upon decoding, the error appeared:

Bits are in place
100%|██████████████████████████████████████████████████████████████████████████████████| 52/52 [02:10<00:00,  2.51s/it]
Unzipping...
Traceback (most recent call last):
  File "C:\Users\dobrosketchkun\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\dobrosketchkun\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "D:\recovery3\python\random\fvid_env\lib\site-packages\fvid\__main__.py", line 28, in <module>
    sys.exit(main())
  File "D:\recovery3\python\random\fvid_env\lib\site-packages\fvid\fvid.py", line 288, in main
    save_bits_to_file(file_path, bits, key)
  File "D:\recovery3\python\random\fvid_env\lib\site-packages\fvid\fvid.py", line 162, in save_bits_to_file
    bitstring = fo.read()
  File "C:\Users\dobrosketchkun\AppData\Local\Programs\Python\Python39\lib\gzip.py", line 300, in read
    return self._buffer.read(size)
  File "C:\Users\dobrosketchkun\AppData\Local\Programs\Python\Python39\lib\gzip.py", line 495, in read
    uncompress = self._decompressor.decompress(buf, size)
zlib.error: Error -3 while decompressing data: invalid code lengths set

Then I tried 30, and there was another error:

Traceback (most recent call last):
  File "C:\Users\dobrosketchkun\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\dobrosketchkun\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "D:\recovery3\python\random\fvid_env\lib\site-packages\fvid\__main__.py", line 28, in <module>
    sys.exit(main())
  File "D:\recovery3\python\random\fvid_env\lib\site-packages\fvid\fvid.py", line 288, in main
    save_bits_to_file(file_path, bits, key)
  File "D:\recovery3\python\random\fvid_env\lib\site-packages\fvid\fvid.py", line 162, in save_bits_to_file
    bitstring = fo.read()
  File "C:\Users\dobrosketchkun\AppData\Local\Programs\Python\Python39\lib\gzip.py", line 300, in read
    return self._buffer.read(size)
  File "C:\Users\dobrosketchkun\AppData\Local\Programs\Python\Python39\lib\gzip.py", line 478, in read
    self._read_eof()
  File "C:\Users\dobrosketchkun\AppData\Local\Programs\Python\Python39\lib\gzip.py", line 524, in _read_eof
    raise BadGzipFile("CRC check failed %s != %s" % (hex(crc32),
gzip.BadGzipFile: CRC check failed 0x1447e231 != 0xaa41731d

The same was for -f 24

This is the same for Win and Arch.

[dobrosketchkun]

I tried on one file, and it is working from default up to -f 7 in my case.

Maybe the deal is about codec and its quality settings?

[dobrosketchkun]

Btw without the -tune zerolatency flag, it's working up to 30

[Theelx]

Hm ok, so I'll remove the tune flag and squash merge it. Thanks for testing!