Installation

[NauticalMile64]

Hi,

I'm having some installation issues using the pip install method. I think it's a problem with the source, which is why I'm posting here.

3 quick notes:

• I'm running windows 7 64-bit, and Python 3.3.
• I've confirmed that OpenCL and Boost are installed correctly (I have used them in other projects). All of my GPU drivers are up to date.
• I also have pyopencl installed from here. This could be part of the problem

The pip log is long, and I'm pretty sure I can see the relevant error so I'll put it here:

Traceback (most recent call last):
      File ".\aksetup_helper.py", line 572, in set_up_shipped_boost_if_requested
        symlink("boost", bpl_project_boost_inc)
    FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'boost'

I noticed that line 569 of aksetup_helper.py is:

if not isdir(bpl_project_boost_inc):

so the code was testing if there was already a directory at that path. I'm wondering if I have a file named boost at that path, and according to the docs os.path.isdir only checks if there is a directory at that path, and says nothing about a file. That's just a theory anyway.

Below is the full pip.txt log. Thanks in advance!

------------------------------------------------------------
C:\Python\Python33\Scripts\pip run on 05/06/14 17:03:57
Downloading/unpacking pyviennacl
  Getting page https://pypi.python.org/simple/pyviennacl/
  URLs to search for versions for pyviennacl:
  * https://pypi.python.org/simple/pyviennacl/
  Analyzing links from page https://pypi.python.org/simple/pyviennacl/
    Skipping https://pypi.python.org/packages/2.7/p/pyviennacl/pyviennacl-1.0.1-cp27-none-win32.whl#md5=69f87071116b1620f4f815f317258e4b (from https://pypi.python.org/simple/pyviennacl/) because it is not compatible with this Python
    Skipping https://pypi.python.org/packages/2.7/p/pyviennacl/pyviennacl-1.0.1-cp27-none-win_amd64.whl#md5=7f3a676cf3dca7bbb286b83170011931 (from https://pypi.python.org/simple/pyviennacl/) because it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/2.7/p/pyviennacl/pyviennacl-1.0.1.win-amd64-py2.7.exe#md5=924d29d5c45eb6f8975002d818b0bd4f (from https://pypi.python.org/simple/pyviennacl/); unknown archive format: .exe
    Skipping link https://pypi.python.org/packages/2.7/p/pyviennacl/pyviennacl-1.0.1.win32-py2.7.exe#md5=e0eeb938b3335a76d3e9c7d566886fc8 (from https://pypi.python.org/simple/pyviennacl/); unknown archive format: .exe
    Skipping https://pypi.python.org/packages/3.3/p/pyviennacl/pyviennacl-1.0.1-cp33-none-win32.whl#md5=7a7848332eefe4785e1b7bf5141d3d74 (from https://pypi.python.org/simple/pyviennacl/) because it is not compatible with this Python
    Found link https://pypi.python.org/packages/3.3/p/pyviennacl/pyviennacl-1.0.1-cp33-none-win_amd64.whl#md5=b88dd2864f88834a5078ce08f14cf7f6 (from https://pypi.python.org/simple/pyviennacl/), version: 1.0.1
    Skipping link https://pypi.python.org/packages/3.3/p/pyviennacl/pyviennacl-1.0.1.win-amd64-py3.3.exe#md5=fa931142aac9a0f262d04e15db73fe5e (from https://pypi.python.org/simple/pyviennacl/); unknown archive format: .exe
    Skipping link https://pypi.python.org/packages/3.3/p/pyviennacl/pyviennacl-1.0.1.win32-py3.3.exe#md5=0e7f0f09c46acffb923f6e89677b4010 (from https://pypi.python.org/simple/pyviennacl/); unknown archive format: .exe
    Found link https://pypi.python.org/packages/source/p/pyviennacl/pyviennacl-1.0.1.tar.gz#md5=789292d481da9c58445743f213c8f97c (from https://pypi.python.org/simple/pyviennacl/), version: 1.0.1
    Found link https://pypi.python.org/packages/source/p/pyviennacl/pyviennacl-1.0.1.zip#md5=4b67ca9d49c1d8774b3a8152fe1a1934 (from https://pypi.python.org/simple/pyviennacl/), version: 1.0.1
    Found link https://pypi.python.org/packages/source/p/pyviennacl/pyviennacl-1.0.2.tar.gz#md5=fac351269845305c4a935acbb14bed52 (from https://pypi.python.org/simple/pyviennacl/), version: 1.0.2
    Found link https://pypi.python.org/packages/source/p/pyviennacl/pyviennacl-1.0.2.zip#md5=f3cecd1c9006183ec064fd27020f37f9 (from https://pypi.python.org/simple/pyviennacl/), version: 1.0.2
  Using version 1.0.2 (newest of versions: 1.0.2, 1.0.2, 1.0.1, 1.0.1, 1.0.1)
  Downloading from URL https://pypi.python.org/packages/source/p/pyviennacl/pyviennacl-1.0.2.tar.gz#md5=fac351269845305c4a935acbb14bed52 (from https://pypi.python.org/simple/pyviennacl/)
  Running setup.py (path:c:\users\...\appdata\local\temp\pip_build_User\pyviennacl\setup.py) egg_info for package pyviennacl
    running egg_info
    creating pip-egg-info\pyviennacl.egg-info
    writing top-level names to pip-egg-info\pyviennacl.egg-info\top_level.txt
    writing dependency_links to pip-egg-info\pyviennacl.egg-info\dependency_links.txt
    writing pip-egg-info\pyviennacl.egg-info\PKG-INFO
    writing manifest file 'pip-egg-info\pyviennacl.egg-info\SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found

    reading manifest file 'pip-egg-info\pyviennacl.egg-info\SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    no previously-included directories found matching 'doc\_build'
    warning: no files found matching '*.inl' under directory 'external\boost-python-ublas-subset\boost_subset\boost'
    warning: no files found matching '*.txt' under directory 'external\boost-python-ublas-subset\boost_subset\boost'
    warning: no files found matching '*.h' under directory 'external\boost-python-ublas-subset\boost_subset\libs'
    warning: no files found matching '*.ipp' under directory 'external\boost-python-ublas-subset\boost_subset\libs'
    warning: no files found matching '*.pl' under directory 'external\boost-python-ublas-subset\boost_subset\libs'
    warning: no files found matching '*.h' under directory 'external\boost_numpy\boost'
    warning: no files found matching '*.inl' under directory 'external\boost_numpy\boost'
    warning: no files found matching '*.ipp' under directory 'external\boost_numpy\boost'
    warning: no files found matching '*.pl' under directory 'external\boost_numpy\boost'
    warning: no files found matching '*.txt' under directory 'external\boost_numpy\boost'
    warning: no files found matching '*.h' under directory 'external\boost_numpy\libs'
    warning: no files found matching '*.hpp' under directory 'external\boost_numpy\libs'
    warning: no files found matching '*.inl' under directory 'external\boost_numpy\libs'
    warning: no files found matching '*.ipp' under directory 'external\boost_numpy\libs'
    warning: no files found matching '*.pl' under directory 'external\boost_numpy\libs'
    writing manifest file 'pip-egg-info\pyviennacl.egg-info\SOURCES.txt'
  Source in c:\users\...\appdata\local\temp\pip_build_User\pyviennacl has version 1.0.2, which satisfies requirement pyviennacl
Installing collected packages: pyviennacl
  Running setup.py install for pyviennacl
    Running command C:\Python\Python33\python.exe -c "import setuptools, tokenize;__file__='c:\\users\\...\\appdata\\local\\temp\\pip_build_User\\pyviennacl\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record c:\users\...\appdata\local\temp\pip-av4arl-record\install-record.txt --single-version-externally-managed --compile
    Copying files, hang on... (do not interrupt)
    Traceback (most recent call last):
      File ".\aksetup_helper.py", line 572, in set_up_shipped_boost_if_requested
        symlink("boost", bpl_project_boost_inc)
    FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'boost'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "c:\users\...\appdata\local\temp\pip_build_User\pyviennacl\setup.py", line 224, in <module>
        main()
      File "c:\users\...\appdata\local\temp\pip_build_User\pyviennacl\setup.py", line 108, in main
        boost_chrono="header_only")
      File ".\aksetup_helper.py", line 576, in set_up_shipped_boost_if_requested
        copytree(main_boost_inc, bpl_project_boost_inc)
      File "C:\Python\Python33\lib\shutil.py", line 301, in copytree
        os.makedirs(dst)
      File "C:\Python\Python33\lib\os.py", line 269, in makedirs
        mkdir(name, mode)
    FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'external/boost-python-ublas-subset/boost_subset/pyviennaclboost'
    Complete output from command C:\Python\Python33\python.exe -c "import setuptools, tokenize;__file__='c:\\users\\...\\appdata\\local\\temp\\pip_build_User\\pyviennacl\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record c:\users\...\appdata\local\temp\pip-av4arl-record\install-record.txt --single-version-externally-managed --compile:
    Copying files, hang on... (do not interrupt)

Traceback (most recent call last):

  File ".\aksetup_helper.py", line 572, in set_up_shipped_boost_if_requested

    symlink("boost", bpl_project_boost_inc)

FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'boost'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "<string>", line 1, in <module>

  File "c:\users\...\appdata\local\temp\pip_build_User\pyviennacl\setup.py", line 224, in <module>

    main()

  File "c:\users\...\appdata\local\temp\pip_build_User\pyviennacl\setup.py", line 108, in main

    boost_chrono="header_only")

  File ".\aksetup_helper.py", line 576, in set_up_shipped_boost_if_requested

    copytree(main_boost_inc, bpl_project_boost_inc)

  File "C:\Python\Python33\lib\shutil.py", line 301, in copytree

    os.makedirs(dst)

  File "C:\Python\Python33\lib\os.py", line 269, in makedirs

    mkdir(name, mode)

FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'external/boost-python-ublas-subset/boost_subset/pyviennaclboost'

----------------------------------------
Cleaning up...
  Removing temporary dir c:\users\...\appdata\local\temp\pip_build_User...
Command C:\Python\Python33\python.exe -c "import setuptools, tokenize;__file__='c:\\users\\...\\appdata\\local\\temp\\pip_build_User\\pyviennacl\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record c:\users\...\appdata\local\temp\pip-av4arl-record\install-record.txt --single-version-externally-managed --compile failed with error code 1 in c:\users\...\appdata\local\temp\pip_build_User\pyviennacl
Exception information:
Traceback (most recent call last):
  File "C:\Python\Python33\lib\site-packages\pip\basecommand.py", line 122, in main
    status = self.run(options, args)
  File "C:\Python\Python33\lib\site-packages\pip\commands\install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "C:\Python\Python33\lib\site-packages\pip\req.py", line 1435, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "C:\Python\Python33\lib\site-packages\pip\req.py", line 706, in install
    cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
  File "C:\Python\Python33\lib\site-packages\pip\util.py", line 697, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command C:\Python\Python33\python.exe -c "import setuptools, tokenize;__file__='c:\\users\\...\\appdata\\local\\temp\\pip_build_User\\pyviennacl\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record c:\users\...\appdata\local\temp\pip-av4arl-record\install-record.txt --single-version-externally-managed --compile failed with error code 1 in c:\users\...\appdata\local\temp\pip_build_User\pyviennacl

[tsmithe]

Hm, that is upsetting! I'm in the process of preparing a new (bug-fix) release 1.0.2, but my Windows virtual hard drive has become corrupted, and so preparing the binaries is taking longer than I had hoped. Normally, pip would install a binary package directly.

Nonetheless, you are probably right that there is also a bug in the source. I will investigate that when my Windows installation is working again.

For now, you can install version 1.0.1 (without requiring a source build) by running

pip install pyviennacl==1.0.1

but note that there is a bug in computing matrix-vector products with non-square matrices, and eigenvalue computations are not entirely working. See issue #11 and issue #13 for more info.

If you would rather build from source using 1.0.2, then you can try removing the following path, if it exists,

external/boost-python-ublas-subset/boost_subset/pyviennaclboost

which is a symlink that probably shouldn't be there on Windows.

[tsmithe]

OK, I've fixed this in the new 1.0.2 tarball uploaded to PyPI. Binaries for Windows should be available in the next couple of days. If you do want to build 1.0.2 from source yourself, please do, though I recommend the 'Custom installation' method rather than relying on pip to get everything right on Windows.

[NauticalMile64]

Thanks for the quick fix. I will clone the repo and try out the custom installation.

[NauticalMile64]

@tsmithe I've run into some more errors. Is it useful to you if I report them (I'd be happy to do this if it helps fix more bugs), or do you just want to wait until you get your Windows virtual drive up and running?

[tsmithe]

Oh, no: please do report them! I of course want it to be easy to install both from source and binary.

[NauticalMile64]

@tsmithe Ok I solved that problem (needed to use the MS SDK 7.1 command prompt) and ran into another one, this time during the compilation of the c++ viennacl (thousands of lines of compiler warnings, usually about type conversions, and notifications of some sort about template parameters). I think I'll try installing the c++ viennacl from source to see if I can isolate the issue, and I'll post it as an issue for the c++ version if that is the case.

[tsmithe]

Ah, I think you've probably just run into an error that occurs with MSVC somewhere in the Lanczos algorithm. I'm discussing it with the main ViennaCL developers at the moment[1]. It's a bit baffling because the same code compiles fine with GCC, and it would be a shame to lose the ability to compute the largest eigenvalues of a dense matrix. Sparse matrices work fine..

Also note that you probably don't strictly need to install ViennaCL from source -- it's a header-only library, and I bundle the most recent release in external/viennacl-dev.

[1] http://thread.gmane.org/gmane.comp.mathematics.viennacl.devel/542

[tsmithe]

(Nonetheless, any investigation on your behalf would be appreciated; I just don't want you to waste your time!)

[NauticalMile64]

@tsmithe you are correct: it does stop for me in the Lanczos algorithm.

On another note, I have the gcc compiler (TDM-GCC) for windows. Is there an option I could change in setup.py or elsewhere to use gcc instead?

[tsmithe]

Alas, no: the usual Python binaries for Windows are built using MSVC, and link against the MSVC runtimes. If you try building extensions against different runtime libraries, things tend to go haywire, hence my fairly detailed prescriptions in the README. You are of course free to build Python yourself with GCC, but then you'd be stuck compiling all the extensions, too -- and I'd be surprised if your dedication to the Lanczos algorithm were so great!

[NauticalMile64]

@tsmithe Thanks for the info. I'm not that ambitious :).

I don't have any particular insights into the problem, other than what I've mentioned above. I will be testing the code once the changes have been made, and I'll be sure to report anymore errors.

[tsmithe]

@ndyck -- just FYI, you should have a fixed release around the middle of next week :)

[tsmithe]

OK, so I've not announced it yet, but if you have a recent version of pip, you should be able to run pip install pyviennacl and get version 1.0.3, which fixes all the issues we've spotted. It'll be a binary installation, so you don't need to worry about compilers etc, but do get back to me if it works or doesn't!

[NauticalMile64]

No errors during the install, I'll try it out tomorrow and let you know of any issues.