rogermcintosh commented 9 years ago

Hi I am evaluating kdb+/q for use in managing large data volumes for an investment process and was keen on utilising PyQ to help manage data wrangling into a kdb database. I am restricted to operating on Windows and noticed that the pip install or build from source won't work as it is written only for Unix variants OSs. Is there any chance this could be extended to Windows? I'd have a crack at it myself, but my programming skills not that extensive .. Would it require much work to adjust to compile on WIndows? I am running Python 3.4

abalkin commented 9 years ago

We have no plans to support Windows at the moment. The major supported platforms are MacOS and Linux while we had some success with Solaris and FreeBSD.

The code itself should compile without any problems on Windows, but some changes will be required to setup.py.

sashkab commented 7 years ago

Experimental support for Windows has landed with the PyQ 4.1.0.

Ensure that kdb+ is installed under C:\q, or QHOME environment variable properly set.

Then, you can install PyQ on Windows using

pip install -U pyq

You can start PyQ by running

c:\q\w32\q.exe python.q

Note, you will have to press ^Z and then Enter key in order to get into Python REPL. This is known limitation of this version.

You can run tests too: first install required packages:

pip install pytest pytest-pyq

Then run:

set QBIN=c:\q\w32\q.exe
c:\q\w32\q.exe python.q -mpytest --pyargs pyq < nul

We appreciate all bug reports and any feedback.

bevinahal commented 7 years ago

Thanks so much for this release - this is a very good feature.

My installation fails with below error. Is this because I'm using 64-bit q.exe version? I did run vcvarsall.bat x64 before calling "python setup.py install"

Python version : 3.5.2 KDB version : 3.1

Error -

C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIB
ATH:C:\Users\44025173\AppData\Local\Continuum\Anaconda3\libs /LIBPATH:C:\Users\44025173\AppData\Local\Continuum\Anaconda3\PCbuild\amd64 "/LIBPATH:C:\Progr
m Files (x86)\Microsoft Visual Studio 14.0\VC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\LIB\amd64" "/LIBPATH:C:\P
ogram Files (x86)\Windows Kits\10\lib\10.0.10240.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64" "/LIBPATH:C:\Program
Files (x86)\Windows Kits\8.1\lib\winv6.3\um\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\LIB\amd64" "/LIBPATH:C:\Program Files (x
6)\Microsoft Visual Studio 14.0\VC\ATLMFC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.10240.0\ucrt\x64" "/LIBPATH:C:\Program File
 (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\8.1\lib\winv6.3\um\x64" /LIBPATH:C:\SYBASE\OCS-12_5\lib /EXPO
T:PyInit__k build\temp.win-amd64-3.5\Release\src/pyq/_k.obj /OUT:build\lib.win-amd64-3.5\pyq\_k.cp35-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.5\Releas
\src/pyq\_k.cp35-win_amd64.lib src\pyq\kx\w64\q.lib
_k.obj : warning LNK4197: export 'PyInit__k' specified multiple times; using first specification
   Creating library build\temp.win-amd64-3.5\Release\src/pyq\_k.cp35-win_amd64.lib and object build\temp.win-amd64-3.5\Release\src/pyq\_k.cp35-win_amd64.e
p
_k.obj : error LNK2001: unresolved external symbol clr
build\lib.win-amd64-3.5\pyq\_k.cp35-win_amd64.pyd : fatal error LNK1120: 1 unresolved externals
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\link.exe' failed with exit status 1120

abalkin commented 7 years ago

Hi @bevinahal,

Thank you for your report. I don't think 64bit is an issue here. More likely it is your use of kdb+ 3.1. We only fully tested PyQ with the latest (3.5, 32bit) kdb+, but it does compile with (3.5, 64bit) as well.

The problem is that with kdb+ < 3.5, PyQ uses an undocumented clr() function which apparently is not included in q.lib. I see the following options for you

Upgrade to kdb+ 3.5.
Add void clr(void) definition to q.lib. Your Kx support may be able to help you with that.
Comment out line 158 in _k.c.

If you decide to take option (3), see "Installing from Source Code" instructions. Please let us know if any of these solutions work for you.

abalkin commented 7 years ago

Dear @chenkangcrack,

Thank you for your report. For the 4.1 release, we've only tested PyQ on Windows with Python 3.6. However, it looks like only a minor change is needed to make it compile under Python 2.7.

I am attaching the necessary patch. To apply, clone this repository, download the attachment and run

git apply 0001-BUG-946-Fixed-compilation-errors-for-Python-2.7-on-W.patch.txt

You should then be able to install pyq with

pip install .

Please let us know if you have any other issues. We will include these changes in the next bug-fix release.

0001-BUG-946-Fixed-compilation-errors-for-Python-2.7-on-W.patch.txt

sashkab commented 7 years ago

@chenkangcrack Please try to start PyQ as described above by runningc:\q\w32\q.exe python.q (replace c:\q\w32 with path to your kdb+ installation).

When installing PyQ, please use Command Prompt, not git/mingw shell. You might need to install Visual Studio 9 for Python if using Python 2.7.x, and pip will instruct you what to do.

abalkin commented 7 years ago

Hi @chenkangcrack,

Thank you for trying the patch. Please note that PyQ runs Python inside a kdb+ process and cannot be loaded in a standard Python. To start an interactive session, please enter the following command:

c:\q\w32\q.exe python.q

As explained above, you will have to press Control+Z and then Enter keys to break out of the kdb+ REPL and start entering Python commands. (When running python scripts, redirect STDIN to nul.)

To make sure you've succeeded in building PyQ, please run the included test suit:

pip install pytest pytest-pyq
set QBIN=c:\q\w32\q.exe
c:\q\w32\q.exe python.q -mpytest --pyargs pyq < nul

ghlian commented 6 years ago

This is error I got. win 10 64bit python 36 32bit compile the pyq successfully but cannot launch it.

(python36_32) C:\Users\ligu1>c:\q\w32\q.exe python.q
KDB+ 3.5 2017.09.06 Copyright (C) 1993-2017 Kx Systems
w32/ 4()core 4095MB ghlian liandelllaptop 192.168.43.33 NONEXPIRE

Welcome to kdb+ 32bit edition
For support please see http://groups.google.com/d/forum/personal-kdbplus
Tutorials can be found at http://code.kx.com/wiki/Tutorials
To exit, type \\
To remove this startup msg, edit q.q
Fatal Python error: Py_Initialize: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x000060c0 (most recent call first):

ghlian commented 6 years ago

After failing to launch it in python36 above (the pyq crashed)

I tried to do it in python 27 as you described, got the error and cannot start it.

(python27_32) C:\Users\ligu1\pyq>pip install .
Processing c:\users\ligu1\pyq
Installing collected packages: pyq
  Running setup.py install for pyq ... done
Exception:
Traceback (most recent call last):
  File "F:\Anaconda_3_64\envs\python27_32\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "F:\Anaconda_3_64\envs\python27_32\lib\site-packages\pip\commands\install.py", line 342, in run
    prefix=options.prefix_path,
  File "F:\Anaconda_3_64\envs\python27_32\lib\site-packages\pip\req\req_set.py", line 784, in install
    **kwargs
  File "F:\Anaconda_3_64\envs\python27_32\lib\site-packages\pip\req\req_install.py", line 919, in install
    prepend_root(filename), egg_info_dir)
  File "F:\Anaconda_3_64\envs\python27_32\lib\ntpath.py", line 529, in relpath
    % (path_prefix, start_prefix))
ValueError: path is on drive C:, start on drive F:

Is it because my python is installed in F driver?

Thanks a lot.

abalkin commented 6 years ago

Dear @ghlian,

It looks like you are using conda. We have not tested PyQ with it. I suggest that you install the official CPython distribution from https://www.python.org/downloads/windows/. For the 32-bit version, choose "Download Windows x86 MSI installer".

abalkin commented 6 years ago

Dear @ghlian,

I've just downloaded Miniconda3 (32bit) and successfully installed pyq:

(C:\Users\a\Miniconda3) C:\Users\a>pip install --no-binary pyq pyq
Collecting pyq
  Downloading pyq-4.1.0.post1.tar.gz (93kB)
    100% |████████████████████████████████| 102kB 655kB/s
Skipping bdist_wheel for pyq, due to binaries being disabled for it.
Installing collected packages: pyq
  Running setup.py install for pyq ... done
Successfully installed pyq-4.1.0

(C:\Users\a\Miniconda3) C:\Users\a>c:\q\w32\q python.q
KDB+ 3.5 2017.03.28 Copyright (C) 1993-2017 Kx Systems
w32/ 1()core 4095MB a enlnt-win 10.200.144.21 NONEXPIRE

Python 3.6.1 |Continuum Analytics, Inc.| (default, May 11 2017, 14:16:49) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
^Z

>>> 1+2
3
>>> from pyq import q
>>> q.til(5)
k('0 1 2 3 4')
>>> exit()

There was a slight problem running tests: the standard pytest package had a trouble parsing Anaconda Python version:

(C:\Users\a\Miniconda3) C:\Users\a>pip install pytest pytest-pyq
..
(C:\Users\a\Miniconda3) C:\Users\a>set QBIN=c:\q\w32\q.exe
(C:\Users\a\Miniconda3) C:\Users\a>c:\q\w32\q.exe python.q -mpytest --pyargs pyq < nul
..
INTERNALERROR> ValueError: failed to parse CPython sys.version: '3.6.1 |Continuum Analytics, Inc.| (default, May 11 2017, 14:16:49) [MSC v.1900 32 bit (Intel)]'

As a work-around, I ran pytest in a "quiet" mode:

(C:\Users\a\Miniconda3) C:\Users\a>c:\q\w32\q.exe python.q -mpytest --quiet --pyargs pyq < nul
KDB+ 3.5 2017.03.28 Copyright (C) 1993-2017 Kx Systems
w32/ 1()core 4095MB a enlnt-win 10.200.144.21 NONEXPIRE

s................................................................................................................................................................................................................................s...........................................s........................................................................................................................................s....................................................s................................................................x..................................................................................................s....ssssssssssssssssssssssssssss.......s.......................................................................................................s.........................................................................................................................................................................................ss.....
921 passed, 38 skipped, 1 xfailed in 3.59 seconds

Overall, it seems to work for me. I suspect you have some custom modifications in your Anaconda installation. Can you try to install a fresh version of Miniconda3? Please make sure that you use 32bit Python with 32bit q.

ghlian commented 6 years ago

Thanks very much. I did get it installed and run when python is installed in C driver.

But when it is installed in other driver. You get the error.

(python36_32) C:\Users\ligu1\Desktop>pip install --no-binary pyq pyq
Collecting pyq
  Using cached pyq-4.1.0.post1.tar.gz
Skipping bdist_wheel for pyq, due to binaries being disabled for it.
Installing collected packages: pyq
  Running setup.py install for pyq ... done
Exception:
Traceback (most recent call last):
  File "F:\Python\Anaconda_36_64\envs\python36_32\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "F:\Python\Anaconda_36_64\envs\python36_32\lib\site-packages\pip\commands\install.py", line 342, in run
    prefix=options.prefix_path,
  File "F:\Python\Anaconda_36_64\envs\python36_32\lib\site-packages\pip\req\req_set.py", line 784, in install
    **kwargs
  File "F:\Python\Anaconda_36_64\envs\python36_32\lib\site-packages\pip\req\req_install.py", line 919, in install
    prepend_root(filename), egg_info_dir)
  File "F:\Python\Anaconda_36_64\envs\python36_32\lib\ntpath.py", line 585, in relpath
    path_drive, start_drive))
ValueError: path is on mount 'C:', start on mount 'F:'

abalkin commented 6 years ago

@ghlian, the problem that you see does not look like anything specific to pyq. It looks like an issue with pip. I don't have a Windows setup with multiple drives at the moment, but I would suggest that you try to set the TMPDIR environment variable to some path on the F: drive, e.g. F:\tmp, assuming that such directory exists and you have permissions to read and write into it.

ghlian commented 6 years ago

Thank you!! I will leave it in C driver for now. How to launch it pyq in Jupyter in Windows? I guess the way for linux (pyq -m notebook) does not fit Windows because pyq is not defined.

abalkin commented 6 years ago

Try

c:\q\w32\q python.q -mnotebook

Make sure that there is no space between -m and the name of the python module. Alternatively, you can add @ after -m:

c:\q\w32\q python.q -m@ notebook

This is necessary because q tries to interpret any single-letter option as its own.

You may also want to install the pyq.kernel add-on. See https://pypi.python.org/pypi/pyq-kernel.

ghlian commented 6 years ago

Is there any problem with the specification here

"kernelspecs": {
    "pyq_3": {
      "resource_dir": "C:\\Users\\ligu1\\AppData\\Roaming\\jupyter\\kernels\\pyq_3",
      "spec": {
        "argv": [
          null,
          "pyq-kernel.p",
          "1024",
          "65535",
          "{connection_file}"
        ],
        "env": {
          "QHOME": "C:\\q"
        },
        "display_name": "PyQ 3",
        "language": "python"
      }
    },

The jupyter was launched but the kernel of pyq cannot be started with the error

Traceback (most recent call last):
  File "C:\python_36_64_raw\lib\site-packages\notebook\base\handlers.py", line 540, in wrapper
    result = yield gen.maybe_future(method(self, *args, **kwargs))
  File "C:\python_36_64_raw\lib\site-packages\tornado\gen.py", line 1055, in run
    value = future.result()
  File "C:\python_36_64_raw\lib\site-packages\tornado\concurrent.py", line 238, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "C:\python_36_64_raw\lib\site-packages\tornado\gen.py", line 1063, in run
    yielded = self.gen.throw(*exc_info)
  File "C:\python_36_64_raw\lib\site-packages\notebook\services\sessions\handlers.py", line 75, in post
    type=mtype))
  File "C:\python_36_64_raw\lib\site-packages\tornado\gen.py", line 1055, in run
    value = future.result()
  File "C:\python_36_64_raw\lib\site-packages\tornado\concurrent.py", line 238, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "C:\python_36_64_raw\lib\site-packages\tornado\gen.py", line 1063, in run
    yielded = self.gen.throw(*exc_info)
  File "C:\python_36_64_raw\lib\site-packages\notebook\services\sessions\sessionmanager.py", line 79, in create_session
    kernel_id = yield self.start_kernel_for_session(session_id, path, name, type, kernel_name)
  File "C:\python_36_64_raw\lib\site-packages\tornado\gen.py", line 1055, in run
    value = future.result()
  File "C:\python_36_64_raw\lib\site-packages\tornado\concurrent.py", line 238, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "C:\python_36_64_raw\lib\site-packages\tornado\gen.py", line 1063, in run
    yielded = self.gen.throw(*exc_info)
  File "C:\python_36_64_raw\lib\site-packages\notebook\services\sessions\sessionmanager.py", line 92, in start_kernel_for_session
    self.kernel_manager.start_kernel(path=kernel_path, kernel_name=kernel_name)
  File "C:\python_36_64_raw\lib\site-packages\tornado\gen.py", line 1055, in run
    value = future.result()
  File "C:\python_36_64_raw\lib\site-packages\tornado\concurrent.py", line 238, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "C:\python_36_64_raw\lib\site-packages\tornado\gen.py", line 307, in wrapper
    yielded = next(result)
  File "C:\python_36_64_raw\lib\site-packages\notebook\services\kernels\kernelmanager.py", line 123, in start_kernel
    super(MappingKernelManager, self).start_kernel(**kwargs)
  File "C:\python_36_64_raw\lib\site-packages\jupyter_client\multikernelmanager.py", line 110, in start_kernel
    km.start_kernel(**kwargs)
  File "C:\python_36_64_raw\lib\site-packages\jupyter_client\manager.py", line 244, in start_kernel
    kernel_cmd = self.format_kernel_cmd(extra_arguments=extra_arguments)
  File "C:\python_36_64_raw\lib\site-packages\jupyter_client\manager.py", line 196, in format_kernel_cmd
    return [ pat.sub(from_ns, arg) for arg in cmd ]
  File "C:\python_36_64_raw\lib\site-packages\jupyter_client\manager.py", line 196, in <listcomp>
    return [ pat.sub(from_ns, arg) for arg in cmd ]
TypeError: expected string or bytes-like object

hzadonis commented 6 years ago

Exception report at Windows 10, KDB+ 3.5 2017.03.15

D:\Program\Python36\Scripts>where q
C:\q\w32\q.exe

D:\Program\Python36\Scripts>D:\Program\Python36\Scripts\pip.exe install --no-binary pyq pyq
Collecting pyq
  Using cached pyq-4.1.0.post1.tar.gz
Skipping bdist_wheel for pyq, due to binaries being disabled for it.
Installing collected packages: pyq
  Running setup.py install for pyq ... error
Exception:
Traceback (most recent call last):
  File "d:\program\python36\lib\site-packages\pip\compat\__init__.py", line 73, in console_to_str
    return s.decode(sys.__stdout__.encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 26: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "d:\program\python36\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "d:\program\python36\lib\site-packages\pip\commands\install.py", line 342, in run
    prefix=options.prefix_path,
  File "d:\program\python36\lib\site-packages\pip\req\req_set.py", line 784, in install
    **kwargs
  File "d:\program\python36\lib\site-packages\pip\req\req_install.py", line 878, in install
    spinner=spinner,
  File "d:\program\python36\lib\site-packages\pip\utils\__init__.py", line 676, in call_subprocess
    line = console_to_str(proc.stdout.readline())
  File "d:\program\python36\lib\site-packages\pip\compat\__init__.py", line 75, in console_to_str
    return s.decode('utf_8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 26: invalid start byte

abalkin commented 6 years ago

@hzadonis, it looks like your pip process is having trouble displaying the progress bar. Please try running it in silent mode by adding the -qqq option on the command line. You can also try setting the PYTHONIOENCODING environment variable to utf-8.

abalkin commented 6 years ago

@ghlian -

Is there any problem with the specification here?

"kernelspecs": {
  "pyq_3": {
    "resource_dir": "C:\Users\ligu1\AppData\Roaming\jupyter\kernels\pyq_3",
    "spec": {
      "argv": [
        null,
        "pyq-kernel.p",
        "1024",
        "65535",
        "{connection_file}"
    ],
    "env": {
      "QHOME": "C:\q"
    },
    "display_name": "PyQ 3",
    "language": "python"
  }
},

Yes, first I believe json requires that backslashes are escaped. (Double every backslash in the file paths.) Second, make sure the environment variable QBIN is set to the full path to q.exe before you run q python.q -mpyq.kernel install. Alternatively, edit the kernel.json file and replace null in argv list with the path to q.exe. Finally, I am not sure it causes any harm, but the trailing comma is unnecessary.

abalkin commented 6 years ago

Installing Jupyter kernel

Since we have not ported the pyq executable to the Windows platform yet, setting up a working PyQ environment on Windows requires several manual steps.

First, it is strongly recommended to use a dedicated Python virtual environment and install q in %VIRTUAL_ENV%. Assuming that you have downloaded windows.zip from https://kx.com/download/ in your Downloads folder, enter the following commands:

python -mvenv py36
py36\Scripts\activate.bat
set QHOME=%VIRTUAL_ENV%\q
"C:\Program Files\7-Zip\7z.exe" x -y -o%VIRTUAL_ENV% %HOMEPATH%\Downloads\windows.zip
del %QHOME%\q.q
set PYTHONPATH=%VIRTUAL_ENV%\lib\site-packages
set QBIN=%QHOME%\w32\q.exe

Now you should be able to install jupyter, pyq and pyq-kernel in one commad

pip install jupyter pyq pyq-kernel

Finally, to install pyq kernel specs, run

%QBIN% python.q -mpyq.kernel install

If everything is successful, you should see pyq_3 listed in the kernelspec list:

>jupyter kernelspec list
Available kernels:
  pyq_3      C:\Users\a\AppData\Roaming\jupyter\kernels\pyq_3
  python3    c:\users\a\py36\share\jupyter\kernels\python3

Now, start the notebook server

jupyter-notebook

and select "PyQ 3" from the "New" menu. For examples of what can be done in a PyQ notebook, please see my presentation at https://youtu.be/v2UoP0l6mOw.

ghlian commented 6 years ago

Thank. I figured out the problem is the file "pyq-kernel.p" is not accessible by pyq. Specifying the full path solves the problem.

ghlian commented 6 years ago

@abalkin When pyq is launch and q is imported, is there any way that we can use other IDE sublime or q-studio to connect to the same q session in this pyq? It will be helpful and efficient to have this figure for working.

abalkin commented 6 years ago

@ghlian,

Yes, as long as you don't load python.q, which makes your q process enter into Python REPL, you should be able to have normal kdb+ connections and run python code. The easiest way to achieve that is to install Jupyter and pyq-kernel. I posted Windows instructions in my earlier comment.

Once you have pyq-kernel running inside your q process, you can connect a Jupyter notebook or an IPython console to it as well as qcon or any q IDE that can talk to a kdb+ server.

hzadonis commented 6 years ago

@abalkin Thanks, I had set PYTHONIOENCODING and use -qqq parameter for installation. But both solution failed. such as:

D:\>D:\Program\Python36\Scripts\pip.exe install --no-binary pyq pyq
Collecting pyq
  Using cached pyq-4.1.0.post1.tar.gz
Skipping bdist_wheel for pyq, due to binaries being disabled for it.
Installing collected packages: pyq
  Running setup.py install for pyq ... error
Exception:
Traceback (most recent call last):
  File "d:\program\python36\lib\site-packages\pip\compat\__init__.py", line 73, in console_to_str
    return s.decode(sys.__stdout__.encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 26: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "d:\program\python36\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "d:\program\python36\lib\site-packages\pip\commands\install.py", line 342, in run
    prefix=options.prefix_path,
  File "d:\program\python36\lib\site-packages\pip\req\req_set.py", line 784, in install
    **kwargs
  File "d:\program\python36\lib\site-packages\pip\req\req_install.py", line 878, in install
    spinner=spinner,
  File "d:\program\python36\lib\site-packages\pip\utils\__init__.py", line 676, in call_subprocess
    line = console_to_str(proc.stdout.readline())
  File "d:\program\python36\lib\site-packages\pip\compat\__init__.py", line 75, in console_to_str
    return s.decode('utf_8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 26: invalid start byte

D:\>D:\Program\Python36\Scripts\pip.exe install --no-binary -qqq pyq pyq
Collecting pyq
  Using cached pyq-4.1.0.post1.tar.gz
Building wheels for collected packages: pyq
  Running setup.py bdist_wheel for pyq ... error
  Failed building wheel for pyq
  Running setup.py clean for pyq
Failed to build pyq
Installing collected packages: pyq
  Running setup.py install for pyq ... error
Exception:
Traceback (most recent call last):
  File "d:\program\python36\lib\site-packages\pip\compat\__init__.py", line 73, in console_to_str
    return s.decode(sys.__stdout__.encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 26: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "d:\program\python36\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "d:\program\python36\lib\site-packages\pip\commands\install.py", line 342, in run
    prefix=options.prefix_path,
  File "d:\program\python36\lib\site-packages\pip\req\req_set.py", line 784, in install
    **kwargs
  File "d:\program\python36\lib\site-packages\pip\req\req_install.py", line 878, in install
    spinner=spinner,
  File "d:\program\python36\lib\site-packages\pip\utils\__init__.py", line 676, in call_subprocess
    line = console_to_str(proc.stdout.readline())
  File "d:\program\python36\lib\site-packages\pip\compat\__init__.py", line 75, in console_to_str
    return s.decode('utf_8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 26: invalid start byte

Do you have other comments? thanks!

abalkin commented 6 years ago

@hzadonis, what you see is puzzling. Let's get some more diagnostic. Please do set DISTUTILS_DEBUG=1 and run pip with a verbose (-v) flag:

set DISTUTILS_DEBUG=1
pip uninstall pyq
pip install -v pyq

When you post the output, please surround it with the triple backquotes as I did with the command snippet above. This way file names with underscores will not get mangled.

sashkab commented 6 years ago

@hzadonis, which versions of pip and setuptools do you have installed? Which version of the Windows do you have, is it 32-bit or 64-bit? Is it English language Windows, or international?

sashkab commented 6 years ago

PyQ 4.1.1 has been released, which incorporated previously shared patches in this thread.

hzadonis commented 6 years ago

@sashkab pip version is 9.0.1 setuptools version is 36.4.0 Windows version is Windows 10 Professional, 64-bit, Chinese version.

hzadonis commented 6 years ago

@abalkin the output of "pip install -v pyq":

C:\Users\hzado>D:\Program\Python36\Scripts\pip.exe install -v pyq
Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect
Config variable 'WITH_PYMALLOC' is unset, Python ABI tag may be incorrect
Collecting pyq
  1 location(s) to search for versions of pyq:
  * https://pypi.python.org/simple/pyq/
  Getting page https://pypi.python.org/simple/pyq/
  Looking up "https://pypi.python.org/simple/pyq/" in the cache
  Current age based on date: 426005
  Freshness lifetime from max-age: 600
  Freshness lifetime from request max-age: 600
  The cached response is "stale" with no etag, purging
  Starting new HTTPS connection (1): pypi.python.org
  "GET /simple/pyq/ HTTP/1.1" 200 306
  Updating cache with response from "https://pypi.python.org/simple/pyq/"
  Caching b/c date exists and max-age > 0
  Analyzing links from page https://pypi.python.org/simple/pyq/
    Found link https://pypi.python.org/packages/66/2c/d1e1891e293edd7905ac4d1d6e4f9f30ff7e40cebf8e2a2629bf4f9f4152/pyq-4.1.0.post1.tar.gz#md5=01cefe3276ac91a40856b4ab62a8dd6f (from https://pypi.python.org/simple/pyq/), version: 4.1.0.post1
    Found link https://pypi.python.org/packages/c0/a6/6dad579c65ded93d9da0fdf9e0da268870c9cea5abd6e2d0606b67f0905d/pyq-4.1.1.tar.gz#md5=ac587d42c57d3558254b90b7b98677db (from https://pypi.python.org/simple/pyq/), version: 4.1.1
  Using version 4.1.1 (newest of versions: 4.1.0.post1, 4.1.1)
  Looking up "https://pypi.python.org/packages/c0/a6/6dad579c65ded93d9da0fdf9e0da268870c9cea5abd6e2d0606b67f0905d/pyq-4.1.1.tar.gz" in the cache
  No cache entry available
  "GET /packages/c0/a6/6dad579c65ded93d9da0fdf9e0da268870c9cea5abd6e2d0606b67f0905d/pyq-4.1.1.tar.gz HTTP/1.1" 200 94302
  Downloading pyq-4.1.1.tar.gz (94kB)
  Downloading from URL https://pypi.python.org/packages/c0/a6/6dad579c65ded93d9da0fdf9e0da268870c9cea5abd6e2d0606b67f0905d/pyq-4.1.1.tar.gz#md5=ac587d42c57d3558254b90b7b98677db (from https://pypi.python.org/simple/pyq/)
    97% |███████████████████████████████▎| 92kB 123kB/s eta 0:00:01  Updating cache with response from "https://pypi.python.org/packages/c0/a6/6dad579c65ded93d9da0fdf9e0da268870c9cea5abd6e2d0606b67f0905d/pyq-4.1.1.tar.gz"
  Caching due to etag
    100% |████████████████████████████████| 102kB 134kB/s
  Running setup.py (path:C:\Users\hzado\AppData\Local\Temp\pip-build-7luaz37j\pyq\setup.py) egg_info for package pyq
    Running command python setup.py egg_info
    options (after parsing config files):
    options (after parsing command line):
    option dict for 'aliases' command:
      {}
    option dict for 'egg_info' command:
      {'egg_base': ('command line', 'pip-egg-info')}
    running egg_info
    Distribution.get_command_obj(): creating 'egg_info' command object
      setting options for 'egg_info' command:
        egg_base = pip-egg-info (from command line)
    creating pip-egg-info\pyq.egg-info
    writing pip-egg-info\pyq.egg-info\PKG-INFO
    writing dependency_links to pip-egg-info\pyq.egg-info\dependency_links.txt
    writing requirements to pip-egg-info\pyq.egg-info\requires.txt
    writing top-level names to pip-egg-info\pyq.egg-info\top_level.txt
    writing manifest file 'pip-egg-info\pyq.egg-info\SOURCES.txt'
    Distribution.get_command_obj(): creating 'build_py' command object
    Distribution.get_command_obj(): creating 'build' command object
    Distribution.get_command_obj(): creating 'build_ext' command object
    Distribution.get_command_obj(): creating 'build_qext' command object
    Distribution.get_command_obj(): creating 'config' command object
    Distribution.get_command_obj(): creating 'build_scripts' command object
    reading manifest file 'pip-egg-info\pyq.egg-info\SOURCES.txt'
    exclude_pattern: applying regex r'(^|\\)(RCS|CVS|\.svn)\\'
    writing manifest file 'pip-egg-info\pyq.egg-info\SOURCES.txt'
  Source in c:\users\hzado\appdata\local\temp\pip-build-7luaz37j\pyq has version 4.1.1, which satisfies requirement pyq from https://pypi.python.org/packages/c0/a6/6dad579c65ded93d9da0fdf9e0da268870c9cea5abd6e2d0606b67f0905d/pyq-4.1.1.tar.gz#md5=ac587d42c57d3558254b90b7b98677db
Building wheels for collected packages: pyq
  Running setup.py bdist_wheel for pyq ...   Destination directory: C:\Users\hzado\AppData\Local\Temp\tmp4lyfnu2spip-wheel-
  Running command d:\program\python36\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\hzado\\AppData\\Local\\Temp\\pip-build-7luaz37j\\pyq\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d C:\Users\hzado\AppData\Local\Temp\tmp4lyfnu2spip-wheel- --python-tag cp36
  options (after parsing config files):
  options (after parsing command line):
  option dict for 'aliases' command:
    {}
  option dict for 'bdist_wheel' command:
    {'dist_dir': ('command line',
                  'C:\\Users\\hzado\\AppData\\Local\\Temp\\tmp4lyfnu2spip-wheel-'),
     'python_tag': ('command line', 'cp36')}
  running bdist_wheel
  Distribution.get_command_obj(): creating 'bdist_wheel' command object
    setting options for 'bdist_wheel' command:
      dist_dir = C:\Users\hzado\AppData\Local\Temp\tmp4lyfnu2spip-wheel- (from command line)
      python_tag = cp36 (from command line)
  Distribution.get_command_obj(): creating 'bdist' command object
  Distribution.get_command_obj(): creating 'build' command object
  Distribution.get_command_obj(): creating 'build_scripts' command object
  running build
  running build_py
  Distribution.get_command_obj(): creating 'build_py' command object
  creating build
  creating build\lib.win-amd64-3.6
  creating build\lib.win-amd64-3.6\pyq
  copying src\pyq\cmd.py -> build\lib.win-amd64-3.6\pyq
  copying src\pyq\conftest.py -> build\lib.win-amd64-3.6\pyq
  copying src\pyq\magic.py -> build\lib.win-amd64-3.6\pyq
  copying src\pyq\ptk.py -> build\lib.win-amd64-3.6\pyq
  copying src\pyq\version.py -> build\lib.win-amd64-3.6\pyq
  copying src\pyq\_n.py -> build\lib.win-amd64-3.6\pyq
  copying src\pyq\_pt_run.py -> build\lib.win-amd64-3.6\pyq
  copying src\pyq\__init__.py -> build\lib.win-amd64-3.6\pyq
  creating build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_ci.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_com.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_k.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_magic.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_memoryview.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_n.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_no_numpy.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_numpy.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_p.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_pathlib.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_pypath.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_pyq_executable.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_python_q.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_q.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_sd.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_threads.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_version.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\__init__.py -> build\lib.win-amd64-3.6\pyq\tests
  running build_ext
  Distribution.get_command_obj(): creating 'build_ext' command object
  Distribution.get_command_obj(): creating 'build_qext' command object
  Distribution.get_command_obj(): creating 'config' command object
  building 'pyq._k' extension
  creating build\temp.win-amd64-3.6
  creating build\temp.win-amd64-3.6\Release
  creating build\temp.win-amd64-3.6\Release\src
  creating build\temp.win-amd64-3.6\Release\src\pyq
  D:\Program\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG -DKXVER=3 -DKXVER2=5 -Id:\program\python36\include -Id:\program\python36\include "-ID:\Program\Microsoft Visual Studio 14.0\VC\INCLUDE" "-ID:\Program\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\winrt" /Tcsrc/pyq/_k.c /Fobuild\temp.win-amd64-3.6\Release\src/pyq/_k.obj /WX
  _k.c
  D:\Program\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:d:\program\python36\libs /LIBPATH:d:\program\python36\PCbuild\amd64 "/LIBPATH:D:\Program\Microsoft Visual Studio 14.0\VC\LIB\amd64" "/LIBPATH:D:\Program\Microsoft Visual Studio 14.0\VC\ATLMFC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.14393.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.14393.0\um\x64" /EXPORT:PyInit__k build\temp.win-amd64-3.6\Release\src/pyq/_k.obj /OUT:build\lib.win-amd64-3.6\pyq\_k.cp36-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.6\Release\src/pyq\_k.cp36-win_amd64.lib src\pyq\kx\w32\q.lib
error
  Failed building wheel for pyq
  Running setup.py clean for pyq
  Running command d:\program\python36\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\hzado\\AppData\\Local\\Temp\\pip-build-7luaz37j\\pyq\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" clean --all
  options (after parsing config files):
  options (after parsing command line):
  option dict for 'aliases' command:
    {}
  option dict for 'clean' command:
    {'all': ('command line', 1)}
  running clean
  Distribution.get_command_obj(): creating 'clean' command object
    setting options for 'clean' command:
      all = 1 (from command line)
  Distribution.get_command_obj(): creating 'build' command object
  Distribution.get_command_obj(): creating 'bdist' command object
  removing 'build\temp.win-amd64-3.6' (and everything under it)
  removing 'build\lib.win-amd64-3.6' (and everything under it)
  'build\bdist.win-amd64' does not exist -- can't clean it
  'build\scripts-3.6' does not exist -- can't clean it
  removing 'build'
Failed to build pyq
Installing collected packages: pyq
  Running setup.py install for pyq ...     Running command d:\program\python36\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\hzado\\AppData\\Local\\Temp\\pip-build-7luaz37j\\pyq\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\hzado\AppData\Local\Temp\pip-jye5rmwm-record\install-record.txt --single-version-externally-managed --compile
    options (after parsing config files):
    options (after parsing command line):
    option dict for 'aliases' command:
      {}
    option dict for 'install' command:
      {'compile': ('command line', 1),
       'record': ('command line',
                  'C:\\Users\\hzado\\AppData\\Local\\Temp\\pip-jye5rmwm-record\\install-record.txt'),
       'single_version_externally_managed': ('command line', 1)}
    running install
    Distribution.get_command_obj(): creating 'install' command object
      setting options for 'install' command:
        record = C:\Users\hzado\AppData\Local\Temp\pip-jye5rmwm-record\install-record.txt (from command line)
        single_version_externally_managed = 1 (from command line)
        compile = 1 (from command line)
    config vars:
    {'abiflags': '',
     'base': 'd:\\program\\python36',
     'dist_fullname': 'pyq-4.1.1',
     'dist_name': 'pyq',
     'dist_version': '4.1.1',
     'exec_prefix': 'd:\\program\\python36',
     'platbase': 'd:\\program\\python36',
     'prefix': 'd:\\program\\python36',
     'py_version': '3.6.2',
     'py_version_nodot': '36',
     'py_version_short': '3.6',
     'sys_exec_prefix': 'd:\\program\\python36',
     'sys_prefix': 'd:\\program\\python36',
     'userbase': 'C:\\Users\\hzado\\AppData\\Roaming\\Python',
     'usersite': 'C:\\Users\\hzado\\AppData\\Roaming\\Python\\Python36\\site-packages'}
    Distribution.get_command_obj(): creating 'build' command object
    running build
    running build_py
    Distribution.get_command_obj(): creating 'build_py' command object
    creating build
    creating build\lib.win-amd64-3.6
    creating build\lib.win-amd64-3.6\pyq
    copying src\pyq\cmd.py -> build\lib.win-amd64-3.6\pyq
    copying src\pyq\conftest.py -> build\lib.win-amd64-3.6\pyq
    copying src\pyq\magic.py -> build\lib.win-amd64-3.6\pyq
    copying src\pyq\ptk.py -> build\lib.win-amd64-3.6\pyq
    copying src\pyq\version.py -> build\lib.win-amd64-3.6\pyq
    copying src\pyq\_n.py -> build\lib.win-amd64-3.6\pyq
    copying src\pyq\_pt_run.py -> build\lib.win-amd64-3.6\pyq
    copying src\pyq\__init__.py -> build\lib.win-amd64-3.6\pyq
    creating build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_ci.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_com.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_k.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_magic.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_memoryview.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_n.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_no_numpy.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_numpy.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_p.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_pathlib.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_pypath.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_pyq_executable.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_python_q.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_q.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_sd.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_threads.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_version.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\__init__.py -> build\lib.win-amd64-3.6\pyq\tests
    running build_ext
    Distribution.get_command_obj(): creating 'build_ext' command object
    Distribution.get_command_obj(): creating 'build_qext' command object
    Distribution.get_command_obj(): creating 'config' command object
    building 'pyq._k' extension
    creating build\temp.win-amd64-3.6
    creating build\temp.win-amd64-3.6\Release
    creating build\temp.win-amd64-3.6\Release\src
    creating build\temp.win-amd64-3.6\Release\src\pyq
    D:\Program\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG -DKXVER=3 -DKXVER2=5 -Id:\program\python36\include -Id:\program\python36\include "-ID:\Program\Microsoft Visual Studio 14.0\VC\INCLUDE" "-ID:\Program\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\winrt" /Tcsrc/pyq/_k.c /Fobuild\temp.win-amd64-3.6\Release\src/pyq/_k.obj /WX
    _k.c
    D:\Program\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:d:\program\python36\libs /LIBPATH:d:\program\python36\PCbuild\amd64 "/LIBPATH:D:\Program\Microsoft Visual Studio 14.0\VC\LIB\amd64" "/LIBPATH:D:\Program\Microsoft Visual Studio 14.0\VC\ATLMFC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.14393.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.14393.0\um\x64" /EXPORT:PyInit__k build\temp.win-amd64-3.6\Release\src/pyq/_k.obj /OUT:build\lib.win-amd64-3.6\pyq\_k.cp36-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.6\Release\src/pyq\_k.cp36-win_amd64.lib src\pyq\kx\w32\q.lib
error
Cleaning up...
  Removing source in C:\Users\hzado\AppData\Local\Temp\pip-build-7luaz37j\pyq
Exception:
Traceback (most recent call last):
  File "d:\program\python36\lib\site-packages\pip\compat\__init__.py", line 73, in console_to_str
    return s.decode(sys.__stdout__.encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 26: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "d:\program\python36\lib\site-packages\pip\commands\install.py", line 342, in run
    prefix=options.prefix_path,
  File "d:\program\python36\lib\site-packages\pip\req\req_set.py", line 784, in install
    **kwargs
  File "d:\program\python36\lib\site-packages\pip\req\req_install.py", line 878, in install
    spinner=spinner,
  File "d:\program\python36\lib\site-packages\pip\utils\__init__.py", line 676, in call_subprocess
    line = console_to_str(proc.stdout.readline())
  File "d:\program\python36\lib\site-packages\pip\compat\__init__.py", line 75, in console_to_str
    return s.decode('utf_8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 26: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "d:\program\python36\lib\site-packages\pip\commands\install.py", line 385, in run
    requirement_set.cleanup_files()
  File "d:\program\python36\lib\site-packages\pip\req\req_set.py", line 729, in cleanup_files
    req.remove_temporary_source()
  File "d:\program\python36\lib\site-packages\pip\req\req_install.py", line 977, in remove_temporary_source
    rmtree(self.source_dir)
  File "d:\program\python36\lib\site-packages\pip\_vendor\retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "d:\program\python36\lib\site-packages\pip\_vendor\retrying.py", line 212, in call
    raise attempt.get()
  File "d:\program\python36\lib\site-packages\pip\_vendor\retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "d:\program\python36\lib\site-packages\pip\_vendor\six.py", line 686, in reraise
    raise value
  File "d:\program\python36\lib\site-packages\pip\_vendor\retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "d:\program\python36\lib\site-packages\pip\utils\__init__.py", line 102, in rmtree
    onerror=rmtree_errorhandler)
  File "d:\program\python36\lib\shutil.py", line 494, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "d:\program\python36\lib\shutil.py", line 393, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "d:\program\python36\lib\site-packages\pip\utils\__init__.py", line 114, in rmtree_errorhandler
    func(path)
PermissionError: [WinError 32] another program is using the file，process can't access.: 'C:\\Users\\hzado\\AppData\\Local\\Temp\\pip-build-7luaz37j\\pyq'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "d:\program\python36\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "d:\program\python36\lib\site-packages\pip\commands\install.py", line 385, in run
    requirement_set.cleanup_files()
  File "d:\program\python36\lib\site-packages\pip\utils\build.py", line 38, in __exit__
    self.cleanup()
  File "d:\program\python36\lib\site-packages\pip\utils\build.py", line 42, in cleanup
    rmtree(self.name)
  File "d:\program\python36\lib\site-packages\pip\_vendor\retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "d:\program\python36\lib\site-packages\pip\_vendor\retrying.py", line 212, in call
    raise attempt.get()
  File "d:\program\python36\lib\site-packages\pip\_vendor\retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "d:\program\python36\lib\site-packages\pip\_vendor\six.py", line 686, in reraise
    raise value
  File "d:\program\python36\lib\site-packages\pip\_vendor\retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "d:\program\python36\lib\site-packages\pip\utils\__init__.py", line 102, in rmtree
    onerror=rmtree_errorhandler)
  File "d:\program\python36\lib\shutil.py", line 494, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "d:\program\python36\lib\shutil.py", line 384, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "d:\program\python36\lib\shutil.py", line 393, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "d:\program\python36\lib\site-packages\pip\utils\__init__.py", line 114, in rmtree_errorhandler
    func(path)
PermissionError: [WinError 32] another program is using the file，process can't access.: 'C:\\Users\\hzado\\AppData\\Local\\Temp\\pip-build-7luaz37j\\pyq'

sashkab commented 6 years ago

@hzadonis It looks like is related to a bug in pip (pypa/pip#4110, fixed by pypa/pip#4486), which will eventually be released as pip 10.

Try running following command before running pip and entering your virtual environment in your Command Prompt:

chcp 65001

This will change your Command Prompt default encoding to UTF8, which is what Python 3.6 expects.

If that doesn't help, try upgrading pip to the latest development snapshot in your virtual environment and try again:

pip install -U git+https://github.com/pypa/pip

After upgrading pip, try to install PyQ again.

hzadonis commented 6 years ago

@sashkab , thanks! At my site: C:\Users\hzado>chcp Active code page: 65001

hzadonis commented 6 years ago

@sashkab And I failed to upgrade pip to V10

C:\Users\hzado>chcp
Active code page: 65001

C:\Users\hzado>D:\Program\Python36\Scripts\pip.exe install -v pyq
Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect
Config variable 'WITH_PYMALLOC' is unset, Python ABI tag may be incorrect
Collecting pyq
  1 location(s) to search for versions of pyq:
  * https://pypi.python.org/simple/pyq/
  Getting page https://pypi.python.org/simple/pyq/
  Looking up "https://pypi.python.org/simple/pyq/" in the cache
  Current age based on date: 7622
  Freshness lifetime from max-age: 600
  Freshness lifetime from request max-age: 600
  The cached response is "stale" with no etag, purging
  Starting new HTTPS connection (1): pypi.python.org
  "GET /simple/pyq/ HTTP/1.1" 200 306
  Updating cache with response from "https://pypi.python.org/simple/pyq/"
  Caching b/c date exists and max-age > 0
  Analyzing links from page https://pypi.python.org/simple/pyq/
    Found link https://pypi.python.org/packages/66/2c/d1e1891e293edd7905ac4d1d6e4f9f30ff7e40cebf8e2a2629bf4f9f4152/pyq-4.1.0.post1.tar.gz#md5=01cefe3276ac91a40856b4ab62a8dd6f (from https://pypi.python.org/simple/pyq/), version: 4.1.0.post1
    Found link https://pypi.python.org/packages/c0/a6/6dad579c65ded93d9da0fdf9e0da268870c9cea5abd6e2d0606b67f0905d/pyq-4.1.1.tar.gz#md5=ac587d42c57d3558254b90b7b98677db (from https://pypi.python.org/simple/pyq/), version: 4.1.1
  Using version 4.1.1 (newest of versions: 4.1.0.post1, 4.1.1)
  Looking up "https://pypi.python.org/packages/c0/a6/6dad579c65ded93d9da0fdf9e0da268870c9cea5abd6e2d0606b67f0905d/pyq-4.1.1.tar.gz" in the cache
  Current age based on date: 7623
  Freshness lifetime from max-age: 31557600
  The response is "fresh", returning cached response
  31557600 > 7623
  Using cached pyq-4.1.1.tar.gz
  Downloading from URL https://pypi.python.org/packages/c0/a6/6dad579c65ded93d9da0fdf9e0da268870c9cea5abd6e2d0606b67f0905d/pyq-4.1.1.tar.gz#md5=ac587d42c57d3558254b90b7b98677db (from https://pypi.python.org/simple/pyq/)
  Running setup.py (path:C:\Users\hzado\AppData\Local\Temp\pip-build-4ukhq3rg\pyq\setup.py) egg_info for package pyq
    Running command python setup.py egg_info
    running egg_info
    creating pip-egg-info\pyq.egg-info
    writing pip-egg-info\pyq.egg-info\PKG-INFO
    writing dependency_links to pip-egg-info\pyq.egg-info\dependency_links.txt
    writing requirements to pip-egg-info\pyq.egg-info\requires.txt
    writing top-level names to pip-egg-info\pyq.egg-info\top_level.txt
    writing manifest file 'pip-egg-info\pyq.egg-info\SOURCES.txt'
    reading manifest file 'pip-egg-info\pyq.egg-info\SOURCES.txt'
    writing manifest file 'pip-egg-info\pyq.egg-info\SOURCES.txt'
  Source in c:\users\hzado\appdata\local\temp\pip-build-4ukhq3rg\pyq has version 4.1.1, which satisfies requirement pyq from https://pypi.python.org/packages/c0/a6/6dad579c65ded93d9da0fdf9e0da268870c9cea5abd6e2d0606b67f0905d/pyq-4.1.1.tar.gz#md5=ac587d42c57d3558254b90b7b98677db
Building wheels for collected packages: pyq
  Running setup.py bdist_wheel for pyq ...   Destination directory: C:\Users\hzado\AppData\Local\Temp\tmpzgx_czhppip-wheel-
  Running command D:\Program\Python36\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\hzado\\AppData\\Local\\Temp\\pip-build-4ukhq3rg\\pyq\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d C:\Users\hzado\AppData\Local\Temp\tmpzgx_czhppip-wheel- --python-tag cp36
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build\lib.win-amd64-3.6
  creating build\lib.win-amd64-3.6\pyq
  copying src\pyq\cmd.py -> build\lib.win-amd64-3.6\pyq
  copying src\pyq\conftest.py -> build\lib.win-amd64-3.6\pyq
  copying src\pyq\magic.py -> build\lib.win-amd64-3.6\pyq
  copying src\pyq\ptk.py -> build\lib.win-amd64-3.6\pyq
  copying src\pyq\version.py -> build\lib.win-amd64-3.6\pyq
  copying src\pyq\_n.py -> build\lib.win-amd64-3.6\pyq
  copying src\pyq\_pt_run.py -> build\lib.win-amd64-3.6\pyq
  copying src\pyq\__init__.py -> build\lib.win-amd64-3.6\pyq
  creating build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_ci.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_com.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_k.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_magic.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_memoryview.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_n.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_no_numpy.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_numpy.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_p.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_pathlib.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_pypath.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_pyq_executable.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_python_q.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_q.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_sd.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_threads.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\test_version.py -> build\lib.win-amd64-3.6\pyq\tests
  copying src\pyq\tests\__init__.py -> build\lib.win-amd64-3.6\pyq\tests
  running build_ext
  building 'pyq._k' extension
  creating build\temp.win-amd64-3.6
  creating build\temp.win-amd64-3.6\Release
  creating build\temp.win-amd64-3.6\Release\src
  creating build\temp.win-amd64-3.6\Release\src\pyq
  D:\Program\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG -DKXVER=3 -DKXVER2=5 -ID:\Program\Python36\include -ID:\Program\Python36\include "-ID:\Program\Microsoft Visual Studio 14.0\VC\INCLUDE" "-ID:\Program\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\winrt" /Tcsrc/pyq/_k.c /Fobuild\temp.win-amd64-3.6\Release\src/pyq/_k.obj /WX
  _k.c
  D:\Program\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:D:\Program\Python36\libs /LIBPATH:D:\Program\Python36\PCbuild\amd64 "/LIBPATH:D:\Program\Microsoft Visual Studio 14.0\VC\LIB\amd64" "/LIBPATH:D:\Program\Microsoft Visual Studio 14.0\VC\ATLMFC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.14393.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.14393.0\um\x64" /EXPORT:PyInit__k build\temp.win-amd64-3.6\Release\src/pyq/_k.obj /OUT:build\lib.win-amd64-3.6\pyq\_k.cp36-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.6\Release\src/pyq\_k.cp36-win_amd64.lib src\pyq\kx\w32\q.lib
error
  Failed building wheel for pyq
  Running setup.py clean for pyq
  Running command D:\Program\Python36\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\hzado\\AppData\\Local\\Temp\\pip-build-4ukhq3rg\\pyq\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" clean --all
  running clean
  removing 'build\temp.win-amd64-3.6' (and everything under it)
  removing 'build\lib.win-amd64-3.6' (and everything under it)
  'build\bdist.win-amd64' does not exist -- can't clean it
  'build\scripts-3.6' does not exist -- can't clean it
  removing 'build'
Failed to build pyq
Installing collected packages: pyq
  Running setup.py install for pyq ...     Running command D:\Program\Python36\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\hzado\\AppData\\Local\\Temp\\pip-build-4ukhq3rg\\pyq\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\hzado\AppData\Local\Temp\pip-ftc74m7l-record\install-record.txt --single-version-externally-managed --compile
    running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-3.6
    creating build\lib.win-amd64-3.6\pyq
    copying src\pyq\cmd.py -> build\lib.win-amd64-3.6\pyq
    copying src\pyq\conftest.py -> build\lib.win-amd64-3.6\pyq
    copying src\pyq\magic.py -> build\lib.win-amd64-3.6\pyq
    copying src\pyq\ptk.py -> build\lib.win-amd64-3.6\pyq
    copying src\pyq\version.py -> build\lib.win-amd64-3.6\pyq
    copying src\pyq\_n.py -> build\lib.win-amd64-3.6\pyq
    copying src\pyq\_pt_run.py -> build\lib.win-amd64-3.6\pyq
    copying src\pyq\__init__.py -> build\lib.win-amd64-3.6\pyq
    creating build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_ci.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_com.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_k.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_magic.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_memoryview.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_n.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_no_numpy.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_numpy.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_p.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_pathlib.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_pypath.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_pyq_executable.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_python_q.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_q.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_sd.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_threads.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\test_version.py -> build\lib.win-amd64-3.6\pyq\tests
    copying src\pyq\tests\__init__.py -> build\lib.win-amd64-3.6\pyq\tests
    running build_ext
    building 'pyq._k' extension
    creating build\temp.win-amd64-3.6
    creating build\temp.win-amd64-3.6\Release
    creating build\temp.win-amd64-3.6\Release\src
    creating build\temp.win-amd64-3.6\Release\src\pyq
    D:\Program\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG -DKXVER=3 -DKXVER2=5 -ID:\Program\Python36\include -ID:\Program\Python36\include "-ID:\Program\Microsoft Visual Studio 14.0\VC\INCLUDE" "-ID:\Program\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\winrt" /Tcsrc/pyq/_k.c /Fobuild\temp.win-amd64-3.6\Release\src/pyq/_k.obj /WX
    _k.c
    D:\Program\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:D:\Program\Python36\libs /LIBPATH:D:\Program\Python36\PCbuild\amd64 "/LIBPATH:D:\Program\Microsoft Visual Studio 14.0\VC\LIB\amd64" "/LIBPATH:D:\Program\Microsoft Visual Studio 14.0\VC\ATLMFC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.14393.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.14393.0\um\x64" /EXPORT:PyInit__k build\temp.win-amd64-3.6\Release\src/pyq/_k.obj /OUT:build\lib.win-amd64-3.6\pyq\_k.cp36-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.6\Release\src/pyq\_k.cp36-win_amd64.lib src\pyq\kx\w32\q.lib
error
Cleaning up...
  Removing source in C:\Users\hzado\AppData\Local\Temp\pip-build-4ukhq3rg\pyq
Exception:
Traceback (most recent call last):
  File "D:\Program\Python36\lib\site-packages\pip-9.0.1-py3.6.egg\pip\compat\__init__.py", line 73, in console_to_str
    return s.decode(sys.__stdout__.encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 26: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Program\Python36\lib\site-packages\pip-9.0.1-py3.6.egg\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "D:\Program\Python36\lib\site-packages\pip-9.0.1-py3.6.egg\pip\commands\install.py", line 342, in run
    prefix=options.prefix_path,
  File "D:\Program\Python36\lib\site-packages\pip-9.0.1-py3.6.egg\pip\req\req_set.py", line 784, in install
    **kwargs
  File "D:\Program\Python36\lib\site-packages\pip-9.0.1-py3.6.egg\pip\req\req_install.py", line 878, in install
    spinner=spinner,
  File "D:\Program\Python36\lib\site-packages\pip-9.0.1-py3.6.egg\pip\utils\__init__.py", line 676, in call_subprocess
    line = console_to_str(proc.stdout.readline())
  File "D:\Program\Python36\lib\site-packages\pip-9.0.1-py3.6.egg\pip\compat\__init__.py", line 75, in console_to_str
    return s.decode('utf_8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 26: invalid start byte

hzadonis commented 6 years ago

Python distribution is a normal one, just installed from a Python setup file. I have an Anaconda env at the server as well, but don't use it because I always give the absolute directory of pip.

sashkab commented 6 years ago

@hzadonis The problem you're having isn't PyQ related -- this is clearly pip's failure to properly parse output from the linker. On non-English version of Windows I was able to install PyQ without a problem after setting codepage to 65001 and upgrading to pip 10. Here are my steps:

chcp 65001
cd  c:\Users\вася\Work
py36\Scripts\activate
git clone https://github.com/pypa/pip
cd pip
python setup.py install

Afterwards, I've installed PyQ without a problem.

I suggest you follow instructions on how to install PyQ into a virtual environment.

sashkab commented 6 years ago

We have uploaded pre-release wheels for PyQ 4.1.2 to PyPI. We have made available both 32 and 64 versions for Python 2.7 and Python 3.6. All published wheels require kdb+ 3.5.

Requirements:

Python 3.6 or Python 2.7.
32-bit version of kdb+ Downloadable from https://kx.com and saved as %HOMEPATH%\Downloads\windows.zip
Windows 10 (but might work on earlier versions).
Virtual Environment.

python -mvenv py36
py36\Scripts\activate.bat
set QHOME=%VIRTUAL_ENV%\q
"C:\Program Files\7-Zip\7z.exe" x -y -o%VIRTUAL_ENV% %HOMEPATH%\Downloads\windows.zip
del %QHOME%\q.q
set QBIN=%QHOME%\w32\q.exe

Now you should be able to install PyQ:

pip install --pre pyq

Finally, start PyQ:

%QBIN% python.q

Note, you will have to press ^Z and then Enter keys in order to get into Python REPL. This is known limitation of this version.

dassybr commented 6 years ago

Hi, I am using Windows 7 (64-bit), Python 3.6 (64-bit, Anaconda) and the free version of q which is 32-bit only. As far as I can tell I was able to install PyQ from the downloaded 64-bit wheel (gave me no errors) and according to the above instructions. However, when I start PyQ I get the following error in Q:

>%QBIN% python.q
KDB+ 3.5 2017.09.06 Copyright (C) 1993-2017 Kx Systems
w32/ 8()core 4095MB

'The specified module could not be found.
  [4]  C:\Users\*\py36\q\p.k:3: (`p 2:(`p_init;1))python_dll

I ran the python.q script step by step and even gave the full path to python_dll but still get the same error. it looks like there is an incompatibility with the different 32-bit and 64-bit parts? Is that the end of the road?

sashkab commented 6 years ago

Windows 7 (64-bit), Python 3.6 (64-bit, Anaconda) and the free version of q which is 32-bit only.

In order to run PyQ with the free 32-bit version of kdb+ you will need to install 32-bit Python distribution. Despite we did not test Anconda version of the Python, they do have 32-bit version, so it should be fairly easy to install it.

We recommend using Python downloaded from the python.org — 32-bit Python 3.6.3 (this is direct link on installer).

abalkin commented 6 years ago

@sashkab - I actually played with Anaconda and Miniconda a little bit. See my comment above. Other than occasional issues with the version string, I did not see any problems.

dassybr commented 6 years ago

Thanks for that. Unfortunately, that doesnt work for me because it an existing code base used by many people and based on Python 64-bit. I wanted to ship q + pyq with that code base in order to make reading large datasets from q faster.

sashkab commented 6 years ago

@dassybr you can't mix and match architectures: you have 32-bit kdb+, so it will work only with the 32-bit version of Python.

bevinahal commented 6 years ago

Facing a new error - "runtime error R6030 - CRT not initialized"

sashkab commented 6 years ago

Facing a new error - "runtime error R6030 - CRT not initialized"

Please provide steps to reproduce this, what environment, etc?

liutongwei commented 6 years ago

Hi, I'm using Windows 10 (64-bit) Chinese version, Python 3.6 (32-bit, Anaconda) and the free version of q which is 32-bit. I've successfully installed pyq and pytest.

(py3.6.32bit) C:\Users\Tongwei>pip install -U pyq
Collecting pyq
  Downloading pyq-4.1.2-cp36-cp36m-win32.whl (173kB)
    100% |████████████████████████████████| 174kB 15kB/s
Installing collected packages: pyq
Successfully installed pyq-4.1.2

(py3.6.32bit) C:\Users\Tongwei>pip install pytest pytest-pyq
Collecting pytest
  Downloading pytest-3.2.5-py2.py3-none-any.whl (188kB)
    100% |████████████████████████████████| 194kB 17kB/s
Collecting pytest-pyq
  Downloading pytest_pyq-1.1-py2.py3-none-any.whl
Collecting py>=1.4.33 (from pytest)
  Downloading py-1.5.0-py2.py3-none-any.whl (88kB)
    100% |████████████████████████████████| 92kB 14kB/s
Requirement already satisfied: setuptools in d:\anaconda3\envs\py3.6.32bit\lib\site-packages (from pytest)
Collecting colorama; sys_platform == "win32" (from pytest)
  Downloading colorama-0.3.9-py2.py3-none-any.whl
Installing collected packages: py, colorama, pytest, pytest-pyq
Successfully installed colorama-0.3.9 py-1.5.0 pytest-3.2.5 pytest-pyq-1.1

But when I run the pytest, it shows that "The system cannot find the file specified." There isn't python.q in the directory c:\q\w32.

(py3.6.32bit) C:\Users\Tongwei>c:\q\w32\q.exe python.q -mpytest --pyargs pyq < nul
KDB+ 3.5 2017.03.17 Copyright (C) 1993-2017 Kx Systems
w32/ 8()core 4095MB Tongwei liutongwei 169.254.65.45 NONEXPIRE

Welcome to kdb+ 32bit edition
For support please see http://groups.google.com/d/forum/personal-kdbplus
Tutorials can be found at http://code.kx.com/wiki/Tutorials
To exit, type \\
To remove this startup msg, edit q.q
'python.q. OS reports: 系统找不到指定的文件。
  [0]  (<load>)

  )  )

Any idea that could help to solve this question?

sashkab commented 6 years ago

@liutongwei, when using wheels on Windows, you need to follow instructions I provided in the comment above.

In particular, Q should be installed in the %VIRTUAL_ENV%\q directory, not in the c:\q.

liutongwei commented 6 years ago

@sashkab, thanks for your answer, I need to follow the instructions strictly.

liutongwei commented 6 years ago

@sashkab, finally it works after I followed the virtual environment instructions above. Thank you so much and I appreciate your time.

jljs commented 6 years ago

Where can you fine pyq-kernel.p in linux? I am unable to locate that file.

Thanks

Jeffrey

abalkin commented 6 years ago

To install pyq-kernel, please run

pip install pyq-kernel
pyq -m pyq.kernel install

(See https://pypi.org/project/pyq-kernel)

This should place pyq-kernel.p in your $QHOME or in $VIRTUAL_ENV/q if you use a virtual environment installation (as you should).

@jljs - this is not the right issue for this question. If you have further questions about pyq-kernel, please open a separate issue.

jtzharry commented 6 years ago

Wonder if anyone see this error? Windows 10 64bit KDB 32bit newest version. Could it be that the visual studio C++ build tools has some problem? Thanks!

_k.obj : warning LNK4197: export 'PyInit__k' specified multiple times; using first specification
   Creating library build\temp.win-amd64-3.6\Release\src/pyq\_k.cp36-win_amd64.lib and object build\temp.win-amd64-3.6\Release\src/pyq\_k.cp36-win_amd64.exp
_k.obj : error LNK2001: unresolved external symbol ka
_k.obj : error LNK2001: unresolved external symbol kh
_k.obj : error LNK2001: unresolved external symbol ki
_k.obj : error LNK2001: unresolved external symbol kz
_k.obj : error LNK2001: unresolved external symbol kg
_k.obj : error LNK2001: unresolved external symbol js
_k.obj : error LNK2001: unresolved external symbol sn
_k.obj : error LNK2001: unresolved external symbol kc
_k.obj : error LNK2001: unresolved external symbol ku
_k.obj : error LNK2001: unresolved external symbol krr
_k.obj : error LNK2001: unresolved external symbol sd0
_k.obj : error LNK2001: unresolved external symbol dl
_k.obj : error LNK2001: unresolved external symbol jv
_k.obj : error LNK2001: unresolved external symbol ymd
_k.obj : error LNK2001: unresolved external symbol ee
_k.obj : error LNK2001: unresolved external symbol ktj
_k.obj : error LNK2001: unresolved external symbol kb
_k.obj : error LNK2001: unresolved external symbol jk
_k.obj : error LNK2001: unresolved external symbol ke
_k.obj : error LNK2001: unresolved external symbol d9
_k.obj : error LNK2001: unresolved external symbol kj
_k.obj : error LNK2001: unresolved external symbol m9
_k.obj : error LNK2001: unresolved external symbol sd1
_k.obj : error LNK2001: unresolved external symbol dot
_k.obj : error LNK2001: unresolved external symbol r0
_k.obj : error LNK2001: unresolved external symbol ktd
_k.obj : error LNK2001: unresolved external symbol k
_k.obj : error LNK2001: unresolved external symbol r1
_k.obj : error LNK2001: unresolved external symbol setm
_k.obj : error LNK2001: unresolved external symbol ss
_k.obj : error LNK2001: unresolved external symbol b9
_k.obj : error LNK2001: unresolved external symbol dj
_k.obj : error LNK2001: unresolved external symbol xT
_k.obj : error LNK2001: unresolved external symbol ktn
_k.obj : error LNK2001: unresolved external symbol ks
_k.obj : error LNK2001: unresolved external symbol kd
_k.obj : error LNK2001: unresolved external symbol xD
_k.obj : error LNK2001: unresolved external symbol kf
_k.obj : error LNK2001: unresolved external symbol ja
_k.obj : error LNK2001: unresolved external symbol kt
_k.obj : error LNK2001: unresolved external symbol kpn
_k.obj : error LNK2001: unresolved external symbol okx
_k.obj : error LNK2001: unresolved external symbol knk
build\lib.win-amd64-3.6\pyq\_k.cp36-win_amd64.pyd : fatal error LNK1120: 43 unresolved externals
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\link.exe' failed with exit status 1120

sashkab commented 6 years ago

@jtzharry Could you please share the command you used to get this error message? Did you use virtual environment? Which version of Python did you use? Which version of kdb+ did you use (i.e. 32-bit or 64-bit)?

KxSystems / pyq

Create version of PyQ for Windows? #1

Experimental support for Windows has landed with the PyQ 4.1.0.

Installing Jupyter kernel