Conda package for Artiq 2.2 broken on Windows: PyQt5 DLL load failed

[cjbe]

On a fresh Conda / Artiq installation the dashboard fails with the error:

[...]
     from PyQt5 import QtCore, QtGui, QtWidgets
ImportError: DLL load failed: The specified module could not be found.

This can be reproduced on a clean Win7 32 bit VM with the following steps:

• install Anaconda3-4.3.0.1-Windows-x86 (current Win32 Python3 Anaconda release)
• add M-Labs main channel
• install artiq-kc705-nist_clock

The package listing of this installation is here Downgrading icu from 57.1 to 56.1 (as per #566) does not fix this.

I have also seen this issue on Win7 64 bit.

[jordens]

Do you have a previous conda environment around where this was working? If yes, please provide that conda list. Also, you have a bunch of packages installed both as conda packages and through pip/setuptools. This is likely to cause (potentially unrelated) problems.

[cjbe]

@jordens all of the packages in that environment were installed by running 'conda create -n artiq-main artiq-kc705-nist_clock'.

I made a clean VM, installed conda, added the M-Labs main channel, then installed the Artiq package, so I am confident about this.

[jordens]

Please try conda install qt5=5.5.1=2 pyqt5=5.5.1=py35_0 in the non-working environment and paste the output.

Weird that you see all the pip packages. I haven't seen that before but I can reproduce it.

[cjbe]

No joy - here is the output:

(artiq-main) c:\>conda install qt5=5.5.1=2 pyqt5=5.5.1=py35_0
Fetching package metadata .............
Solving package specifications: .

Package plan for installation in environment C:\ProgramData\Anaconda3\envs\artiq-main:

The following packages will be UPDATED:

    qt5: 5.5.1-1 m-labs/label/main --> 5.5.1-2 m-labs/label/main

Proceed ([y]/n)? y

qt5-5.5.1-2.ta 100% |###############################| Time: 0:00:11   2.70 MB/s

(artiq-main) c:\>artiq_dashboard
[...]
    from PyQt5 import QtCore, QtGui, QtWidgets
ImportError: DLL load failed: The specified module could not be found.

conda list here

[jordens]

And now with conda install icu=56.1?

I am at a loss why it doesn't install qt5=5.5.1=2 right away.

[jordens]

It also seems to ignore that icu dependency in qt5=5.5.1=2.

[cjbe]

Oops - I attached the wrong conda list last post. This is the list after the last 'conda install'

As per this list icu is at 56.1, and 'conda install icu=56.1' gives 'All requested packages are already installed.' as expected

[cjbe]

@jordens any ideas on how to fix this? This is now holding us up. This issue does not seem specific to me: I have been unable to get the dashboard working on 3 Win7 machines so far, and I am aware of at least one new Artiq user that is now blocked by this.

[jordens]

@cjbe Not right now. I'd have to debug this on windows. There are the packages (pyqt, qt, icu, but start with pyqt as the top level package of that hierarchy) for qt 5.6 in standard anaconda. You can upgrade to the after installing artiq. Last I checked they work fine.

@whitequark could you help?

[sbourdeauducq]

@cjbe if the standard Anaconda Qt5 packages work (please test on Linux and Windows), could you send a patch for ARTIQ that makes it use them?

[sbourdeauducq]

(i.e. do #261)

[r-srinivas]

I tried installing 2.2 and had the same error with the dll file. I installed icu=56.1 and tried opening the dashboard but got this error instead,

(artiq-2017-02-27) C:\artiq-magtrap2.2>artiq_dashboard
Traceback (most recent call last):
  File "C:\Anaconda3\envs\artiq-2017-02-27\Scripts\artiq_dashboard-script.py", l
ine 10, in <module>
    load_entry_point('artiq==2.2', 'gui_scripts', 'artiq_dashboard')()
  File "C:\Anaconda3\envs\artiq-2017-02-27\lib\site-packages\artiq\frontend\arti
q_dashboard.py", line 107, in main
    args.server, args.port_control, "master_" + target))
  File "C:\Anaconda3\envs\artiq-2017-02-27\lib\site-packages\Quamash-0.5.5-py3.5
.egg\quamash\__init__.py", line 273, in run_until_complete
  File "C:\Anaconda3\envs\artiq-2017-02-27\lib\asyncio\futures.py", line 293, in
 result
    raise self._exception
  File "C:\Anaconda3\envs\artiq-2017-02-27\lib\asyncio\tasks.py", line 239, in _
step
    result = coro.send(None)
  File "C:\Anaconda3\envs\artiq-2017-02-27\lib\site-packages\artiq\protocols\pc_
rpc.py", line 204, in connect_rpc
    await asyncio.open_connection(host, port, limit=4*1024*1024)
  File "C:\Anaconda3\envs\artiq-2017-02-27\lib\asyncio\streams.py", line 75, in
open_connection
    lambda: protocol, host, port, **kwds)
  File "C:\Anaconda3\envs\artiq-2017-02-27\lib\asyncio\base_events.py", line 775
, in create_connection
    raise exceptions[0]
  File "C:\Anaconda3\envs\artiq-2017-02-27\lib\asyncio\base_events.py", line 762
, in create_connection
    yield from self.sock_connect(sock, address)
  File "C:\Anaconda3\envs\artiq-2017-02-27\lib\asyncio\proactor_events.py", line
 449, in sock_connect
    return self._proactor.connect(sock, address)
  File "C:\Anaconda3\envs\artiq-2017-02-27\lib\site-packages\Quamash-0.5.5-py3.5
.egg\quamash\_windows.py", line 135, in connect
  File "C:\Anaconda3\envs\artiq-2017-02-27\lib\asyncio\windows_events.py", line
506, in connect
    ov.ConnectEx(conn.fileno(), address)
OSError: [WinError 10022] An invalid argument was supplied

My conda list is,

(artiq-2017-02-27) c:\artiq-3-test>conda list
# packages in environment at C:\Anaconda3\envs\artiq-2017-02-27:
#
Using Anaconda Cloud api site https://api.anaconda.org
alabaster                 0.7.9                    py35_0
babel                     2.3.4                    py35_0
cffi                      1.9.1                    py35_0
colorama                  0.3.7                    py35_0
docutils                  0.13.1                   py35_0
h5py                      2.6.0               np112py35_2
hdf5                      1.8.15.1                 vc14_4  [vc14]
imagesize                 0.7.1                    py35_0
jinja2                    2.9.5                    py35_0
jpeg                      9b                       vc14_0  [vc14]
libpng                    1.6.27                   vc14_0  [vc14]
aiohttp                   0.17.2                     py_3    m-labs/channel/main

artiq                     2.2             py_0+gitf5cd092    m-labs/channel/main

asyncserial               0.1             py_11+git758c1e5    m-labs/channel/mai
n
binutils-or1k-linux       2.25.1                        4    m-labs/channel/main

chardet                   2.2.1                      py_3    m-labs/channel/main

dateutil                  2.4.2                      py_2    m-labs/channel/main

icu                       56.1                          0    m-labs/channel/main

levenshtein               0.12.0                   py35_1    m-labs/channel/main

libgit2                   0.24.1                        7    m-labs/channel/main

libssh2                   1.7.0.git                     5    m-labs/channel/main

lit                       0.4.1                      py_2    m-labs/channel/main

llvmlite-artiq            0.10.0.dev              py35_24    m-labs/channel/main

outputcheck               0.4.2                      py_1    m-labs/channel/main

prettytable               0.7.2                      py_3    m-labs/channel/main

pygit2                    0.24.0                   py35_4    m-labs/channel/main

pyqt5                     5.5.1                    py35_0    m-labs/channel/main

pyqtgraph                 0.9.10          py_112+git7d34e22    m-labs/channel/ma
in
pythonparser              1.0                        py_7    m-labs/channel/main

qt5                       5.5.1                         1    m-labs/channel/main

quamash                   0.5.5                      py_1    m-labs/channel/main

regex                     2015.11.22               py35_1    m-labs/channel/main

sphinx-argparse           0.1.13                     py_4    m-labs/channel/main

sphinxcontrib-wavedrom    1.1.0                      py_1    m-labs/channel/main

markupsafe                0.23                     py35_2
mkl                       2017.0.1                      0
numpy                     1.12.0                   py35_0
openssl                   1.0.2k                   vc14_0  [vc14]
pip                       9.0.1                    py35_1
pycparser                 2.17                     py35_0
pygit2                    0.24.0                    <pip>
pygments                  2.2.0                    py35_0
pyserial                  2.7                      py35_0
python                    3.5.3                         0
pytz                      2016.10                  py35_0
scipy                     0.18.1              np112py35_1
setuptools                27.2.0                   py35_1
sip                       4.18                     py35_0
six                       1.10.0                   py35_0
snowballstemmer           1.2.1                    py35_0
sphinx                    1.4.8                    py35_0
sphinx_rtd_theme          0.1.9                    py35_1
vs2015_runtime            14.0.25123                    0
wheel                     0.29.0                   py35_0
zlib                      1.2.8                    vc14_3  [vc14]

[cjbe]

@sbourdeauducq using the Anaconda default pyqt package (currently at 5.6.0) fixes this problem on Windows, and does not break anything on Linux, so I will submit a PR).

[cjbe]

@r-srinivas this is #508 / #506 appearing again when trying to connect to ::1 - this is because the money-patch is keyed to python 3.5.2, but Anaconda are now distributing 3.5.3. The money-patch still works on 3.5.3, so I will submit a PR for this.

[jordens]

needs backport to release-2

[cjbe]

@jordens could you cherry-pick this on to release-2?

[cjbe]

@jordens the only thing pulling in m-labs/pyqt5 now is quamash: the recipe is fixed, but the conda package at m-labs/main has not been rebuilt with this new recipe (current build is 2016-12-01) - could you trigger a rebuild?

[sbourdeauducq]

@cjbe Things are rarely simple. The rebuild fails (http://buildbot.m-labs.hk/builders/conda-lin64/builds/286). Also what to do about the documentation?

[cjbe]

@sbourdeauducq I see - thanks for fixing that build. There now seems to be a working quamash (0.5.5-py_2) in m-labs/dev, but m-labs/main still has 0.5.5-py_1. Is it possible to update the main as well?

[cjbe]

@sbourdeauducq the documentation relating to 3f556a3 / #361 ? I have not been able to reproduce this on Ubuntu - perhaps @r-srinivas may be able to reproduce?

[sbourdeauducq]

Is it possible to update the main as well?

Not for now because this may break 2.2 on Linux. Better make a 2.3 release with the new Qt dependencies and test it. But the release-2 branch also no longer builds on buildbot; for some reason the artiq-board builder installs the wrong version of artiq-dev and therefore migen/misoc, which causes compatibility issues. Can you figure out why?

@sbourdeauducq the documentation relating to 3f556a3 / #361 ?

Yes.

[cjbe]

@sbourdeauducq the buildbot installing artiq-dev=3.0 for the release-2 build seems to be due to a bug in conda-build. Using conda-build=1.19.2 (as on the buildbot) I can reproduce the problem. Using the latest conda-build version (2.1.5) the release-2 build works fine.

Is it possible to update the conda-build on the buildbot?

[sbourdeauducq]

@cjbe A warm welcome to the wonderful world of conda (and Python packaging in general). The build now fails because conda says it would put the package in /var/lib/buildbot/slaves/debian-stretch-amd64-1/miniconda/conda-bld/noarch, but puts it into /var/lib/buildbot/slaves/debian-stretch-amd64-1/miniconda/conda-bld/linux-64 instead. I do not know if there are other consequences to this cool new problem.

[sbourdeauducq]

Worked around that, but now the windows unittests broke. http://buildbot.m-labs.hk/builders/artiq-win64-test/builds/431/steps/python_unittest/logs/stdio

There is a reason we didn't touch conda-build before...

[sbourdeauducq]

And in addition to that, the conda garbage now leaves some folders around and later trips on them. http://buildbot.m-labs.hk/builders/artiq-board/builds/450/steps/conda_build/logs/stdio

This problem is also new: http://buildbot.m-labs.hk/builders/artiq/builds/1379/steps/conda_create/logs/stdio

[sbourdeauducq]

Is the problem still present with the 2.x packages currently in the dev anaconda channel, or do they install and run just fine?

[cjbe]

Using the nist_clock 2.2 package in dev on Win7 this problem is fixed!

[cjbe]

(Apart from the monkey patch for #506 not working)

[sbourdeauducq]

OK, thanks

[webstersc]

Installing the most recent version of artiq 2.2 on the dev channel on Windows works for me now, apart from the command line tools failing with the error 'failed to create process' - the master & dashboard have to be run using python -m artiq.frontend.xxx.

[whitequark]

I think that error is https://github.com/conan-io/conan/issues/298.

[jordens]

Deeper down, more like pypa/setuptools#398 i.e. buildbot might need updating.

[sbourdeauducq]

@webstersc Thanks for the report, but please create a new issue for new problems.