search

stisa / jupyternim

A Jupyter kernel for nim
https://stisa.space/jupyternim/
MIT License
163 stars 14 forks source link

build failed on Windows 10 #17

Closed dave-doty closed 4 years ago

dave-doty commented 4 years ago

I tried the instructions for installing by cloning the repo (after nimba install jupyternim failed with a similar error).

I'm not sure what to do with this error.

(base) PS C:\git> git clone https://github.com/stisa/jupyternim
Cloning into 'jupyternim'...
remote: Enumerating objects: 77, done.
remote: Counting objects: 100% (77/77), done.
remote: Compressing objects: 100% (48/48), done.
remote: Total 1199 (delta 36), reused 64 (delta 27), pack-reused 1122
Receiving objects: 100% (1199/1199), 976.88 KiB | 5.01 MiB/s, done.
Resolving deltas: 100% (628/628), done.

(base) PS C:\git> cd .\jupyternim\

(base) PS C:\git\jupyternim> nimble.exe install
  Verifying dependencies for jupyternim@0.5.8
      Info: Dependency on zmq@>= 0.3.1 already satisfied
  Verifying dependencies for zmq@0.3.1
      Info: Dependency on hmac@any version already satisfied
  Verifying dependencies for hmac@0.1.9
      Info: Dependency on nimSHA2@any version already satisfied
  Verifying dependencies for nimSHA2@0.1.1
      Info: Dependency on sha1@any version already satisfied
  Verifying dependencies for sha1@1.1
      Info: Dependency on nimSHA2@any version already satisfied
  Verifying dependencies for nimSHA2@0.1.1
      Info: Dependency on nimPNG@any version already satisfied
  Verifying dependencies for nimPNG@0.3.1
 Installing jupyternim@0.5.8
   Building jupyternim/jupyternim.exe using c backend
       Tip: 14 messages have been suppressed, use --verbose to show them.
     Error: Build failed for package: jupyternim
        ... Details:
        ... Execution failed with exit code 1
        ... Command: "C:\programs\nim-1.2.6\bin\nim.exe" c --noNimblePath -d:NimblePkgVersion=0.5.8 -d:release --path:"C:\Users\pexat\.nimble\pkgs\zmq-0.3.1"  --path:"C:\Users\pexat\.nimble\pkgs\hmac-0.1.9"  --path:"C:\Users\pexat\.nimble\pkgs\nimSHA2-0.1.1"  --path:"C:\Users\pexat\.nimble\pkgs\sha1-1.1"  --path:"C:\Users\pexat\.nimble\pkgs\nimSHA2-0.1.1"  --path:"C:\Users\pexat\.nimble\pkgs\nimPNG-0.3.1"  -o:"C:\git\jupyternim\jupyternim.exe" "C:\git\jupyternim\src\jupyternim.nim"
        ... Output: Hint: used config file 'C:\programs\nim-1.2.6\config\nim.cfg' [Conf]
        ... Hint: system [Processing]
        ... Hint: widestrs [Processing]
        ... Hint: io [Processing]
        ... Hint: jupyternim [Processing]
        ... Hint: sockets [Processing]
        ... Hint: zmq [Processing]
        ... Hint: json [Processing]
        ... Hint: hashes [Processing]
        ... Hint: tables [Processing]
        ... Hint: math [Processing]
        ... Hint: bitops [Processing]
        ... Hint: macros [Processing]
        ... Hint: algorithm [Processing]
        ... Hint: strutils [Processing]
        ... Hint: parseutils [Processing]
        ... Hint: unicode [Processing]
        ... Hint: lexbase [Processing]
        ... Hint: streams [Processing]
        ... Hint: parsejson [Processing]
        ... Hint: options [Processing]
        ... Hint: typetraits [Processing]
        ... Hint: os [Processing]
        ... Hint: pathnorm [Processing]
        ... Hint: osseps [Processing]
        ... Hint: winlean [Processing]
        ... Hint: dynlib [Processing]
        ... Hint: times [Processing]
        ... Hint: time_t [Processing]
        ... Hint: osproc [Processing]
        ... Hint: strtabs [Processing]
        ... Hint: cpuinfo [Processing]
        ... Hint: base64 [Processing]
        ... Hint: messages [Processing]
        ... Hint: hmac [Processing]
        ... Hint: md5 [Processing]
        ... Hint: sha1 [Processing]
        ... Hint: nimSHA2 [Processing]
        ... Hint: endians [Processing]
        ... Hint: utils [Processing]
        ... Hint: random [Processing]
        ... Hint: display [Processing]
        ... Hint: mimetypes [Processing]
        ... Hint: nimPNG [Processing]
        ... Hint: buffer [Processing]
        ... Hint: sequtils [Processing]
        ... Hint: nimz [Processing]
        ... Hint: filters [Processing]
        ... Hint: results [Processing]
        ... C:\git\jupyternim\src\jupyternimpkg\sockets.nim(136, 24) Warning: Number of spaces around '/' is not consistent [Spacing]
        ... C:\git\jupyternim\src\jupyternimpkg\sockets.nim(113, 6) Hint: 'startCodeServer' is declared but not used [XDeclaredButNotUsed]
        ... C:\git\jupyternim\src\jupyternimpkg\sockets.nim(393, 6) Hint: 'parseNimsuggest' is declared but not used [XDeclaredButNotUsed]
        ... C:\git\jupyternim\src\jupyternimpkg\sockets.nim(184, 7) Hint: 'MagicsStrings' is declared but not used [XDeclaredButNotUsed]
        ... C:\git\jupyternim\src\jupyternimpkg\sockets.nim(73, 7) Hint: 'codeserver' is declared but not used [XDeclaredButNotUsed]
        ... C:\git\jupyternim\src\jupyternim.nim(2, 21) Error: cannot open file: std/exitprocs

I did download the zeromq files (https://zeromq.org/download/) and put them on my PATH as in the instructions.

stisa commented 4 years ago

Thanks for opening an issue!

Yes, the problem is a change in nim, due to changes in addQuitProc/addExitProc. I "fixed" the build in jupyternim 0.6.0, nimble install should run on nim 1.2.0 up now (proper cleanup of sockets/temporary files on exit will unfortunately only work on nim 1.3.5 and above) and also added some CI to build against more nim versions so I can avoid breaking it again in the future.

Let me know if this works for you.

dave-doty commented 4 years ago

I got further:

(base) PS C:\git> nimble install https://github.com/stisa/jupyternim -y
Downloading https://github.com/stisa/jupyternim using git
  Verifying dependencies for jupyternim@0.6.0
      Info: Dependency on zmq@>= 0.3.1 already satisfied
  Verifying dependencies for zmq@0.3.1
      Info: Dependency on hmac@any version already satisfied
  Verifying dependencies for hmac@0.1.9
      Info: Dependency on nimSHA2@any version already satisfied
  Verifying dependencies for nimSHA2@0.1.1
      Info: Dependency on sha1@any version already satisfied
  Verifying dependencies for sha1@1.1
      Info: Dependency on nimSHA2@any version already satisfied
  Verifying dependencies for nimSHA2@0.1.1
      Info: Dependency on nimPNG@any version already satisfied
  Verifying dependencies for nimPNG@0.3.1
 Installing jupyternim@0.6.0
   Building jupyternim/jupyternim.exe using c backend
    Prompt: jupyternim@0.6.0 already exists. Overwrite? -> [forced yes]
    Prompt: Missing directory C:\Users\pexat\.nimble\pkgs\jupyternim-0.6.0\src\jupyternimpkg. Continue? -> [forced yes]
    Prompt: Missing directory C:\Users\pexat\.nimble\pkgs\jupyternim-0.6.0\src\jupyternimspec. Continue? -> [forced yes]
   Success: jupyternim installed successfully.
could not load: zmq.dll
stack trace: (most recent call last)
C:\Users\pexat\AppData\Local\Temp\nimblecache\nimscriptapi.nim(192, 28)
C:\Users\pexat\.nimble\pkgs\jupyternim-0.6.0\jupyternim_22640.nims(31, 9) installAfter
C:\programs\nim-1.2.6\lib\system\nimscript.nim(260, 7) exec
C:\programs\nim-1.2.6\lib\system\nimscript.nim(260, 7) Error: unhandled exception: FAILED: C:\Users\pexat\.nimble\pkgs\jupyternim-0.6.0\jupyternim.exe [OSError]
       Tip: 31 messages have been suppressed, use --verbose to show them.
     Error: Exception raised during nimble script execution

It can't seem to find zmq.dll.

I downloaded the latest ZeroMQ files here: https://zeromq.org/download/ (Visual Studio 15 2017 x64), and put them in a directory, and added that directory to my PATH. But the file zmq.dll is not contained in that zip file. I'm not sure how to get it.

Perhaps I need to do something else to "install" ZeroMQ? The ZeroMQ download page was not very detailed about how to get it "installed".

stisa commented 4 years ago

Renaming the .dll you have in path to zmq.dll should be enough to make nim-zmq find it ( eg, from libzmq-v141-mt-4_3_2.dll to zmq.dll ).

Afterwards, since it looks like jupyternim itself got installed, you should be able to just run jupyternim to install the kernelspec, assuming you have .nimble/bin in your path, or just run C:\Users\pexat\.nimble\pkgs\jupyternim-0.6.0\jupyternim.exe . You should see something like [Jupyternim] Installing Jupyter Nim Kernel and some other installation log.

And that should be all, you should be able to select "Nim" when creating a new notebook from jupyter etc.

Sorry about this, I thought I had some code in place to fail more gracefully when zmq is not found but it looks like it needs more work.

dave-doty commented 4 years ago

Sorry I'm having so much trouble with this.

I copied libzmq-v141-mt-4_3_2.dll to zmq.dll as you suggested.

The installation looks like it succeeded:

(base) PS C:\git> nimble install https://github.com/stisa/jupyternim -y
Downloading https://github.com/stisa/jupyternim using git
  Verifying dependencies for jupyternim@0.6.0
      Info: Dependency on zmq@>= 0.3.1 already satisfied
  Verifying dependencies for zmq@0.3.1
      Info: Dependency on hmac@any version already satisfied
  Verifying dependencies for hmac@0.1.9
      Info: Dependency on nimSHA2@any version already satisfied
  Verifying dependencies for nimSHA2@0.1.1
      Info: Dependency on sha1@any version already satisfied
  Verifying dependencies for sha1@1.1
      Info: Dependency on nimSHA2@any version already satisfied
  Verifying dependencies for nimSHA2@0.1.1
      Info: Dependency on nimPNG@any version already satisfied
  Verifying dependencies for nimPNG@0.3.1
 Installing jupyternim@0.6.0
   Building jupyternim/jupyternim.exe using c backend
    Prompt: jupyternim@0.6.0 already exists. Overwrite? -> [forced yes]
    Prompt: Missing directory C:\Users\pexat\.nimble\pkgs\jupyternim-0.6.0\src\jupyternimpkg. Continue? -> [forced yes]
    Prompt: Missing directory C:\Users\pexat\.nimble\pkgs\jupyternim-0.6.0\src\jupyternimspec. Continue? -> [forced yes]
   Success: jupyternim installed successfully.
[Jupyternim] Installing Jupyter Nim Kernel
[Jupyternim] Copying Jupyternim kernelspec to C:\Users\pexat\AppData\Roaming\jupyter\kernels\jupyternimspec
[Jupyternim] Nim kernel registered, you can now try it in `jupyter lab`
[Jupyternim] Found zmq library: true

But jupyter lab doesn't seem to work:

(base) PS C:\git> jupyter lab
Traceback (most recent call last):
  File "C:\Users\pexat\Anaconda3\lib\site-packages\zmq\__init__.py", line 43, in <module>
    from zmq.utils import initthreads # initialize threads
ImportError: DLL load failed: The specified module could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\pexat\Anaconda3\lib\site-packages\jupyterlab_server\server.py", line 20, in <module>
    from notebook.notebookapp import aliases, flags, NotebookApp as ServerApp
  File "C:\Users\pexat\Anaconda3\lib\site-packages\notebook\notebookapp.py", line 49, in <module>
    from zmq.eventloop import ioloop
  File "C:\Users\pexat\Anaconda3\lib\site-packages\zmq\__init__.py", line 45, in <module>
    raise ImportError("%s\nAre you trying to `import zmq` from the pyzmq source dir?" % e)
ImportError: DLL load failed: The specified module could not be found.
Are you trying to `import zmq` from the pyzmq source dir?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\pexat\Anaconda3\Scripts\jupyter-lab-script.py", line 5, in <module>
    from jupyterlab.labapp import main
  File "C:\Users\pexat\Anaconda3\lib\site-packages\jupyterlab\labapp.py", line 14, in <module>
    from jupyterlab_server import slugify, WORKSPACE_EXTENSION
  File "C:\Users\pexat\Anaconda3\lib\site-packages\jupyterlab_server\__init__.py", line 4, in <module>
    from .app import LabServerApp
  File "C:\Users\pexat\Anaconda3\lib\site-packages\jupyterlab_server\app.py", line 9, in <module>
    from .server import ServerApp
  File "C:\Users\pexat\Anaconda3\lib\site-packages\jupyterlab_server\server.py", line 26, in <module>
    from jupyter_server.base.handlers import (                          # noqa
ModuleNotFoundError: No module named 'jupyter_server'

I'm not sure exactly what "jupyter lab" is, so I tried running jupyter notebook (how I normally open Python notebooks), and get a similar error:

(base) PS C:\git> jupyter notebook
Traceback (most recent call last):
  File "C:\Users\pexat\Anaconda3\lib\site-packages\zmq\__init__.py", line 43, in <module>
    from zmq.utils import initthreads # initialize threads
ImportError: DLL load failed: The specified module could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\pexat\Anaconda3\Scripts\jupyter-notebook-script.py", line 6, in <module>
    from notebook.notebookapp import main
  File "C:\Users\pexat\Anaconda3\lib\site-packages\notebook\notebookapp.py", line 49, in <module>
    from zmq.eventloop import ioloop
  File "C:\Users\pexat\Anaconda3\lib\site-packages\zmq\__init__.py", line 45, in <module>
    raise ImportError("%s\nAre you trying to `import zmq` from the pyzmq source dir?" % e)
ImportError: DLL load failed: The specified module could not be found.
Are you trying to `import zmq` from the pyzmq source dir?

Seems like maybe the jupyternim installation script was able to find zmq.dll, but jupyter doesn't know where to look?

stisa commented 4 years ago

Sorry I'm having so much trouble with this.

Don't worry, it's good to have someone else try to use this, otherwise I'd never find those bugs.

I'm not sure exactly what "jupyter lab" is

Jupyter Lab is sort of like a new interface for jupyter, ipython, jupyter notebook, jupyter lab, etc are all frontends for the jupyter protocol, see here for more information. The kernel is meant to work on any jupyter-compatible frontend though, so I'll change that message to be more generic.

The crash you get is... interesting. It looks like it's failing at opening jupyter notebook itself. Usually, jupyter install it's own copy of zmq, so that shouldn't happen, I think. Are you able to run python notebooks without errors? If you can't, and you used to be able to, you could try to remove the new zmq from path to see if that's throwing off python, and if that's the case, I'll need to figure out how to make nim-zmq find the right dll.

dave-doty commented 4 years ago

Are you able to run python notebooks without errors?

Not anymore. That error is in response to just running jupyter notebook at the command line (how I normally would start jupyter to run Python notebooks).

stisa commented 4 years ago

That sucks, I'm sorry.
Try removing zmq folder from path and see if that fixes jupyter notebook for python, once that work we can figure out where that is getting the zmq.dll from and try to point nim to that path. Jupyternim itself doesn't touch anything that jupyter uses, the only thing it does is copying a jupyternimspec folder to getEnv("APPDATA") / "jupyter" / "kernels" / "jupyternimspec", so the change in path is the only thing I can of that could mess up your jupyter installation.

dave-doty commented 4 years ago

That sucks, I'm sorry. Try removing zmq folder from path and see if that fixes jupyter notebook for python, once that work we can figure out where that is getting the zmq.dll from and try to point nim to that path. Jupyternim itself doesn't touch anything that jupyter uses, the only thing it does is copying a jupyternimspec folder to getEnv("APPDATA") / "jupyter" / "kernels" / "jupyternimspec", so the change in path is the only thing I can of that could mess up your jupyter installation.

That didn't work.

If you look at the error:

(base) PS C:\git> jupyter notebook
Traceback (most recent call last):
  File "C:\Users\pexat\Anaconda3\lib\site-packages\zmq\__init__.py", line 43, in <module>
    from zmq.utils import initthreads # initialize threads
ImportError: DLL load failed: The specified module could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\pexat\Anaconda3\Scripts\jupyter-notebook-script.py", line 6, in <module>
    from notebook.notebookapp import main
  File "C:\Users\pexat\Anaconda3\lib\site-packages\notebook\notebookapp.py", line 49, in <module>
    from zmq.eventloop import ioloop
  File "C:\Users\pexat\Anaconda3\lib\site-packages\zmq\__init__.py", line 45, in <module>
    raise ImportError("%s\nAre you trying to `import zmq` from the pyzmq source dir?" % e)
ImportError: DLL load failed: The specified module could not be found.
Are you trying to `import zmq` from the pyzmq source dir?

In particular this line:

  File "C:\Users\pexat\Anaconda3\lib\site-packages\zmq\__init__.py", line 45, in <module>
    raise ImportError("%s\nAre you trying to `import zmq` from the pyzmq source dir?" % e)

I may have tried installing pyzmq (or something like that) using conda at some point in an attempt to get things working.

Long story short, I think I was getting these errors because I had some python programs running, and attempting to update conda packages while they were running wasn't working. I shut down all python instances, executed conda update --all, and now I'm able to run jupyter notebook and open a Nim notebook:

image

Thanks!

stisa commented 4 years ago

Great, I'm glad you got it to work. I'd appreciate any feedback if you end up using it!

I have a pr open to make nim-zmq also accept libzmq.dll for the dll name, which is the name of the zmq lib that jupyter usually installs, hopefully that should simplify installation for people already running jupyter notebooks.

stisa commented 4 years ago

Solved, closing