Cannot run commands

[torbonde]

I have just installed iF# as described in the readme. I then run the ipython notebook --profile ifsharp command, which starts up the server. At this point, all seems to be working fine. But when I open the Feature Notebook, the following is written in the console, and nothing really seems to work in the notebook itself.

2014-10-20 19:46:39.091 [NotebookApp] WARNING | Notebook /Feature Notebook.ipynb is not trusted
2014-10-20 19:46:39.400 [NotebookApp] ERROR | Unhandled error in API request
Traceback (most recent call last):
  File "/Users/torbonde/anaconda/lib/python2.7/site-packages/IPython/html/base/handlers.py", line 336, in wrapper
    result = method(self, *args, **kwargs)
  File "/Users/torbonde/anaconda/lib/python2.7/site-packages/IPython/html/services/sessions/handlers.py", line 65, in post
    kernel_id = km.start_kernel(path=path)
  File "/Users/torbonde/anaconda/lib/python2.7/site-packages/IPython/html/services/kernels/kernelmanager.py", line 90, in start_kernel
    kernel_id = super(MappingKernelManager, self).start_kernel(**kwargs)
  File "/Users/torbonde/anaconda/lib/python2.7/site-packages/IPython/kernel/multikernelmanager.py", line 116, in start_kernel
    km.start_kernel(**kwargs)
  File "/Users/torbonde/anaconda/lib/python2.7/site-packages/IPython/kernel/manager.py", line 217, in start_kernel
    **kw)
  File "/Users/torbonde/anaconda/lib/python2.7/site-packages/IPython/kernel/manager.py", line 173, in _launch_kernel
    return launch_kernel(kernel_cmd, **kw)
  File "/Users/torbonde/anaconda/lib/python2.7/site-packages/IPython/kernel/launcher.py", line 260, in launch_kernel
    stdin=_stdin, stdout=_stdout, stderr=_stderr, cwd=cwd, env=os.environ)
  File "/Users/torbonde/anaconda/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/Users/torbonde/anaconda/lib/python2.7/subprocess.py", line 1327, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
ERROR:tornado.access:{
  "Origin": "http://localhost:8888",
  "Content-Length": "56",
  "Accept-Language": "da-dk",
  "Accept-Encoding": "gzip, deflate",
  "Connection": "keep-alive",
  "Accept": "application/json, text/javascript, */*; q=0.01",
  "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10) AppleWebKit/600.1.25 (KHTML, like Gecko) Version/8.0 Safari/600.1.25",
  "Host": "localhost:8888",
  "Referer": "http://localhost:8888/notebooks/Feature%20Notebook.ipynb",
  "X-Requested-With": "XMLHttpRequest",
  "Content-Type": "application/x-www-form-urlencoded; charset=UTF-8"
}
ERROR:tornado.access:500 POST /api/sessions (::1) 37.50ms referer=http://localhost:8888/notebooks/Feature%20Notebook.ipynb
WARNING:tornado.access:404 GET /static/components/codemirror/mode/fsharp/fsharp.js (::1) 1.02ms referer=http://localhost:8888/notebooks/Feature%20Notebook.ipynb

Can anyone tell me what is going on? I should probably tell you that I am running OS X. I have a feeling this might cause the problems.

[prosconi]

OSError: [Errno 2] No such file or directory

Inside the ipython_config.py the line

c.KernelManager.kernel_cmd = [r"C:\Program Files\BayardRock\IFSharp\ifsharp.exe", "{connection_file}"]

Should point to the executable on your system

I wouldn't expect it to work on Mono / OS X anyways since I currently don't have a way of testing in that environment.

[torbonde]

Ok. The kernel_cmd points to the executable, so that's not the problem.

I have gotten some of the way, to having it running in OS X. It is still not running, but at least I get another error now. Specifically, I get a System.DllNotFoundException, complaining about libzmq. If you by chance know what could be the issue here, I would of course like to know. Otherwise, I'll keep searching. If I ever get it running, I will let you know.

[torbonde]

I think I've managed to fix the exception. Now I got another error, which I think you might be more able to fix than the others. It occurs right when I open a notebook. Hope you're able to help, since I feel I might be nearing and end.

2014-10-26 11:23:34.261 [NotebookApp] CRITICAL | Malformed message: ['<IDS|MSG>', '', '{"msg_id":"225b9db5-9b44-42ec-a860-18e26aee1437","username":"username","session":"2DF8C5FAE15C497181A2060EDF89AB30","msg_type":"kernel_info_reply"}', '{"msg_id":"B14685468A9B4B019CB74EB19A3D23E7","username":"username","session":"2DF8C5FAE15C497181A2060EDF89AB30","msg_type":"kernel_info_request"}', '{}', '{"protocol_version":[4,0],"ipython_version":null,"language_version":[1,0,0],"language":"fsharp"}']
Traceback (most recent call last):
  File "/Users/torbonde/anaconda/lib/python2.7/site-packages/IPython/html/base/zmqhandlers.py", line 118, in _on_zmq_reply
    msg = self._reserialize_reply(msg_list)
  File "/Users/torbonde/anaconda/lib/python2.7/site-packages/IPython/html/base/zmqhandlers.py", line 101, in _reserialize_reply
    msg = self.session.unserialize(msg_list)
  File "/Users/torbonde/anaconda/lib/python2.7/site-packages/IPython/kernel/zmq/session.py", line 821, in unserialize
    raise ValueError("Unsigned Message")
ValueError: Unsigned Message

[prosconi]

How did you fix the problem from before?

Are you using a session key in the ipython_config.py file?

[torbonde]

Yeah, that's a good question. I removed all traces of the libzmq.dll file you included, and then installed zeromq using homebrew. As far as I recall, that did the trick. The thing is, during my search for a solution to the new problem, I fucked some thing up, and went straight back to the missing dll-error. And now I'm stuck.

No, I don't use a sessions key. Should I be? My ipython_config.py file contains the following

c = get_config()
c.KernelManager.kernel_cmd = ["mono", "/path/to/ifsharp.exe", "{connection_file}"]

[prosconi]

No, ifsharp doesn't support that feature.

Add these two lines to the bottom of the file

c.Session.key = ''
c.Session.keyfile = ''

The DLL stuff makes sense since that is a native Windows DLL. I will look into switching to a .NET implementation of ZMQ or including the OS X binary

[torbonde]

I'm getting closer. Now I don't get any errors, and can compute simple things like 1+2. Also, IntelliSense is working. And

open System.Collections.Generic
let arr = [1. .. 100.]
Seq.average arr

also gives me the expected result. I haven't checked anything else than that. Anyway - IT'S ALIVE!

[EDIT] I have just tried a few more things. LaTeX works perfectly. Nuget doesn't work. #N "Deedle"gives me NuGet error: Unable to find package 'Deedle'. Also, charting doesn't work.

let xs = [| 0. .. 0.01 .. System.Math.PI |] 
let ys = xs |> Array.map sin
let pairs = (xs, ys) ||> Array.zip
Chart.Line pairs

gives me

System.ArgumentNullException: Argument cannot be null.
Parameter name: key
  at System.Collections.Generic.Dictionary`2[System.Object,System.Boolean].ContainsKey (System.Object key) [0x00000] in <filename unknown>:0 
  at FSharp.Charting.ChartTypes.layoutSubCharts@791 (SeriesChartType chartType, System.Collections.Generic.Dictionary`2 visited, Microsoft.FSharp.Core.FSharpOption`1 targetParent, System.Object target) [0x00000] in <filename unknown>:0 
  at FSharp.Charting.ChartTypes.layoutSubCharts@791 (SeriesChartType chartType, System.Collections.Generic.Dictionary`2 visited, Microsoft.FSharp.Core.FSharpOption`1 targetParent, System.Object target) [0x00000] in <filename unknown>:0 
  at FSharp.Charting.ChartTypes+layoutSubCharts@791-1.Invoke (Microsoft.FSharp.Core.FSharpOption`1 targetParent, System.Object target) [0x00000] in <filename unknown>:0 
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@3028[System.Object,Microsoft.FSharp.Core.Unit,Microsoft.FSharp.Core.FSharpOption`1[System.Object]].Invoke (System.Object u) [0x00000] in <filename unknown>:0 
  at FSharp.Charting.ChartTypes+applyPropertyDefaults@803-1[System.Windows.Forms.DataVisualization.Charting.ChartArea].Invoke (System.Windows.Forms.DataVisualization.Charting.ChartArea arg10) [0x00000] in <filename unknown>:0 
  at Microsoft.FSharp.Core.FSharpFunc`2[Microsoft.FSharp.Core.FSharpOption`1[System.Object],System.Windows.Forms.DataVisualization.Charting.ChartArea].InvokeFast[Unit] (Microsoft.FSharp.Core.FSharpFunc`2 func, Microsoft.FSharp.Core.FSharpOption`1 arg1, System.Windows.Forms.DataVisualization.Charting.ChartArea arg2) [0x00000] in <filename unknown>:0 
  at FSharp.Charting.ChartTypes.applyPropertyDefaults[ChartArea] (SeriesChartType chartType, System.Windows.Forms.DataVisualization.Charting.ChartArea target) [0x00000] in <filename unknown>:0 
  at FSharp.Charting.ChartTypes+GenericChart..ctor (SeriesChartType chartType) [0x00000] in <filename unknown>:0 
  at FSharp.Charting.Chart.Line[Double,Double,IEnumerable`1] (IEnumerable`1 data, Microsoft.FSharp.Core.FSharpOption`1 Name, Microsoft.FSharp.Core.FSharpOption`1 Title, Microsoft.FSharp.Core.FSharpOption`1 Labels, Microsoft.FSharp.Core.FSharpOption`1 Color, Microsoft.FSharp.Core.FSharpOption`1 XTitle, Microsoft.FSharp.Core.FSharpOption`1 YTitle) [0x00000] in <filename unknown>:0 
  at <StartupCode$FSI_0012>.$FSI_0012.main@ () [0x00000] in <filename unknown>:0 
  at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <filename unknown>:0 
Stopped due to error

Just keeping you updated.:)

[prosconi]

OK, apparently FSharp.Charting supports non-Windows via GTK. Currently the default script does not automatically reference some libraries for GTK. I am looking at this:

https://github.com/fsharp/FSharp.Charting/blob/master/src/FSharp.Charting.Gtk.fsx

[torbonde]

Cool. I'm very interested in hearing more, whenever you have made progress.:)

Do you have any idea on where to go with the NuGet-error?

[prosconi]

Did you change DefaultNuGetSource in ifsharp.exe.config? If the value is not set, it defaults to nuget.org

[torbonde]

Nope, I didn't. Didn't know I had to. I will check if it is already set.

Den 09/11/2014 kl. 19.10 skrev Peter Rosconi notifications@github.com:

Did you change DefaultNuGetSource in ifsharp.exe.config? If the value is not set, it defaults to nuget.org

— Reply to this email directly or view it on GitHub.

[torbonde]

Ok, so I tried setting the DefaultNuGetSource to https://www.myget.org/F/nuget;https://www.nuget.org/api/v2, as is commented out in the config-file. I also tried just setting it to https://nuget.org. Both of them gives me the error NuGet error: The requested feature is not implemented.

[cgravill]

We're trying to do this on Ubuntu and also have the basic repl functioning. On the GTK issue it looks like we need to switch to the FSharp.Charting.Gtk.dll instead of FSharp.Charting.dll. Doing that gets further but ChartControl isn't implemented there:

/IfSharp/src/IfSharp.Kernel/Util.fs(76,38): error FS0039: The type 'ChartControl' is not defined

This is used to convert the chart to an image in the kernel:

use ms = new MemoryStream()
let actualChart = ctl.Controls.[0] :?> System.Windows.Forms.DataVisualization.Charting.Chart
actualChart.Dock <- DockStyle.None
actualChart.Size <- Size(width, height)
actualChart.SaveImage(ms, ImageFormat.Png)
ms.ToArray()

It seems like this is possible on the GTK side. I can have a go, is it something that could be integrated?

[mndrake]

@torbonde or @cgravill do either of you have a fork that you could share of the mono version you have been working on? I would love to help test and contribute. Hopefully this can either be a branch of this project or #if mono directives could be added to the main project once it is working.

Thanks,

Dave

[mndrake]

It looks like NetMQ (C# implementation of 0qm) might be an option to make this cross-platform. I was trying to rewrite IfSharp.Kernel, but haven't had the time to figure out what the following is doing and what the equivalent in NetMQ is.

    for ident in envelope.Identifiers do
        socket <~| (encode ident) |> ignore

    socket
        <~| (encode "<IDS|MSG>")
        <~| (encode "")
        <~| (encode (serialize header))
        <~| (encode (serialize envelope.Header))
        <~| (encode "{}")
        <<| (encode (serialize content))

[prosconi]

@mndrake The operator <~| is the equivalent of sendMore and the <<| operator is the equivalent of send. I created two new issues #32 and #33.

[cgravill]

@mndrake we didn't make any changes to the IfSharp project to get the basics working following the Manual Installation. Have you got the ifsharp.exe console app compiled? The FileNotFoundException on running that is the first fix needed, @torbonde apparently replaced the dll and dependencies entirely from homebrew. On Ubuntu we copied in the relevant .so files from the python distribution and got it working that way. That should then function for straight F# code if you set the kernel_cmd configuration as above and launch the notebook.

The charting will need some additional code to interface with the GTK version of F# Charting, I'll add that to #33 when I get it working.

[mndrake]

I have tried on both OS X 10.10.1 Mono 3.12.0 and Ubuntu 14.04 Mono 3.12.0 and am able to find the libzmq.* (dylib/so) file.

However I get the same error below on both when I try to launch IfSharp. I am able to build and run on Windows.

I did find this issue can happen if fszmq is used outside of windows. https://github.com/zeromq/fszmq/issues/41

@cgravill and @torbonde there were no additional changes you needed to do in order to get IfSharp to work with mono?

[ERROR] FATAL UNHANDLED EXCEPTION: System.Runtime.InteropServices.MarshalDirectiveException: string marshalling conversion 35 not implemented
  at (wrapper managed-to-native) fszmq.C:zmq_bind (intptr,string)
  at fszmq.SocketModule.Bind (fszmq.Socket socket, System.String address) [0x00000] in <filename unknown>:0 
  at IfSharp.Kernel.App.Start$cont@228 (System.String[] args, Microsoft.FSharp.Core.Unit unitVar) [0x00000] in <filename unknown>:0 
  at IfSharp.Kernel.App.Start (System.String[] args) [0x00000] in <filename unknown>:0 
  at IfSharpConsole.Program.Main (System.String[] args) [0x00000] in <filename unknown>:0 
2015-01-24 17:01:51.721 [NotebookApp] KernelRestarter: restarting kernel (3/5)
WARNING:root:kernel 09d4a489-f1a2-4a4b-bb46-483659851b0d restarted

[mndrake]

I was able to get it to working (in OS X, haven't tried Ubuntu yet) by rebuilding the 3.2.7 branch of fszmq and replacing all instances of UnmanagedType.AnsiBStr with UnmanagedType.LPStr in the Native.fs as suggested by @pblasucci, and then just replacing the fszmq.dll library in the IfSharp bin folder. I would have just used the master branch, but methods have been removed/renamed and would require changes to IfSharp.Kernel.

[pblasucci]

@mndrake Out of curiosity, have you tried using the 4.0.x branch of fszmq? It should have addressed the LPStr issue. Although you will need to build it with --define:BSD_EAGAIN in order to run it correctly on OS X (because even agreeing on a simple error code is too much for the major OS developers).

[mndrake]

@pblasucci I was able to use the 4.0.x branch successfully building locally as well as using the latest from nuget. I'm assuming that the latest nuget package was not built with the --define:BSD_EAGAIN flag. Even though it appears to be working with the nuget .dll what is the issue with not using the compiler flag on OS X?

@cgravill I was able to get a basic chart example working after adding and tweaking FSharp.Charting.Gtk and posted an example result in a gist (http://nbviewer.ipython.org/gist/mndrake/5d00eceea245704e8cf1). I was glad to get get something working and will post a fork when I am able to clean it up a bit. I had to create a temp file to write a png to and then read it back as a byte array, since I could not find a OxyPlot method that would write to a memory stream.

[pblasucci]

@mndrake That's terrific news! The main issue is Windows and Linux use the same value for the EAGAIN error code. However, most popular Unix systems define a different value for the same error. It's possible your code hasn't ran into this (it's definitely tucked into a small but important subset of possible scenarios). At any rate, I'm glad you got things working. Well done!

[mndrake]

Here's a fork that is working for me for OS X. I had to create a custom FSharp.Charting.Gtk fork since I was not able to get the Model internal field exposed via reflection. I will try again later, since I would prefer to just have a ToPng method in IfSharp.Kernel instead of creating one in Sharp.Charting.Gtk. Figured "get it working" then others can help with the "get it right".

https://github.com/mndrake/IfSharp/tree/mono_mac https://github.com/mndrake/FSharp.Charting

It even works in QT console!

[cgravill]

That looks excellent. I was having some issues with panics on interacting with GTK. I'll try working from clean and your branch to see if we can get it running on Ubuntu too.

[mndrake]

@cgravill I was able to get the following branch working on Ubuntu 14.04. I included the libzmq files so hopefully it will work by just restoring packages.

https://github.com/mndrake/IfSharp/tree/mono_linux

[cgravill]

I think these issues have been combined and resolved in the Jupyter branch. I'll close this but feel free to open another focused issue if there are remaining issues with the Jupyter branch.