Development and testing

[aolney]

https://github.com/vatlab/jupyterlab-sos/issues/32 was originally started as an issue for jupyterlab-sos but later transformed into a discussion about developing sos-fsharp.

Linking to that issue here so others can find it in the future.

[aolney]

Quick question: how can I keep the tabs open after selenium completes the test? I've tried

chrome_options.add_experimental_option("detach", True)

and commenting out driver.quit(), which only prevents the last tab from closing.

The first tab (which contains most of the tests) reverts back to the Jupyter directory listing, so more specifically I think I need to prevent Jupyter from closing the notebook (if that's possible).

Another option would be to disable the 2nd test (the interface test) so that my current method of preventing closure works for the test suite I care about (the 1st one).

[BoPeng]

When I need more time to read the error messages and try to reproduce the errors from tests, I just add a new test that runs

import time
time.sleep(60)

This would give me enough time to read the notebook, even copy/paste the cells to a notebook not created by selenium.

[aolney]

That kind of works, thanks. Right now I'm running the test capturing output, e.g. pytest > pytest.log, and then running the test again. This lets me consult the failures from the previous run while its paused, since pytest otherwise doesn't display the errors until both tests have finished and the notebook is torn down.

[aolney]

Any thoughts on this error? The cd command runs successfully (based on the output) but the cell execution hangs. I've tried variants in F# that either return null or return the path as a string, and both hang the same way. BTW the F# plugin is hammering the notebook server with intellisense requests.

[BoPeng]

What will happen if you execute the cd_command directly in F# cell? This magic basically executes cd in sos and all subkernels with cd_command, and I do not see why it should fail if this line of code passes. Note that cd_command should not try to print the new directory as the sos kernel will print it.

[aolney]

If I execute System.Environment.CurrentDirectory <- @".." in F# it works as expected. The corresponding line of python is cd_command = 'System.Environment.CurrentDirectory <- @"{dir}"'

I'm noticing in sos_debug.log that put magic is being invoked in this notebook even though there are no put statements. Also, the put is in error (from F#, get seems to work mostly right now, but not put). Would this be causing the kernel to stall somehow?

2019-07-21 13:17:54,241 - DEBUG - MAGIC - Failed to call put_var([]) from ifsharp

[BoPeng]

The %put magic happens for each kernel switch, which is used to put all variables starting with sos to SoS kernel to allow automatic variable transfer. You can comment our your get_... function and see if this is the culprit.

[aolney]

OK, using your clarification, I made more of a MWE. The kernel execution still freezes as before when the %cd .. is the first cell executed:

Here is the log:

aolney@monkmonk:~$ cat ~/.sos/sos_debug.log 
2019-07-21 15:31:39,963 - DEBUG - Using selector: EpollSelector
2019-07-21 15:31:39,966 - DEBUG - Using selector: EpollSelector
2019-07-21 15:31:40,029 - DEBUG - KERNEL - Starting SoS Kernel version 0.19.10 with SoS 0.19.13
2019-07-21 15:31:40,031 - DEBUG - VARIABLE - Set SOS_VERSION to '0.19.13' of type str
2019-07-21 15:31:40,031 - DEBUG - VARIABLE - Set master_id to '' of type str
2019-07-21 15:31:40,031 - DEBUG - VARIABLE - Set workflow_id to '' of type str
2019-07-21 15:31:40,031 - DEBUG - VARIABLE - Set step_name to '' of type str
2019-07-21 15:31:40,031 - DEBUG - VARIABLE - Set step_id to '' of type str
2019-07-21 15:31:40,032 - DEBUG - VARIABLE - Set CONFIG to "'user_name'":"'aolney'" of type dict
2019-07-21 15:31:40,142 - DEBUG - VARIABLE - Set __host__ to 'localhost' of type str
2019-07-21 15:31:40,157 - DEBUG - KERNEL - Found registered language R
2019-07-21 15:31:40,254 - DEBUG - KERNEL - Found registered language F#
2019-07-21 15:32:25,850 - DEBUG - KERNEL - Meta info: {'path': 'manual-tests.ipynb', 'use_panel': True, 'rerun': False, 'cell_id': '5e552204174a4696b68798595d0495f5', 'cell_kernel': 'F#'}
2019-07-21 15:32:25,850 - DEBUG - KERNEL - execute: %cd ..
2019-07-21 15:32:25,851 - DEBUG - CONTROLLER - controller started 22962
2019-07-21 15:32:25,851 - DEBUG - CONTROLLER - 22962 controller master_pull: new socket of type PULL with handler 60 (1 total)
2019-07-21 15:32:25,851 - DEBUG - CONTROLLER - 22962 controller master_request: new socket of type REP with handler 63 (2 total)
2019-07-21 15:32:25,851 - DEBUG - CONTROLLER - 22962 controller backend rep: new socket of type REP with handler 66 (3 total)
2019-07-21 15:32:25,851 - DEBUG - CONTROLLER - 22962 : new socket of type PULL with handler 69 (4 total)
2019-07-21 15:32:25,852 - DEBUG - CONTROLLER - 22962 : new socket of type PULL with handler 72 (5 total)
2019-07-21 15:32:25,852 - DEBUG - CONTROLLER - 22962 : new socket of type PUSH with handler 75 (6 total)
2019-07-21 15:32:25,852 - DEBUG - CONTROLLER - Connecting sockets from 22962
2019-07-21 15:32:25,853 - DEBUG - CONTROLLER - 22962 : new socket of type PUSH with handler 84 (7 total)
2019-07-21 15:32:25,854 - DEBUG - VARIABLE - Set CONFIG to "'user_name'":"'aolney'" of type dict
2019-07-21 15:32:25,854 - DEBUG - KERNEL - Switch from SoS to F#
2019-07-21 15:32:25,855 - DEBUG - KERNEL - Starting subkernel F#
2019-07-21 15:32:25,859 - DEBUG - WORKER - START WORKER: 1 workers (of which 0 is blocking), 0 requested, 0 processed
2019-07-21 15:32:25,860 - DEBUG - CONTROLLER - create context at 22993
2019-07-21 15:32:25,861 - DEBUG - CONTROLLER - Connecting sockets from 22993
2019-07-21 15:32:25,861 - DEBUG - CONTROLLER - 22993 : new socket of type PUSH with handler 100 (8 total)
2019-07-21 15:32:25,862 - DEBUG - CONTROLLER - 22993 worker backend: new socket of type REQ with handler 102 (9 total)
2019-07-21 15:32:25,862 - DEBUG - CONTROLLER - 22993 : new socket of type PAIR with handler 104 (10 total)
2019-07-21 15:32:25,863 - DEBUG - WORKER - WORKER SoS_Worker-1 (22993) creates ports [38407]
2019-07-21 15:32:25,866 - DEBUG - WORKER - PENDING WITH PORT (38407,) AT NUM_PENDING 0: 1 workers (of which 0 is blocking), 0 requested, 0 processed
2019-07-21 15:32:26,873 - DEBUG - WORKER - PENDING WITH PORT (38407,) AT NUM_PENDING 0: 1 workers (of which 0 is blocking), 0 requested, 0 processed
2019-07-21 15:32:27,880 - DEBUG - WORKER - PENDING WITH PORT (38407,) AT NUM_PENDING 0: 1 workers (of which 0 is blocking), 0 requested, 0 processed
2019-07-21 15:32:28,313 - DEBUG - KERNEL - Loading language module for kernel F# (version 0.1.0)
2019-07-21 15:32:28,316 - DEBUG - MESSAGE - IOPUB MSG TYPE status CONTENT  {'execution_state': 'busy'}
2019-07-21 15:32:28,343 - DEBUG - MESSAGE - IOPUB MSG TYPE status CONTENT  {'execution_state': 'busy'}
2019-07-21 15:32:28,347 - DEBUG - MESSAGE - IOPUB MSG TYPE execute_input CONTENT  {'code': '\nopen System.Numerics\nopen System.Collections.Generic\n\nlet printObject x =\n    let t = x.GetType()\n    let properties = t.GetProperties()\n    printfn "-----------"\n    if t.FullName = x.ToString() then\n        printfn "%s" t.FullName\n    else\n        printfn "%s" t.FullName\n        printfn "Value: %O" x\n    properties \n    |> Array.filter( fun prop -> prop.GetIndexParameters().Length = 0 )\n    |> Array.iter (fun prop ->\n        if prop.CanRead then\n            let value = prop.GetValue(x, null)\n            printfn "%s: %O" prop.Name value\n        else\n            printfn "%s: ?" prop.Name)\n\nlet pyReprLogical obj =\n    if obj then\n        "True"\n    else\n        "False"\n        \nlet pyReprInteger obj =\n    obj.ToString()\n\nlet pyReprDouble obj =\n    if obj = nan then\n      "numpy.nan"\n    else if obj = infinity then\n      """float("inf")"""\n    else\n        obj.ToString()\n\nlet pyReprComplex (obj : Complex) =\n    let rl = obj.Real\n    let im = obj.Imaginary\n    "complex(" + rl.ToString() + "," + im.ToString() + ")" \n\nlet pyReprCharacter obj =\n    "r\\"\\"\\"" +  obj + "\\"\\"\\""\n    \nlet pyReprArray converter obj =\n    //1D without shape\n    let dataString = String.concat "," (obj |> Array.toSeq |> Seq.reduce Seq.append |> Seq.map converter) \n    //shape\n    let dimensionString =  String.concat "," ( [| for i = 0 to obj.Rank - 1 do yield obj.GetLength(i).ToString() |] )\n    "numpy.array([" + dataString + "]).reshape((" + dimensionString + "))"\n    \n\nlet rec pyRepr (obj:obj) =\n//https://stackoverflow.com/questions/7901111/f-check-if-a-value-is-an-array-of-strings-an-array-of-arrays-of-string-or-a-s\n    match obj with\n    | :? (seq<bool>[]) as a ->\n        a |> pyReprArray pyReprLogical\n    | :? (seq<int>[]) as a ->\n        a |> pyReprArray pyReprInteger\n    | :? (seq<float>[]) as a ->\n        a |> pyReprArray pyReprDouble\n    | :? (seq<Complex>[]) as a ->\n        a |> pyReprArray pyReprComplex\n    | :? (seq<string>[]) as a ->\n        a |> pyReprArray pyReprCharacter\n\n    //handles nested dictionaries\n    | :? IDictionary<obj,obj> as d ->\n        "{" + ( d |> Seq.map(fun (KeyValue(k,v)) -> (k |> pyRepr) + ":" + (v |> pyRepr) )  |> String.concat "," )  + "}"\n    //handles nested lists\n    | :? IEnumerable<IEnumerable<obj>> as s -> \n        "[" + ( s |> Seq.map pyReprN |> String.concat "," ) +  "]"\n    | :? Option<obj> as a ->\n        match a with \n        | None -> "None"\n        | Some(x) -> pyRepr x\n    | null -> "None"\n    | _ -> "\'Untransferrable variable\'"\n\nand pyReprN (obj:seq<obj>) =\n    "[" + ( obj |> Seq.map pyRepr |> String.concat "," ) + "]"\n', 'execution_count': 0}
2019-07-21 15:32:28,708 - DEBUG - MESSAGE - IOPUB MSG TYPE status CONTENT  {'execution_state': 'idle'}
2019-07-21 15:32:28,708 - DEBUG - MESSAGE - GET SHELL MSG {'header': {'msg_id': 'c191e86d-27e4-4f2f-ac15-81036db50198', 'username': 'aolney', 'session': '1d737293-ad4b9113554c2252f4417d32', 'msg_type': 'execute_reply', 'date': datetime.datetime(2019, 7, 21, 20, 32, 28, 708444, tzinfo=datetime.timezone.utc), 'version': '5.0'}, 'msg_id': 'c191e86d-27e4-4f2f-ac15-81036db50198', 'msg_type': 'execute_reply', 'parent_header': {'msg_id': '8bfdd66a-cb0a538a02e0210aaeb50512', 'username': 'aolney', 'session': '1d737293-ad4b9113554c2252f4417d32', 'msg_type': 'execute_request', 'version': '5.0'}, 'metadata': {}, 'content': {'status': 'ok', 'execution_count': 1, 'payload': [], 'user_expressions': {}}, 'buffers': []}
2019-07-21 15:32:28,712 - DEBUG - MESSAGE - IOPUB MSG TYPE status CONTENT  {'execution_state': 'busy'}
2019-07-21 15:32:28,712 - DEBUG - MESSAGE - IOPUB MSG TYPE execute_input CONTENT  {'code': 'System.Environment.CurrentDirectory <- @".."', 'execution_count': 0}
2019-07-21 15:32:28,723 - DEBUG - MESSAGE - IOPUB MSG TYPE status CONTENT  {'execution_state': 'idle'}
2019-07-21 15:32:28,723 - DEBUG - MESSAGE - GET SHELL MSG {'header': {'msg_id': '3577d647-4373-45cd-a1b9-d42b78ac0fe6', 'username': 'aolney', 'session': '1d737293-ad4b9113554c2252f4417d32', 'msg_type': 'execute_reply', 'date': datetime.datetime(2019, 7, 21, 20, 32, 28, 723848, tzinfo=datetime.timezone.utc), 'version': '5.0'}, 'msg_id': '3577d647-4373-45cd-a1b9-d42b78ac0fe6', 'msg_type': 'execute_reply', 'parent_header': {'msg_id': 'ba736792-a192e95287ce623684dfc88b', 'username': 'aolney', 'session': '1d737293-ad4b9113554c2252f4417d32', 'msg_type': 'execute_request', 'version': '5.0'}, 'metadata': {}, 'content': {'status': 'ok', 'execution_count': 1, 'payload': [], 'user_expressions': {}}, 'buffers': []}
2019-07-21 15:32:28,727 - DEBUG - MESSAGE - IOPUB MSG TYPE status CONTENT  {'execution_state': 'busy'}
2019-07-21 15:32:28,727 - DEBUG - MESSAGE - IOPUB MSG TYPE execute_input CONTENT  {'code': '', 'execution_count': 1}
2019-07-21 15:32:28,727 - DEBUG - MESSAGE - IOPUB MSG TYPE status CONTENT  {'execution_state': 'idle'}
2019-07-21 15:32:28,890 - DEBUG - WORKER - PENDING WITH PORT (38407,) AT NUM_PENDING 0: 1 workers (of which 0 is blocking), 0 requested, 0 processed
2019-07-21 15:32:29,904 - DEBUG - WORKER - PENDING WITH PORT (38407,) AT NUM_PENDING 0: 1 workers (of which 0 is blocking), 0 requested, 0 processed
2019-07-21 15:32:30,918 - DEBUG - WORKER - KILL STANDING [38407]: 0 workers (of which 0 is blocking), 0 requested, 0 processed
2019-07-21 15:32:30,918 - DEBUG - WORKER - WORKER SoS_Worker-1 (22993) quits after receiving None.
2019-07-21 15:32:30,930 - DEBUG - CONTROLLER - 22993 worker master: closes socket with handler 104 (9 left)
2019-07-21 15:32:30,930 - DEBUG - CONTROLLER - 22993 : closes socket with handler 102 (8 left)
2019-07-21 15:32:30,931 - DEBUG - CONTROLLER - 22993 : closes socket with handler 100 (7 left)
2019-07-21 15:32:30,931 - DEBUG - CONTROLLER - Disconnecting sockets from 22993
2019-07-21 15:32:30,931 - DEBUG - CONTROLLER - terminate context at 22993

[BoPeng]

OK, Let me try to reproduce and debug. Could you push all you have to the repo (or branch)? Please also let me know if you are using the latest release or current master. I have worked on widget support recently which might or might not caused the problem.

[BoPeng]

It took me a while to identify this bug. It turns out the F-Sharp kernel, unlike other kernels, does not respond to an empty input, which is actually the content of the cell after removing the magic. This should be considered a bug of the F# kernel (which should always respond for an execute_input message) but I have updated sos-notebook so that it does not send empty code to subkernels and released sos notebook 0.9.14. Please let me know if this fixes the issue.

Note that your v1 branch uses fsharp as language name while the convention for SoS is to use Capital letter for language name (e.g. R, Python3) so at first I thought that the language module was not correctly loaded...

[aolney]

Great, thanks - I've upgraded to sos-notebook 0.19.14 and the %cd command no longer hangs :thumbsup: .

Apologies for being a little slow to respond to your request to push; I had already done so at the end of the day. I've created an issue for the F# kernel problem here: https://github.com/fsprojects/IfSharp/issues/236#issue-471110807 , so hopefully the maintainers of that project will correct this problem for the future.

Re the language name, I see multiple places in the code with "kernel name" as "ifsharp", and "kernel" as "F#", but the only place I'm seeing "language" as "fsharp" is in notebooks that have the kernel loaded. Does that mean this is a kernel issue rather than an sos-fsharp issue?

[BoPeng]

There is a difference between language name (e.g. R), kernel name (e.g. ir), and displayed name (e.g. R). The relationship can be a bit complex but

1. A language name is the name of the language, with first letter capital in SoS convention (e.g. JavaScript, Octave). Used in setup.py and language module definition. (e.g. definition for R)
2. Kernel name is the name of the kernel. There can be multiple kernels for a single language (e.g. multiple kernels for matlab and javascript).
3. Display name is the name displayed in the dropdown box, which is by default the language name, but users can create alternative or multiple display names for the same language (e.g. %user R2 --language R).

So for F# the language name could be F# or Fsharp with the kernel name being ifsharp.

[aolney]

So I think the following changes meet the specification:

In setup.py:

[sos_languages]
F# = sos_fsharp.kernel:sos_Fsharp

In kernel.py

class sos_Fsharp:
    supported_kernels = {'F#': ['ifsharp']}

Happy to change as needed.

[BoPeng]

That looks good.

[zyzhu]

Yesterday. Microsoft announced a preview version of Jupyter kernel for both F# and C#. See below for two blog posts. https://devblogs.microsoft.com/dotnet/net-core-with-juypter-notebooks-is-here-preview-1/ https://devblogs.microsoft.com/dotnet/announcing-ml-net-1-4-global-availability-machine-learning-for-net/ Once it's out of preview and becomes robust, it's worth having an sos-dotnet extension.

[aolney]

Thanks for the nudge on this. I'll try to finish off the existing f# extension in Dec.

@BoPeng knows better than me, but I suspect you'd want to have separate extensions for f# and c# rather than try a single dotnet extension.

The good news is that f# and c# are so close that it would be very straightforward to convert the f# extension to c# once the f# extension is finished.

[zyzhu]

@aolney I suggest to take a fresh look at the new kernel. I think the old IfSharp kernel might phase out eventually as it hasn't supported dotnet core yet. It doesn't run on Jupyter lab properly either.

The new kernel already runs on netstandard and there are current ML.Net samples and MSFT products running on it. MSFT is also working on a dataframe module. The whole ecosystem will be supported in the new kernel.

It would be confusing to newcomers to have two F# extensions on SoS.

[BoPeng]

There is no need for two F# extension because one language module can support multiple kernels, which in this case would be ifsharp and .dot-fsharp. It makes sense for @aolney to switch to the .dot-fsharp kernel if the MS one is more robust. (sos-notebook has had problem talking to the ifsharp kernel since it does not conforms to the jupyter message specification strictly).

On the other hand separate language modules are needed for F# and C#, which i assume would be easy to implement once sos-fsharp is done.