search

guysv / ilua

Portable Lua kernel for Jupyter
GNU General Public License v2.0
115 stars 11 forks source link

Got error in jupyter lab #2

Open Freakwill opened 5 years ago

Freakwill commented 5 years ago

Got error in jupyter lab. The console works well. To my surprise, it would run well at first time, but then it always reports the following error.

My system is macOS10

lua: ...b/python3.6/site-packages/ilua/lualibs/json.lua/json.lua:389: expected argument of type string, got nil
stack traceback:
    [C]: in function 'error'
    ...b/python3.6/site-packages/ilua/lualibs/json.lua/json.lua:389: in field 'decode'
    ...Versions/3.6/lib/python3.6/site-packages/ilua/interp.lua:156: in main chunk
    [C]: in ?
guysv commented 5 years ago

Hey, thanks for the report

I don't have a mac machine close by, but I must admit I also did not fiddle with jupyter lab. Will update soon with what I find :)

guysv commented 5 years ago

Hey, could you recreate the error using

ILUA_LOG_LEVEL=debug jupyterlab --debug

and put your output here? (console & UI errors)

Freakwill commented 5 years ago

The ilua in bash works well. The console and UI always report such error after several executing. I ran it with jupyter lab --debug, but got nothing valuable, such as (I don't know how to set ILUA_LOG_LEVEL=debug)

[D 09:28:23.857 LabApp] activity on 6a280794-e3d2-4ccb-afa9-44dfe80430ca: status
[D 09:28:23.858 LabApp] activity on 6a280794-e3d2-4ccb-afa9-44dfe80430ca: execute_input
[D 09:28:23.862 LabApp] activity on 6a280794-e3d2-4ccb-afa9-44dfe80430ca: error
[D 09:28:23.865 LabApp] activity on 6a280794-e3d2-4ccb-afa9-44dfe80430ca: status
[D 09:28:23.870 LabApp] Accepting token-authenticated connection from ::1
[D 09:28:23.876 LabApp] 200 GET /api/contents/?content=1&1544664503862 (::1) 6.28ms
[D 09:28:27.759 LabApp] Accepting token-authenticated connection from ::1
[D 09:28:27.760 LabApp] 200 GET /api/sessions?1544664507757 (::1) 1.86ms
[D 09:28:27.762 LabApp] Accepting token-authenticated connection from ::1
[D 09:28:27.763 LabApp] 200 GET /api/terminals?1544664507757 (::1) 1.48ms
[D 09:28:34.254 LabApp] Accepting token-authenticated connection from ::1
guysv commented 5 years ago

ILUA_LOG_LEVEL is a environment variable that when set, will configure ILua to pump up verbose messages. prepending ILUA_LOG_LEVEL=debug to jupyter lab should pass it to ILua, and it will post log messages to the jupyter log conosole.

Anyways, I must say I tried to recreate the error on linux and failed. By looking at your error I guess that some component (the kernel, or at least the interpreter communication pipe handlers) of the program died prematurely, and there are two possible sources for that:

  1. Environment configuration error: Right now ILua is not very configurable, although it has reasonable defaults which do work on Windows and Linux. My guess is that maybe your MacOS Lua installation has some unexpected setup I failed to foresee. For this reason I need output logs from the ENTIRE Jupyter-lab session, so I could pin-point unexpected configuration.

  2. ILua uses linux APIs extensively, assuming for now that the same code will work for MacOS (being almost-unix and all). This assumption could be (and probably is) wrong, but because you reported that ILua do work in bash, I can only assume that most of the project do work as expected, so this is the weaker case.

Anyway I should work on emitting more indicative exception messages as I spot that your error could be improved to become less cryptic. I will work on that on the upcoming release, so please stay tuned!

Freakwill commented 5 years ago

Thank you. Following is the output until the cryptic error. It kept working for 19 executions, a new record.

[~] ILUA_LOG_LEVEL=debug jupyter lab
[W 17:37:14.931 LabApp] extensions are aborted，use nbserverextensions
[I 17:37:15.223 LabApp] [jupyter_nbextensions_configurator] enabled 0.4.0
[I 17:37:15.229 LabApp] JupyterLab extension loaded from /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/jupyterlab
[I 17:37:15.229 LabApp] JupyterLab application directory is /Library/Frameworks/Python.framework/Versions/3.6/share/jupyter/lab
[W 17:37:15.231 LabApp] JupyterLab server extension not enabled, manually loading...
[I 17:37:15.235 LabApp] JupyterLab extension loaded from /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/jupyterlab
[I 17:37:15.235 LabApp] JupyterLab application directory is /Library/Frameworks/Python.framework/Versions/3.6/share/jupyter/lab
[I 17:37:15.236 LabApp] launch notebooks in path: /Users/william/Folders/jupyter
[I 17:37:15.236 LabApp] running in: http://localhost:8888/?token=4d60d03c25e4fddc8a52b04b393438df56aa867330ba02b9
[I 17:37:15.236 LabApp] stop the server and close the kernels with control-c (...).
[C 17:37:15.237 LabApp]

    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:8888/?token=4d60d03c25e4fddc8a52b04b393438df56aa867330ba02b9
[I 17:37:15.502 LabApp] Accepting one-time-token-authenticated connection from ::1
[I 17:37:17.017 LabApp] Node v11.4.0

[I 17:37:17.516 LabApp] Build is up to date
[E 17:37:18.409 LabApp] Notebook JSON is invalid: Additional properties are not allowed ('execution_count' was unexpected)

    Failed validating 'additionalProperties' in error:

    On instance['cells'][4]['outputs'][0]:
    {'ename': 'n/a',
     'evalue': "(ilua):1: attempt to concatenate a nil value (global 'a')",
     'execution_count': 1,
     'output_type': 'error',
     'traceback': ["(ilua):1: attempt to concatenate a nil value (global 'a')",
                   'stack traceback:',
                   '\t(ilua):1: in main chunk',
                   "\t[C]: in function 'xpcall'",
                   '\t'
                   '...Versions/3.6/lib/python3.6/site-packages/ilua/interp.lua:70:...',
                   '\t'
                   '...Versions/3.6/lib/python3.6/site-packages/ilua/interp.lua:160...',
                   '\t[C]: in ?']}
[I 17:37:19.733 LabApp] Kernel started: 340a3dc0-a55a-47a0-ad3c-447600d6a356
[I 17:37:20.380 LabApp] Kernel started: c814bcd9-7d2a-4360-81c3-8913f6497f22
[I 17:37:20.432 LabApp] Kernel started: 8daa0f68-f4c8-4680-b0a3-ae5b50b85f03
[I 17:37:21.197 LabApp] Adapting to protocol v5.1 for kernel 340a3dc0-a55a-47a0-ad3c-447600d6a356
[I 17:37:21.198 LabApp] Adapting to protocol v5.1 for kernel 340a3dc0-a55a-47a0-ad3c-447600d6a356
[I 17:37:21.677 LabApp] Adapting to protocol v5.1 for kernel c814bcd9-7d2a-4360-81c3-8913f6497f22
[I 17:37:21.678 LabApp] Adapting to protocol v5.1 for kernel c814bcd9-7d2a-4360-81c3-8913f6497f22
2018-12-14T17:37:21+0800 [ilua.kernel.ILuaKernel#debug] Launching child lua
2018-12-14T17:37:21+0800 [ilua.proto.OutputCapture#debug] Process is running
[I 17:37:21.923 LabApp] Adapting to protocol v5.3.0 for kernel 8daa0f68-f4c8-4680-b0a3-ae5b50b85f03
[I 17:37:21.927 LabApp] Adapting to protocol v5.3.0 for kernel 8daa0f68-f4c8-4680-b0a3-ae5b50b85f03
2018-12-14T17:37:21+0800 [ilua.proto.InterpreterProtocol#debug] Interpreter connections eastablished
2018-12-14T17:37:21+0800 [ilua.kernel.ILuaKernel#debug] Lua version is 5.3
[I 17:37:21.947 LabApp] Adapting to protocol v5.3.0 for kernel 8daa0f68-f4c8-4680-b0a3-ae5b50b85f03
2018-12-14T17:37:56+0800 [ilua.proto.OutputCapture#debug] Received stdout data: "lua: ...b/python3.6/site-packages/ilua/lualibs/json.lua/json.lua:389: expected argument of type string, got nil\nstack traceback:\n\t[C]: in function 'error'\n\t...b/python3.6/site-packages/ilua/lualibs/json.lua/json.lua:389: in field 'decode'\n\t...Versions/3.6/lib/python3.6/site-packages/ilua/interp.lua:156: in main chunk\n\t[C]: in ?\n"

(local language was translated)

guysv commented 5 years ago

Interesting, working on it. One last request though: what version of jupyter-lab do you use?

could you please post the output of something like

pip freeze | grep jupyter
Freakwill commented 5 years ago

I use the latest version.

jupyter==1.0.0
jupyter-client==5.2.4
jupyter-console==6.0.0
jupyter-contrib-core==0.3.3
jupyter-contrib-nbextensions==0.5.0
jupyter-core==4.4.0
jupyter-highlight-selected-word==0.2.0
jupyter-latex-envs==1.4.6
jupyter-nbextensions-configurator==0.4.0
jupyterlab==0.35.4
jupyterlab-launcher==0.13.1
jupyterlab-server==0.2.0
Freakwill commented 5 years ago

What about your work? I also found the error in bash that was not met before. Just keep with your schedule. I am OK, I am using lua.

guysv commented 5 years ago

That is interesting, because I merely added some trivial sqlite code on executions.

Anyway, I bashed around the repo a bit and made the errors clearer. Only problem is I did not create a new release. Could you please clone the repo, follow the dev setup instructions in the README, run ilua with debug output (ILUA_LOG_LEVEL=debug) and post the new errors?

Freakwill commented 5 years ago

That is interesting, because I merely added some trivial sqlite code on executions.

Anyway, I bashed around the repo a bit and made the errors clearer. Only problem is I did not create a new release. Could you please clone the repo, follow the dev setup instructions in the README, run ilua with debug output (ILUA_LOG_LEVEL=debug) and post the new errors?

I caught it in the dev version! There is difference between dev and old versions.

lua: ...william/Programming/myGithub/ilua/ilua/ext/netstring.lua:63: assertion failed! stack traceback: [C]: in function 'assert' ...william/Programming/myGithub/ilua/ilua/ext/netstring.lua:63: in function 'ext.netstring.read' /Users/william/Programming/myGithub/ilua/ilua/interp.lua:155: in main chunk [C]: in ?

And before the output, I got

2019-01-03T16:53:08+0800 [ilua.proto.OutputCapture#debug] Received stdout data: b"lua: ...william/Programming/myGithub/ilua/ilua/ext/netstring.lua:63: assertion failed!\nstack traceback:\n\t[C]: in function 'assert'\n\t...william/Programming/myGithub/ilua/ilua/ext/netstring.lua:63: in function 'ext.netstring.read'\n\t/Users/william/Programming/myGithub/ilua/ilua/interp.lua:155: in main chunk\n\t[C]: in ?\n"

The error appears more early, after several tests.

Freakwill commented 5 years ago

@guysv I have tested the latest version. It work well in shell, but it still reported the error. The shell would also brake down if I injected a code "stream:read(1)" in netstring.lua to see what happened. but print(stream) shown that stream (in netstring.lua) is a file object.

guysv commented 5 years ago

Where did you add 'stream:read'? stream is indeed a file object (a named pipe file in our case). anyway the whole netstring part is very delicate. a single byte going to the wrong place will ruin stuff for sure.

but it still reported the error

do you mean in jupyter-lab?

Freakwill commented 5 years ago

Where did you add 'stream:read'? stream is indeed a file object (a named pipe file in our case). anyway the whole netstring part is very delicate. a single byte going to the wrong place will ruin stuff for sure.

but it still reported the error

do you mean in jupyter-lab?

in netstring.lua

HuangJunye commented 4 years ago

Hi, I am trying to use ILua in Jupyter notebook on macOS. It has the similar error as reported by @Freakwill 

lua: ...local/lib/python3.7/site-packages/ilua/ext/netstring.lua:63: assertion failed!
stack traceback:
    [C]: in function 'assert'
    ...local/lib/python3.7/site-packages/ilua/ext/netstring.lua:63: in function 'ext.netstring.read'
    /usr/local/lib/python3.7/site-packages/ilua/interp.lua:158: in main chunk
    [C]: in ?

Like @Freakwill said, ILua works perfectly in Terminal, but not on Jupyter notebook/console. I tried notebook itself and also Jupyter lab, the error message is the same.

Is there any solution for that?

HuangJunye commented 4 years ago

My Jupyter notebook/lab versions

jupyter==1.0.0
jupyter-client==5.2.4
jupyter-console==6.0.0
jupyter-core==4.4.0
jupyterlab==1.1.4
jupyterlab-server==1.0.6
guysv commented 4 years ago

@HuangJunye Hi, unfortunately it seems like this issue only reproduces on macs. As I don't have a mac machine nearby, I have no sane way to debug it myself. Feel free to investigate though.

HuangJunye commented 4 years ago

@guysv I am not sure I am competent enough to debug it. But I will try. Will update here if I figure out anything.

solosito commented 4 years ago

@guysv I am also having issues when executing commands in macOS. It happens randomly after executing a command (quite often though). For me the error happens when reading the chars of the message in netstring.lua the assertion fails. More precisely, here:

https://github.com/guysv/ilua/blob/aa3a29e276f82d4e6d44c411725a7743edb5a71c/ilua/ext/netstring.lua#L63

I guess that's because the while before polls faster than the writing can do? I am happy to provide more debug data if needed.

guysv commented 4 years ago

Good News! I finally got my hands on a macbook, expect a solution soon!