Execution Issues in Mojo - Local Builds

[mmcky]

@kp992 I am running into some local issues with mojo builds

/Users/mmcky/work/quantecon/project.lecture-mojo/applications/wealth_distribution.md: Executing notebook using local CWD [mystnb] 
[ErrorLog] Failed to parse the module
[ErrorLog] Expression [2]:11:57: use of unknown declaration 'Matrix'
fn random_normal_matrix(mu: Float32, sigma: Float32, M: Matrix):
                                                        ^~~~~~

[ErrorLog] Expression [2]:12:27: unexpected token in expression
    for i in range(M.rows):
                          ^

[ErrorLog] Expression [2]:12:27: statements must start at the beginning of a line
    for i in range(M.rows):

and

/Users/mmcky/work/quantecon/project.lecture-mojo/tutorial/mojo_demo.md: Executing notebook using local CWD [mystnb]               
[ErrorLog] Failed to parse the module
[ErrorLog] Expression [3]:3:5: expression must be mutable for in-place operator destination
    x += 1

[kp992]

@mmcky Can you please provide the steps to reproduce this? It would be helpful to debug.

[mmcky]

Just running

jb clean ./ --all
jb build ./

I have mojo installed along with the mojo jupyter kernel (validated)

[kp992]

Thanks @mmcky. I'll try and reproduce the same.

[kp992]

Hmm, I am unable to produce this locally, see the logs that I get by doing this locally:

Logs

```bash % jb clean ./ --all % jb build ./ Running Jupyter-Book v0.15.1 Source Folder: /Users/kpl/repos/project.lecture-mojo Config Path: /Users/kpl/repos/project.lecture-mojo/_config.yml Output Path: /Users/kpl/repos/project.lecture-mojo/_build/html Running Sphinx v5.0.2 making output directory... done [etoc] Changing master_doc to 'intro' myst v0.18.1: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions=['colon_fence', 'dollarmath', 'linkify', 'substitution', 'tasklist'], disable_syntax=[], all_links_external=False, url_schemes=['mailto', 'http', 'https'], ref_domains=None, highlight_code_blocks=True, number_code_blocks=[], title_to_header=False, heading_anchors=None, heading_slug_func=None, footnote_transition=True, words_per_minute=200, sub_delimiters=('{', '}'), linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area') myst-nb v0.17.2: NbParserConfig(custom_formats={}, metadata_key='mystnb', cell_metadata_key='mystnb', kernel_rgx_aliases={}, execution_mode='cache', execution_cache_path='', execution_excludepatterns=[], execution_timeout=600, execution_in_temp=False, execution_allow_errors=False, execution_raise_on_error=False, execution_show_tb=False, merge_streams=False, render_plugin='default', remove_code_source=False, remove_code_outputs=False, code_prompt_show='Show code cell {type}', code_prompt_hide='Hide code cell {type}', number_source_lines=False, output_stderr='show', render_text_lexer='myst-ansi', render_error_lexer='ipythontb', render_image_options={}, render_figure_options={}, render_markdown_format='commonmark', output_folder='build', append_css=True, metadata_to_fm=False) Using jupyter-cache at: /Users/kpl/repos/project.lecture-mojo/_build/.jupyter_cache building [mo]: targets for 0 po files that are out of date building [html]: targets for 6 source files that are out of date updating environment: [new config] 6 added, 0 changed, 0 removed /Users/kpl/repos/project.lecture-mojo/applications/shortest_paths.md: Executing notebook using local CWD [mystnb] [4008:36136:20231212,200253.859650:ERROR directory_reader_posix.cc:42] opendir /Users/kpl/.modular/crashdb/attachments/2c70c68f-1c92-43d0-83ed-a6f7c5307037: No such file or directory (2) /Users/kpl/repos/project.lecture-mojo/applications/wealth_distribution.md: Executing notebook using local CWD [mystnb] /Users/kpl/repos/project.lecture-mojo/examples/matmul_comparison.ipynb: Executing notebook using local CWD [mystnb] /Users/kpl/repos/project.lecture-mojo/intro.md: Executing notebook using local CWD [mystnb] /Users/kpl/repos/project.lecture-mojo/intro.md: Executed notebook in 1.01 seconds [mystnb] /Users/kpl/repos/project.lecture-mojo/intro.md: Cached executed notebook: ID=1 [mystnb] /Users/kpl/repos/project.lecture-mojo/tutorial/mojo_demo.md: Executing notebook using local CWD [mystnb] /Users/kpl/repos/project.lecture-mojo/applications/shortest_paths.md: WARNING: Executing notebook failed: CellExecutionError [mystnb.exec] /Users/kpl/repos/project.lecture-mojo/applications/shortest_paths.md: WARNING: Notebook exception traceback saved in: /Users/kpl/repos/project.lecture-mojo/_build/html/reports/applications/shortest_paths.err.log [mystnb.exec] /Users/kpl/repos/project.lecture-mojo/applications/wealth_distribution.md: WARNING: Executing notebook failed: CellExecutionError [mystnb.exec] /Users/kpl/repos/project.lecture-mojo/applications/wealth_distribution.md: WARNING: Notebook exception traceback saved in: /Users/kpl/repos/project.lecture-mojo/_build/html/reports/applications/wealth_distribution.err.log [mystnb.exec] /Users/kpl/repos/project.lecture-mojo/examples/matmul_comparison.ipynb: WARNING: Executing notebook failed: CellExecutionError [mystnb.exec] /Users/kpl/repos/project.lecture-mojo/examples/matmul_comparison.ipynb: WARNING: Notebook exception traceback saved in: /Users/kpl/repos/project.lecture-mojo/_build/html/reports/examples/matmul_comparison.err.log [mystnb.exec] /Users/kpl/repos/project.lecture-mojo/tutorial/mojo_demo.md: WARNING: Executing notebook failed: CellExecutionError [mystnb.exec] /Users/kpl/repos/project.lecture-mojo/tutorial/mojo_demo.md: WARNING: Notebook exception traceback saved in: /Users/kpl/repos/project.lecture-mojo/_build/html/reports/tutorial/mojo_demo.err.log [mystnb.exec] looking for now-outdated files... none found pickling environment... done checking consistency... /Users/kpl/repos/project.lecture-mojo/README.md: WARNING: document isn't included in any toctree /Users/kpl/repos/project.lecture-mojo/examples/matmul_comparison.ipynb: WARNING: document isn't included in any toctree done preparing documents... WARNING: unsupported theme option 'dark_logo' given done writing output... [100%] tutorial/mojo_demo /Users/kpl/repos/project.lecture-mojo/applications/shortest_paths.md:22: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/applications/shortest_paths.md:32: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/applications/shortest_paths.md:109: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/applications/shortest_paths.md:125: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/applications/shortest_paths.md:140: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/applications/shortest_paths.md:151: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/applications/shortest_paths.md:170: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/applications/wealth_distribution.md:21: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/applications/wealth_distribution.md:34: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/applications/wealth_distribution.md:54: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/applications/wealth_distribution.md:99: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/applications/wealth_distribution.md:143: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/applications/wealth_distribution.md:165: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/examples/matmul_comparison.ipynb:10002: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/examples/matmul_comparison.ipynb:20002: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/tutorial/mojo_demo.md:29: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/tutorial/mojo_demo.md:54: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/tutorial/mojo_demo.md:66: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/tutorial/mojo_demo.md:89: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/tutorial/mojo_demo.md:108: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/tutorial/mojo_demo.md:120: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/tutorial/mojo_demo.md:150: WARNING: Pygments lexer name 'mojo' is not known /Users/kpl/repos/project.lecture-mojo/tutorial/mojo_demo.md:163: WARNING: Pygments lexer name 'mojo' is not known generating indices... genindex done writing additional pages... search done copying static files... done copying extra files... done dumping search index in English (code: en)... done dumping object inventory... done [etoc] missing index.html written as redirect to 'intro.html' build succeeded, 34 warnings. The HTML pages are in _build/html. =============================================================================== Finished generating HTML for book. Your book's HTML pages are here: _build/html/ You can look at your book by opening this file in a browser: _build/html/index.html Or paste this line directly into your browser bar: file:///Users/kpl/repos/project.lecture-mojo/_build/html/index.html =============================================================================== ```

Checking out the logs for CellExecutionError on of the notebooks, I get:

CellExecutionError reason

```bash % cat /Users/kpl/repos/project.lecture-mojo/_build/html/reports/applications/shortest_paths.err.log Traceback (most recent call last): File "/Users/kpl/mambaforge/envs/qe/lib/python3.10/site-packages/jupyter_cache/executors/utils.py", line 58, in single_nb_execution executenb( File "/Users/kpl/mambaforge/envs/qe/lib/python3.10/site-packages/nbclient/client.py", line 1305, in execute return NotebookClient(nb=nb, resources=resources, km=km, **kwargs).execute() File "/Users/kpl/mambaforge/envs/qe/lib/python3.10/site-packages/jupyter_core/utils/__init__.py", line 166, in wrapped return loop.run_until_complete(inner) File "/Users/kpl/mambaforge/envs/qe/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete return future.result() File "/Users/kpl/mambaforge/envs/qe/lib/python3.10/site-packages/nbclient/client.py", line 705, in async_execute await self.async_execute_cell( File "/Users/kpl/mambaforge/envs/qe/lib/python3.10/site-packages/nbclient/client.py", line 1058, in async_execute_cell await self._check_raise_for_error(cell, cell_index, exec_reply) File "/Users/kpl/mambaforge/envs/qe/lib/python3.10/site-packages/nbclient/client.py", line 914, in _check_raise_for_error raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content) nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell: ------------------ from utils.list import Dim from memory import memset_zero from random import rand from math import min, abs from time import now ------------------ ----- stdout ----- Fontconfig warning: ignoring UTF-8: not a valid region tag ----- stderr ----- error: Execution was interrupted, reason: internal c++ exception breakpoint(-5).. The process has been left at the point where it was interrupted, use "thread return -x" to return to the state before expression evaluation. ------------------ ```

May the reason of the failure that you are getting is due to versions mis-match. I am using the following versions:

% conda list | grep jupyter 
jupyter-book              0.15.1                   pypi_0    pypi
jupyter-cache             0.6.1                    pypi_0    pypi
jupyter-client            8.3.1                    pypi_0    pypi
jupyter-core              5.3.1                    pypi_0    pypi
jupyter-events            0.7.0                    pypi_0    pypi
jupyter-server            2.7.3                    pypi_0    pypi
jupyter-server-mathjax    0.2.6                    pypi_0    pypi
jupyter-server-terminals  0.4.4                    pypi_0    pypi
jupyterlab-pygments       0.2.2                    pypi_0    pypi
sphinx-jupyterbook-latex  0.5.2                    pypi_0    pypi
sphinx-tojupyter          0.3.0                    pypi_0    pypi
% mojo --version
mojo 0.5.0 (6e50a738)

[mmcky]

@kp992 it looks like none of your notebooks are executing successfully via mojo on your machine. Is that right?

That error was associated with not being able to find the mojo binary via the jupyter kernel on my end.

[mmcky]

@kp992 I am using mojo == 0.6.0

[mmcky]

Please let me know once you get this running again on your local.

[kp992]

Please let me know once you get this running again on your local.

Do you mean to try the same with mojo == 0.6.0 or fixing the nbclient.exceptions.CellExecutionError?

[kp992]

Hi @mmcky,

I noticed a very strange issue. I converted the .md file to .ipynb using jupytext. I tried to run this notebook in Jupyter-lab. It failed at the first cell with the error shown above. I re-ran the same cell 2-3 times and it worked. After that whole notebook worked fine for me locally. Is there a configurable option in Jupyter book where we can force re-run the cells? I can show you this strange thing over a call.