search

GaloisInc / MATE

MATE is a suite of tools for interactive program analysis with a focus on hunting for bugs in C and C++ code using Code Property Graphs.
https://galoisinc.github.io/MATE/
BSD 3-Clause "New" or "Revised" License
177 stars 12 forks source link

doc: Make notebook tutorial into a Jupyter notebook #38

Open langston-barrett opened 2 years ago

langston-barrett commented 2 years ago

The notebook tutorial is currently a sequence of code blocks and their outputs embedded into markup. But this is exactly what Jupyter notebooks are for! We should convert the tutorial into a notebook and ship it like Usagefinder. Jupyter even has an option to export to ReST, so we could easily keep it in sync with the version in the docs.

bollwyvl commented 2 years ago

Another option is to adopt myst-nb which allows for using notebooks directly as source with (at best) a single line added to conf.py. These can either be executed at build time, or displayed "as-is" with outputs.

bollwyvl commented 2 years ago

Further on the interactive computing piece: it would be possible to use notebooks (and other sources) directly in mybinder.org, with all the tools installed. It's possible to use a Dockerfile directly, as long as it ends up with an ENTRYPOINT that starts the jupyter server. As the MATE build looks rather... involved... it might be possible to extend the existing, published image with FROM and layer in a few extra python packages, and wrap the bespoke REST service in jupyter-server-proxy with nice branding.

langston-barrett commented 2 years ago

Another option is to adopt myst-nb which allows for using notebooks directly as source with (at best) a single line added to conf.py. These can either be executed at build time, or displayed "as-is" with outputs.

That sounds like a very elegant and maintainable solution!

Further on the interactive computing piece: it would be possible to use notebooks (and other sources) directly in mybinder.org, with all the tools installed. It's possible to use a Dockerfile directly, as long as it ends up with an ENTRYPOINT that starts the jupyter server. As the MATE build looks rather... involved... it might be possible to extend the existing, published image with FROM and layer in a few extra python packages, and wrap the bespoke REST service in jupyter-server-proxy with nice branding.

Thanks for the tips! The MATE notebooks rely on the presence of several other services, including a PostgreSQL database and a MinIO object store - It seems like it would be difficult to deploy and maintain such services in third-party environments like Binder. I don't know much about it, and would be happy to hear if it can spin up such stateful services. It would be concievable (although probably quite involved and not very performant) to stuff all of our services into a single container.

thinkmoore commented 2 years ago

There's also the notebook tutorial that's included as an example when you start the MATE system: https://github.com/GaloisInc/MATE/blob/main/jupyter/examples/cpg-tutorial-no-solutions.ipynb and https://github.com/GaloisInc/MATE/blob/main/jupyter/examples/cpg-tutorial-with-solutions.ipynb. Perhaps we should point these out in the docs as well? I agree adding the notes.c example as another notebook makes sense though.

langston-barrett commented 2 years ago

There's also the notebook tutorial that's included as an example when you start the MATE system: https://github.com/GaloisInc/MATE/blob/main/jupyter/examples/cpg-tutorial-no-solutions.ipynb and https://github.com/GaloisInc/MATE/blob/main/jupyter/examples/cpg-tutorial-with-solutions.ipynb. Perhaps we should point these out in the docs as well? I agree adding the notes.c example as another notebook makes sense though.

Yes, absolutely!! I had completely forgotten about that tutorial, but it's super helpful! See #49, which also provides a template for what we could do with the Notebook tutorial.