peng1999 / typst-pyrunner

Run python code in typst
MIT License
30 stars 0 forks source link

Running third-party packages like numpy and pandas #6

Open psads-git opened 2 months ago

psads-git commented 2 months ago

I find pyrunner an extremely useful typst library. However, it has an important limitation: It cannot run third-party libraries. Package pyodide may lift this so constricting limitation (pyodide is a Python distribution for the browser and Node.js based on WebAssembly):

https://app.scribbler.live/?jsnb=github:gopi-suvanam/scribbler-examples/Python-in-Browser.jsnb

With pyodide one can run numpy, pandas, etc.

So, I would suggest to enhance pyrunner in that direction.

peng1999 commented 2 months ago

We can draw inspiration from Pyodide, but it offers limited help here. The key point is that Pyodide runs on the Emscripten target. The Emscripten runtime provides rich APIs, including FS APIs, which are essential for the CPython import process, while the Typst plugin runtime does not.

To integrate CPython into the Typst plugin system, we must either create a dummy file system and provide API to CPython or make a custom Python module loader to bypass filesystem reads. Even if we overcome this, we'll still need to address packaging wasm versions of C extensions and Typst's poor wasm performance.

That said, none of the above is impossible. However, I'm concerned if the gain is worth the effort. Anyways you can run Python code outside the Typst file and include the output in it. Can you provide a use case where pyrunner is better than alternatives if package import is implemented?

psads-git commented 2 months ago

Thank you, @peng1999, for your response.

I understand that we can run Python code outside of Typst and then import the results into Typst. However, this approach can become cumbersome as the number of code snippets increases and changes in the code snippets need to be performed. This is why I believe a library like Pyrunner could help alleviate these issues.

Quarto, for example, handles this process in a very seamless way, as demonstrated in the attached video. However, it's important to note that Quarto does not support automatic preview by design.

The scenario of creating lecture notes with integrated Python code snippets is one of the use cases.

https://github.com/user-attachments/assets/bc32bb24-8285-4a42-b270-23b424d05452

peng1999 commented 2 months ago

I don't think literate programming will be a valid use case of pyrunner. There are some unsolvable problems:

I think we should wait for Typst to have a native WASM interpreter in the browser before reconsidering the feasibility of literate programming within it.

psads-git commented 2 months ago

Thank you for your response and for explaining the challenges with adding literate programming to pyrruner. I understand the current performance and interactivity limitations within Typst, and I agree that waiting for a native WASM interpreter might be the best course of action.

I appreciate your work on pyrruner and look forward to future developments.