search

zed-industries / zed

Code at the speed of thought – Zed is a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.
https://zed.dev
Other
39.48k stars 2.06k forks source link

Tracking Issue: Interactive Computing with Jupyter #9778

Open rgbkrk opened 3 months ago

rgbkrk commented 3 months ago

Check for existing issues

Problem

As evidenced in https://github.com/zed-industries/zed/issues/5273, Jupyter Notebook users would love to edit notebooks. There's a need for seamless integration that supports the variety of media types in Jupyter notebooks and the interactive runtimes underneath.

Proposed Plan

The fastest path where we'll see progress along the way will be to:

In parallel, we can work on new GPUI features to support rendering custom media types that Jupyter users expect.

Steps

Runtime support

To bring Jupyter kernels (aka runtimes aka REPLs) to Rust, the https://github.com/runtimed/runtimed project has been started.

Notebook File support

Rich Media -> GPUI needs

Some of the needs for notebook and/or in-editor execution will require GPUI work. Some will be able to reuse existing components as is (div, img, markdown preview).

These are the likely mediatypes to support:

Rich Media rendering

Same list as above, with actual implementation

jianghoy commented 3 months ago

Regarding text/latex, I believe today's best option is: MathJax(the default Tex implementation in webstack) -> Rust binding of MathJax running on node(the biggest unknown, talk about it below) -> SVG (which is supported by GPUI). Given Zed has a very low level UI support it should be reasonable to implement a mix of text, md, and SVG layout.

MathJax binding

Today there's a rust crate called MathJax, however, the level of support is questionable. And I feel using a headless chrome as nodejs backend for Mathjax is a big yellow flag.
There's also a KaTex rust binding, which uses quickjs(a small and fast and almost feature complete js engine implementation by Fabrice Bellard) or wasm js as node backend. But I don't think KaTex supports outputting to SVG (correct me if I'm wrong).
Hence, the best option forward I believe would be:

  1. either raise a PR to the current MathJax rust repo and add quickjs as a backend; or create a new crate that does it;
  2. also, it would be wise to setup CICD that automatically updates MathJax version in the rust binding; one concern I have over these 3rd party wrapper is that, when upstream releases a patch, downstream may not be responsive even though the change will be minimal. And CICD could help that.

Once these are sorted out, one can start verifying the work by adding Tex parsing into markdown similar to Jupyter and see if it renders correctly. This verification may or may not ship.

vultix commented 2 months ago

What’s the plan for supporting html rendering? This is an important feature for Jupyter, and I can think of a few other use cases for wanting an embedded browser / web renderer in Zed

rgbkrk commented 2 months ago

My first pass is going to involve using table schema instead of HTML for the most important use case: viewing data frames. Long term HTML support in GPUI is a big unknown to me.

vultix commented 2 months ago

@rgbkrk I have a working proof of concept of embedding a wry webview into GPUI. It's surprisingly simple, and surprisingly performant even with ~100 small webviews in a GPUI scrolling div.

First, add wry as a dependency in your Cargo.toml:

wry = "0.39.0"

Then, add this to the impl WindowContext in crates/gpui/src/window.rs

    /// Returns a reference to window's raw_window_handle::HasWindowHandle type
    pub fn raw_window_handle(&self) -> &dyn HasWindowHandle {
        &self.window.platform_window
    }

Finally, here's how I made a simple webview. Very kludgy, but works as a proof of concept:

use std::sync::Arc;

use gpui::*;
use wry::{dpi, Rect, WebView};
use wry::dpi::LogicalSize;
use wry::raw_window_handle::HasWindowHandle;

struct WebViewTest {
    views: Vec<Arc<WebView>>,
}

impl WebViewTest {
    fn new(num_views: usize, handle: &dyn HasWindowHandle) -> Self {
        let views = (0..num_views)
            .map(|i| {
                Arc::new(
                    wry::WebViewBuilder::new_as_child(&handle)
                        .with_html(format!(
                            "<html><body>Hello, world! I'm webview {i}</body></html>"
                        ))
                        .build()
                        .unwrap(),
                )
            })
            .collect();

        Self { views }
    }
}

impl Render for WebViewTest {
    fn render(&mut self, cx: &mut ViewContext<Self>) -> impl IntoElement {
        let mut parent = div()
            .id("parent")
            .block()
            .overflow_y_scroll()
            .size_full()
            .bg(rgb(0xff0000))
            .justify_center()
            .items_center();

        for (i, view) in self.views.iter().enumerate() {
            parent = parent.child(
                div()
                    .size(Length::Definite(DefiniteLength::Absolute(
                        AbsoluteLength::Pixels(Pixels(100.0)),
                    )))
                    .bg(rgb(0x00ff00))
                    .child(format!("This is webview {}:", i)),
            );
            parent = parent.child(HelloWorldEl { view: view.clone() });
        }

        parent
    }
}

struct HelloWorldEl {
    view: Arc<WebView>,
}
impl IntoElement for HelloWorldEl {
    type Element = HelloWorldEl;

    fn into_element(self) -> Self::Element {
        self
    }
}

impl Element for HelloWorldEl {
    type BeforeLayout = ();
    type AfterLayout = ();

    fn before_layout(&mut self, cx: &mut ElementContext) -> (LayoutId, Self::BeforeLayout) {
        let mut style = Style::default();
        style.flex_grow = 1.0;
        style.size = Size::full();
        let id = cx.request_layout(&style, []);
        (id, ())
    }

    fn after_layout(
        &mut self,
        bounds: Bounds<Pixels>,
        before_layout: &mut Self::BeforeLayout,
        cx: &mut ElementContext,
    ) -> Self::AfterLayout {
        // TODO: Find better way of detecting view visibility
        if bounds.top() > cx.viewport_size().height || bounds.bottom() < Pixels::ZERO {
            self.view.set_visible(false).unwrap();
        } else {
            self.view.set_visible(true).unwrap();

            self.view
                .set_bounds(Rect {
                    size: dpi::Size::Logical(LogicalSize {
                        width: (bounds.size.width.0 - 50.0).into(),
                        height: (bounds.size.height.0 / 2.0).into(),
                    }),
                    position: dpi::Position::Logical(dpi::LogicalPosition::new(
                        bounds.origin.x.into(),
                        bounds.origin.y.into(),
                    )),
                })
                .unwrap();
        }
    }

    fn paint(
        &mut self,
        bounds: Bounds<Pixels>,
        before_layout: &mut Self::BeforeLayout,
        after_layout: &mut Self::AfterLayout,
        cx: &mut ElementContext,
    ) {
        // Do nothing?
    }
}

fn main() {
    App::new().run(|cx: &mut AppContext| {
        let bounds = Bounds::centered(None, size(px(600.0), px(600.0)), cx);
        cx.open_window(
            WindowOptions {
                bounds: Some(bounds),
                ..Default::default()
            },
            |cx| {
                cx.activate_window();

                let view = WebViewTest::new(50, cx.raw_window_handle());
                cx.new_view(|_cx| view)
            },
        );
    });
}
altaic commented 1 month ago

Wondering if anyone has any experience with marimo (GitHub repo)? It looks really nice, with some benefits over Jupyter as discussed here.