denoland / deno

A modern runtime for JavaScript and TypeScript.
https://deno.com
MIT License
94.02k stars 5.23k forks source link

jupyter deno - autoreload equivalent to python ? #24660

Open oscar6echo opened 2 months ago

oscar6echo commented 2 months ago

As a heavy user of jupyter (mostly with python kernels) I would like to outline the large benefit of the autoreload IPython extension and suggest that such a feature could be added/documented in deno jupyter.

Why the benefit ?
It allows a Jupyter user to edit and iterate on a Python class in an IDE (say VS Code) and manipulate instances of such class in a Jupyter notebook which acts as a "control center". Upon change in a method, it can be run immediately in the notebook without kernel restart or even class recreation. This is a massive benefit when dealing with states that are long to reach (e.g. long downloads or computations). Additionally this workflow leverages each tool qualities: VS Code with all the linting/formatting/code completion/etc which jupyter cannot match, and jupyter for its step by step execution with visible cell outputs.
(On the other hand working entirely in a notebook does not scale)

Difference with Python:
Python is interpreted while typescript is transpiled. So I guess that the exact same feature is maybe not possible (?)


Tentative implementation:
However by implementing the class in such a way that methods can be replaced after instantiation and leveraging this comment by @nayeemrmn in issue #6946 , I could achieve a decently satisfactory workflow.

See repo ocar6echo/deno-jupyter-autoreload. The code is short enough to be pasted here.

My questions are :

I would be grateful for your view on the topic.


// util.ts

const buildAutoreload = () => {
    let i = 0;

    const import2 = async (path: string) => {
        const module = await import(`${path}#${i++}`);
        return module;
    };

    return import2;
};

const import2 = buildAutoreload();

export { import2 };
// sample.ts

type Operation = (arg0: number, arg1: number) => number;

interface Process {
    add: Operation;
    sub: Operation;
    mul: Operation;
    div: Operation;
    custom: Operation;
}

const add = (x: number, y: number) => (x || 0) + y;
const sub = (x: number, y: number) => (x || 0) - y;
const mul = (x: number, y: number) => (x || 1) * y;
const div = (x: number, y: number) => (x || 1) / y;

class CalculatorV2 {
    current: number;
    process: Process;

    constructor() {
        this.current = 0;
        this.process = {
            add,
            sub,
            mul,
            div,
            custom: (x: number, _y: number) => x,
        };
    }

    setCustom(custom: Operation) {
        this.process.custom = custom;
    }

    show() {
        console.log(`current: ${this.current}`);
    }

    add(x: number) {
        this.current = this.process.add(this.current, x);
        return this.current;
    }

    sub(x: number) {
        this.current = this.process.sub(this.current, x);
        return this.current;
    }
    mul(x: number) {
        this.current = this.process.mul(this.current, x);
        return this.current;
    }
    div(x: number) {
        this.current = this.process.div(this.current, x);
        return this.current;
    }
    custom(x: number) {
        this.current = this.process.custom(this.current, x);
        return this.current;
    }
}

export { CalculatorV2};
// custom.ts

const customA = (x: number, y: number) => x + 1 + (y || 0);
const customB = (x: number, y: number) => x + 10 + (y || 0);
const customC = (x: number, y: number) => x + 100 + (y || 0);
// const customC = (x: number, y: number) => x + 1000 + (y || 0);

export { customA, customB, customC };

image

bartlomieju commented 1 month ago

I guess you're basically asking for HMR (hot module replacement) for the Jupyter kernel. I'm open to accepting PRs that would implement it, but I'm gonna shout out to @rgbkrk for opinions on this one.

rgbkrk commented 1 month ago

As long as it's possible in Deno/V8, I'd start with making a Deno centric API like Deno.reload(module). It's not something on the Jupyter protocol, it's a language and/or runtime feature.

oscar6echo commented 1 month ago

I guess you're basically asking for HMR (hot module replacement) for the Jupyter kernel.

Roughly yes. But HMR as I understand it, maybe wrongly, is a full reset.
So for the avoidance of doubt. please let me be specific: Here the objective is to keep the state unchanged while updating the code run for the next cells executed (for a full reset a kernel restart does the job). This is already possible in the example I give but maybe there is a better, more generic, less verbose way to do it, that would benefit deno and its users.