Open asg017 opened 3 years ago
Implementation notes:
On every new top.ojs
change, the dataflow dev server will have to parse the code and find every import
for local files, then watch those files with chokidar. When an imported file is update, then some websocket message should be pushed to the client, where the client then updates the specific imported/injected cells.
Problems:
a.ojs has been updated
?a.ojs
imports from sub_a.ojs
, and if sub_a.ojs
gets updated, would we want to update everyything? my guy says yes, but that means we'd have to find every imported .ojs
file downstream and watch all of them. Which probably wont be too big of a perf issue, since its not like you're importing 100+ notebooks at a time.My two cents:
it would be awesome to only update the cells where the text content was actually changed (one of the features Observable does not provide and I immediately started to use is indenting + collapsing parts of file, but currently this causes recompile and resets the state of things like UI inputs)
I'd love to hear more about this! Currently, Dataflow should only update cells that have had their contents changed, so if you changed a notebook file like this:
a = 1
-b = 2
+b = 7
c = a + b
x = 3
y = 4
z = x + y
Dataflow will detect that b=2
is outdated and delete that definition, and will only run b=7
, causing only b and c to update (meaning a, x, y, and z stay the same). But if you're noticing that you only changed 1 cell's definition but others are updating erroneously, that sounds like a bug!
also, I didn't even think about nested imports! So if your notebook tree looks like this:
top.ojs
\_ suba1.ojs
\_ subb1.ojs
\_ subb2.ojs
\_ suba2.ojs
\_ suba3.ojs
When running dataflow run top.ojs
, then changing suba1.ojs
should cause an update. Changing subb1.ojs
should also cause an update, since that changes suba1.ojs
, which I didn't consider before... Should definitely be do-able, many bundlers do this already, but definitely adds a layer of complexity to this.
I'd love to hear more about this! So the file looks like
c = a d = b
a = 1 b = 2
Then I add a "heading" (a comment) and indent lines below so the file looks like this:
c = a d = b
// various utilities a = 1 b = 2
And after I indent, the cells `a` and `b` (and accordingly `c` and `d` too) get recalculated, though obviously their values end up the same as before, only the leading whitespace has changed
![image](https://user-images.githubusercontent.com/424619/118376738-8390da00-b5d2-11eb-880e-a40d94405b2e.png)
Re: nested imports: yeah :) another potentially tricky case to consider in Observable's javascript (and the one I really wish was available in regular ES modules) to consider is import ... with ...
where you can override some cells of the notebook you're importing (and I assume in this case different imports of the same notebook should create different cell instances, parameterized by whatever was in the corresponding with
clauses)
Say you have
top.ojs
:a.ojs
:b.ojs
:If you're running
dataflow run top.ojs
, then the rendered output will showa=100, b=200
as expected. Iftop.ojs
gets updated, then those changes will be reflected as well.But, if
a.ojs
orb.ojs
gets updated, then the changes do not immediately get reflected. You would have to refresh the entire page in order to fetch the latest version of the compileda.ojs
orb.ojs
file.This is currently the same case for editor and imports at observablehq.com, but since dataflow works with the filesystem, it should be easier to implement this.
So, when running
dataflow run top.ojs
, anda.ojs
is edited toa = 400
, then the live rendered notebook should reload the import and the new400
value should appear.