Ability to send and persist data to a notebook remotely

[petulla]

Is your feature request related to a problem? Please describe. I'd like to be able to send and persist data to an Observable notebook data store up to a reasonably low number, say, 10 or 100mb. For example, when training machine learning models, I'd like to be able to persist my training logs to an Observable notebook.

Describe the solution you'd like For example, say I have a training loop with a set of steps and a loss at the step. Similar to Cloudfare KV store, I'd like to be able to send the data to an Observable notebook, which can be used for storage. The notebook could then visualize streaming data from an external source. For example, tensorboard is often used locally to as a visualization tool that sits on top of logged data. I'd like to be able to use Observable notebooks instead.

Describe alternatives you've considered S3, Dynamodb etc.. are all dedicated stores. There are also many paid platforms for ML training that operate as contexts around your training and store and log runs. Observable could fit right into this mix and let the full power of its visualization/open-source ecysostem build tools for ML developers. We just need to get the data onto the platform.

I'm imaging an API to a notebook like:

{
"notebook": str,
"data": JSON,
"username": str,
"secret": str,
"cell": str
}

This request would append the data to an existing cell collection. Of course, might be a more Observable way.

[CobusT]

Wondering if Puppeteer could be used to do something like this... https://observablehq.com/@asg017/introduction-to-the-new-observable-prerender-cli

[mootari]

@CobusT Can you describe how? From what I understand this issue requests an API to attach data to a notebook.