nteract / scrapbook

A library for recording and reading data in notebooks.
https://nteract-scrapbook.readthedocs.io
BSD 3-Clause "New" or "Revised" License
281 stars 26 forks source link

Add binary data translation support #15

Closed MSeal closed 4 years ago

MSeal commented 5 years ago

Determine the best ways to support binary data or various forms.

base64 encode in document, or arrow external store, etc

rgbkrk commented 5 years ago

I'm definitely a fan of a media type that contains a reference that (visual) frontends can resolve.

  "application/vnd.nteract.scrapbook+json": {
    "ref": "s3://somewhere"
  }
MSeal commented 5 years ago

Definitely agree with the media type.

So far scrapbook is saving scraps in the format:

{
     "data": {
      "application/scrapbook.scrap+json": {
       name: value
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
}

with metadata housing display source information for snaps.

I think it'd be good to formulate an opinion on the exact payload differences for saving actual data vs references. I was thinking the metadata should differentiate if the payload is complete within data or if requires loading the ref data.

MSeal commented 4 years ago

This was done for the pandas dataframe translation support