search

paulroth3d / jupyter-ijavascript-utils

Utility library for working with iJavaScript - a Jupyter Kernel
1 stars 0 forks source link

Discuss writing large files #56

Open paulroth3d opened 10 months ago

paulroth3d commented 10 months ago

This seems to be tripping people up , especially with very large JSON objects like geoJSON

there are some libraries that can help with this - want to link to them here.

the simplest option however is to use file read and write streams to write out large files

Additionally the Jupyter-IJavaScript-utils works great with file streaming libraries like scramjet or other multithreaded functional streaming libraries

paulroth3d commented 10 months ago

This project handles the logic and grouping of objects, and we don't want to recreate scramjet

sometimes we want to work with large geoJSON files (or others that have very specific formats.

Describe more about how others can do the same in a documentation page.