Implement ProcessPerChunk - Githubissues

mmomtchev / node-gdal-async

Node.js bindings for GDAL (Geospatial Data Abstraction Library) with full async support

https://mmomtchev.github.io/node-gdal-async/

Apache License 2.0

133 stars 26 forks source link

Implement ProcessPerChunk #3

Open mmomtchev opened 3 years ago

mmomtchev commented 3 years ago

GDALMDArray::ProcessPerChunk is a very interesting mechanism - that calls for some very interesting solutions in JS.

A naive, sync, event-blocking solution where GDAL calls a JS callback to process the data - limited to one thread at a time blocking the event loop
A set of primitives implemented in C++ that can be pushed from JS to be run in parallel C++ in async mode
A more complex approach allowing the JS code to construct and push a pipeline to be run in parallel C++ in async mode

Anything but the very first solution is probably out of scope for 3.3

mmomtchev commented 2 years ago

This will be implemented using https://github.com/mmomtchev/exprtk.js - which allows for the most complex pipelines possible