Closed zdila closed 1 year ago
I confirm the crash, it is due to the fact that you are trying to read more than 4GB of data in a single operation. gdal-async
uses Node.js NAN, the older Node.js API, which does not accept 64 bit integers. This means that anything over 4GB will have to use a floating point buffer size which is obviously not without risks. If I am able to come up with a safe implementation, I will extend the buffer limit to 2^53 - 1
, JS max safe integer, or otherwise I will add a safety check at 4GB.
TypedArray
s in Node.js seem to be limited to 4G elements:
$ node -e "new Uint8Array(66000 * 66000)"
[eval]:1
new Uint8Array(66000 * 66000)
^
RangeError: Invalid typed array length: 4356000000
at new Uint8Array (<anonymous>)
at [eval]:1:1
at Script.runInThisContext (node:vm:129:12)
at Object.runInThisContext (node:vm:307:38)
at node:internal/process/execution:79:19
at [eval]-wrapper:6:22
at evalScript (node:internal/process/execution:78:60)
at node:internal/main/eval_string:28:3
Node.js v18.16.0
The underlying V8 ArrayBuffer
has recently been extended to handle more than 4GB of data, but at the moment JS arrays are limited to 4G elements. So the hard limit for the amount of data retrieved in a single read or write operation will be 4G values for now - which depending on the element type might be more than 4GB on recent Node.js versions.
Hello,
I am getting Segmentation fault when using this library with https://github.com/FreemapSlovakia/dem-iron/tree/master and
DTM Croatia 20m v1 by Sonny.tif
from https://drive.google.com/drive/folders/1U152Bg0G1au_74ppTkZ3uNBdG-bMCXgNI am running it on Debian 12 and Node 18 with gdal-async of the latest version on npm. Also tried older versions of Node and the library.
With Node 16 and gdal-async 3.1 I am getting