Open lastmjs opened 7 months ago
It seems like I have found the root cause of the issue. The file reading is supposed to be batch processed using buffers of length 16 * 1024, and the algorithm seems setup for that, but the parameters to this.file.read
or very incorrect, leading to the batching algorithm not being used, so if you have a file over the original buffer size it will simply fail.
This is where the code needs to be changed: https://github.com/second-state/wasmedge-quickjs/blob/main/modules/internal/fs/stream.js#L218
Here's what I've done so far that seems to fix the issue:
const preLength = opts.end - opts.start - curPos + 1;
const { bytesRead: n } = await this.file.read(buffer, 0, preLength > buffer.byteLength ? buffer.byteLength : preLength, curPos === 0 ? opts.start : opts.start + curPos);
It seems that this code has not been well-tested...are there not Node.js or other test suites available to test against?
I haven't found a solution to this problem yet, but I've tracked the issue down to here: https://github.com/second-state/wasmedge-quickjs/blob/main/modules/internal/fs/stream.js#L175
In my Express app I'm trying to load a file that is just slightly bigger than the size of this buffer. I get this error:
I will keep trying to track down a solution, but this is a major problem.