LinusU
commented
6 years ago Would absolutely love to see a PR that adds this using a small WASM module 😍
Open josephrocca opened 6 years ago
LinusU
commented
6 years ago Would absolutely love to see a PR that adds this using a small WASM module 😍
josephrocca
commented
6 years ago Here's a working example using @kenchris's repo as a starting point: https://gist.github.com/josephrocca/a2ec179ef24462c7c21bd494d98a8988
And here's one with the wasm file inlined: https://gist.github.com/josephrocca/3d26325f1b76b3b10cb5e7c402c6dfd8
I'm new to all this wasm stuff so it may not be the best code, but it's a start! 👍
LinusU
commented
6 years ago I might be wrong, but from what I can tell, that code uses the asm.js version instead of the WASM one.
It should probably be one binary webp.wasm file, together with some small code like this:
const fs = require('fs')
const path = require('path')
const code = fs.readFileSync(path.join(__dirname, 'webp.wasm'))
const module = new WebAssembly.Module(code)
const instance = new WebAssembly.Instance(module, {})
exports.decode = function (input) {
// Allocate memory to hand over the input data to WASM
const inputPointer = instance.exports.allocate(input.byteLength)
const targetView = new Uint8Array(instance.exports.memory, inputPointer, input.byteLength)
// Copy input data into WASM readable memory
targetView.set(input)
// Decode input data into
const metadataPointer = instance.exports.decode(inputPointer, input.byteLength)
// Free the input data in WASM land
instance.exports.free(inputPointer)
// Read returned metadata
const metadata = new Uint32Array(instance.exports.memory, metadataPointer, 3)
const [width, height, outputPointer] = metadata
// Create an empty buffer for the resulting data
const outputSize = (width * height * 4)
const output = new Uint8Array(outputSize)
// Copy decoded data from WASM memory to JS
output.set(new Uint8Array(instance.exports.memory, outputPointer + 4, outputSize))
// Free WASM copy of decoded data
instance.exports.free(outputPointer)
// Return decoded image as raw data
return { width, height, data: output }
}edit: added support for returning width & height
kenchris
commented
6 years ago LinusU
commented
6 years ago Yeah, I was referring to the Gist made by @josephrocca ☺️
I'm going to try and plug your compiled wasm into my glue code and see if I can get it working 🙌
josephrocca
commented
6 years ago The gists I posted are just rearranged versions of kenchris's code so it works in node. I don't know if the webp.c file would support the standalone stuff, but it might: https://github.com/kripken/emscripten/wiki/WebAssembly-Standalone
In any case, looking forward to testing out webp stuff with node-canvas!
LinusU
commented
6 years ago Hmm, the Emscripten shim is 30 KiB 😅 would be nice to not use that.
I didn't know that you had to provide your own malloc and free 🤔
Going to try and write some glue using wasmception and see how it goes 😄
edit: super much work in progress: https://github.com/LinusU/js-webp
asturur
commented
6 years ago but why not the official C lib? wouldn't be more straight forward?
LinusU
commented
6 years ago The advantage would be that we could add a small .wasm file which will work in all versions of Node.js on all platforms (x86, arm, etc) without the end user having to compile anything. Seeing how many installation problems we have now, I would love to see this for gif, jpeg, etc. as well, potentially even cairo & pango.
It is the official C lib that I'm compiling btw. ☺️ so it should support all webp files and behave exactly as the normal lib
asturur
commented
6 years ago can you share some compilation detail? i need the webp muxer and encoder!
kenchris
commented
6 years ago 
zbjornson
commented
6 years ago Once the WASM version is working, I'd love to benchmark it against the C version. I'm happy to make the binding for that.
kenchris
commented
6 years ago Remember, wasm will be faster with time. Also future versions will support SIMD and threads, which emscripten then need to build for
asturur
commented
6 years ago Also that would make node canvas browserifiable? i mean transitioning all C to wasm. ( cairo included )
zbjornson
commented
6 years ago @asturur I suppose, but it seems redundant given that node-canvas exists to emulate the Canvas API that's already in browsers.
asturur
commented
6 years ago well i would never do that. But for people really caring about the same output OR just because till now to people that asked me how to browserify canvas i said that it could not be done.
I was just exploring.
LinusU
commented
6 years ago Okay, a status update, I'm soooo close to getting it working. It can decode some webp files currently, but I'm running into problems with YUV-images. To be honest I'm not exactly sure where the problem lies, but I've filed a bug report on libwebp here:
https://bugs.chromium.org/p/webp/issues/detail?id=403
The current code can be found here:
https://github.com/LinusU/cwasm-webp
I have published the first version of it on npm 🎉
Currently, only Node.js is supported, but browser support should be easy to add. Just want to research how to best do the loading. Also, only decoding is supported, but it should be trivial to add encoding as well.
I would absolutely love some help in tracking down the memory out of bounds issue ❤️
LinusU
commented
4 years ago Update! During the weekend I wrote a Node.js compatible Image class with support for png, gif, bmp, jpeg & webp!
It uses WebAssembly to decode all the image formats 🎉
@canvas/image - https://github.com/node-gfx/image
Should be trivial to start using that Image class here, started trying it out but ran out of time, hope to look at it soon again :)
chearon
commented
4 years ago Amazing work!! Looks like you were able to get different C libs compiled with the latest LLVM's WASM support?
I've actually been experimenting with this too. In an unrelated project, I'm working on a JS implementation of (parts of) Pango. It also relies on some WebAssembly-compiled C libraries. If we were to use it in node-canvas that would be another native dependency gone, plus it would get us perfect font matching.
It would be interesting to experiment with a fork that only uses Cairo/pixman. I'm going to bet though that performance will suffer enough that we might still need to maintain the native version somehow 😑
LinusU
commented
4 years ago Looks like you were able to get different C libs compiled with the latest LLVM's WASM support?
Yeah 🙌 the WASI SDK made it really easy to get going!
I was thinking of trying out to do the same with Cairo and see if I could get a nice @cwasm/cairo with a somewhat nice, but still low level, api
I don't know if there is a way to link a bunch of different wasm files together into one file, it might be expensive to go thru JavaScript every time the different part needs to talk to each other (on the other hand, maybe it doesn't matter that much, since it will probably only be when loading in new images 🤔)
It's going to be very interesting to see how the performance will be. Even if it's not stellar, it would be cool if we could have the natively compiled parts as an optionalDependency so that npm will try to install it, but if it fails everything will still work, just not as fast.
Also, potentially it's faster to call between JavaScript and WASM then JavaScript and the C++ functions. I don't have any data to back this up 😁, but I think that JS and WASM can be JIT compiled into the same VM, and then the calls are very cheap. This could potentially be a win when calling into Cairo which is mostly calling functions with a few integers and it does a lot of pixel manipulation internally. Anyhow, the only way to find out is to try 😄
Jytesh
commented
3 years ago Is this issue fixed, how do I import remote webp images from urls to node canvas to use the ctx.drawImage(webp_image) function
Xetera
commented
3 years ago The @canvas/image package doesn't work for me as the libraries I'm using, namely face-api.js have their own image instanceof Image prototype checks that fail when passing in the custom Image class over the canvas image. It's very frustrating to have spent all this time converting images to webp only to find that I probably need to convert them back to jpg before I can do anything with them in node.
LinusU
commented
3 years ago @Xetera if they are doing instanceof Image than it probably wouldn't work with the Image from node-canvas either? 🤔 It seems like that library is intended to be run inside a browser?
Xetera
commented
3 years ago After playing around with it for a bit I realized face-api.js has a monkeyPatch function that allows me to pass a custom Image implementation but node-canvas still doesn't seem to like that very much. It fails with TypeError: Image or Canvas expected. I feel like this has something to do with the check in
if (Nan::New(Image::constructor)->HasInstance(obj)) { ... }Where node-canvas internally relies on its own definition of Image. This behavior is reproducible with just node-canvas and no external libraries using
const imageResponse = await axios.get(image.rawUrl, {
responseType: "arraybuffer",
});
const ca = canvas.createCanvas(500, 500);
const ctx = ca.getContext("2d");
const imageElem = await imageFromBuffer(imageResponse.data);
ctx.drawImage(imageElem, 0, 0, image.width, image.height);I wanted to keep the conversation related to the solution here but I'm happy with moving over to the @canvas/image repo if you like
EDIT: I tried patching the node-canvas instance check to see if that's the issue but I didn't have much luck. I either get a black canvas output or runtime errors.
EDIT2: I was able to solve my issue by combining @canvas/image with @tensorflow/tfjs-node without canvas although I don't know how useful that solution might be for others. Check out the linked issue for more context if relevant.
ringcrl
commented
2 years ago After playing around with it for a bit I realized
face-api.jshas a monkeyPatch function that allows me to pass a customImageimplementation but node-canvas still doesn't seem to like that very much. It fails withTypeError: Image or Canvas expected. I feel like this has something to do with the check inif (Nan::New(Image::constructor)->HasInstance(obj)) { ... }Where node-canvas internally relies on its own definition of
Image. This behavior is reproducible with just node-canvas and no external libraries usingconst imageResponse = await axios.get(image.rawUrl, { responseType: "arraybuffer", }); const ca = canvas.createCanvas(500, 500); const ctx = ca.getContext("2d"); const imageElem = await imageFromBuffer(imageResponse.data); ctx.drawImage(imageElem, 0, 0, image.width, image.height);I wanted to keep the conversation related to the solution here but I'm happy with moving over to the
@canvas/imagerepo if you likeEDIT: I tried patching the node-canvas instance check to see if that's the issue but I didn't have much luck. I either get a black canvas output or runtime errors.
EDIT2: I was able to solve my issue by combining
@canvas/imagewith@tensorflow/tfjs-nodewithoutcanvasalthough I don't know how useful that solution might be for others. Check out the linked issue for more context if relevant.
The same issue, I can convert webp to png before I use it on node-canvas, It's an extraordinary and inefficient way...
LegendaryEmoji
commented
1 year ago For those who want to decode WebP images so they can use them in their canvas. Here you go:
const webp = require("@cwasm/webp");
const source = fs.readFileSync("./image.webp");
// Decoding the WebP file and putting it on the canvas.
const image = webp.decode(source);
const imageData = ctx.createImageData(image.width, image.height);
imageData.data.set(image.data);
ctx.putImageData(imageData, 0, 0);
// Converting the canvas to buffer and saving it.
writeFileSync("./result.png", canvas.toBuffer());This is a synchronous solution (Bundlephobia: @cwasm/webp).
Issue or Feature
I'd like to be able to draw webp images onto the canvas like I can in the browser. Note that I'm not talking about encoding (i.e.
toDataURL("image/webp")), since that already has an open issue, and an extension.Steps to Reproduce
Below is a minimal example that should work, but doesn't (It predictably throws
Error: Unsupported image type). You can comment out the webp dataURL and uncomment the PNG url to test that it's working fine with PNGs (no surprise).Your Environment