search

lovell / sharp

High performance Node.js image processing, the fastest module to resize JPEG, PNG, WebP, AVIF and TIFF images. Uses the libvips library.
https://sharp.pixelplumbing.com
Apache License 2.0
29.37k stars 1.3k forks source link

"File too short" error on lambci/docker-lambda images #1543

Closed jamespedid closed 5 years ago

jamespedid commented 5 years ago

Hello, I am trying to use sharp on a replica AWS lambda image, and it appears that the vendored linux version of libvips is not properly working.

image

The lambci images derive ultimately from the "scratch" image of docker, which apparently is one that is super empty. I've tried rebuilding the sharp library in various ways, but it always seems to be downloading the image from the pre-vendored libraries that you have provided.

Any suggestions on how to work around this would be appreciated.

jamespedid commented 5 years ago

Figured out that there was a different issue in my pipeline.

jamespedid commented 5 years ago

If anyone wants to know what my issue was: the symlinks were getting removed when copying sharp files using a nodejs module. I had to dereference the symlinks to make sure the appropriate files were copied over. If you run into this issue, make sure disappearing symlinks is not the cause of your issue.

codephobia commented 4 years ago

@jamespedid Do you mind sharing how you solved the issue? I am running into the same issue packing with claudiajs.

jamespedid commented 4 years ago

@codephobia Hello, not sure if I remember exactly what the issue was, but I believe this is what I did.

I was using nodejs to copy the node modules from one location to another using the fs-extra module. One option in the fs copy command is to 'defererence' the items. I enabled this to true when copying some modules.

await fs.copy('/opt/project/subproject1/node_modules', '/opt/project/output/${jobName}/subproject1/node_modules', { deference: true });
await fs.copy('/opt/project/subproject2/node_modules', '/opt/project/output/${jobName}/subproject1/subproject2/node_modules', { deference: true });
await fs.copy('/opt/project/job/node_modules', '/opt/project/output/${jobName}/subproject1/subproject2/job/node_modules', { deference: true });

The 'tower' of node_modules was used to make resolving dependencies easier, so you can probably ignore that unless you're doing something similar. The deference: true option is I think what you're looking for.