Open lovell opened 10 years ago
Hi Lovell,
at https://kraken.io we process millions of images every month. Speed and resources consumption are the most important factors for us. That's why now we look closely at your project and libvips
itself. After our initial tests we decided to give it a try and deploy our sharp-enabled apps to a tiny number of our API machines and see how it behaves in production.
Keep up your good work. We will definitely give you a shout after our trials.
@lovell
Great work and thanks for introducing libvips to the node.js ecosystem.
At http://timesmachine.nytimes.com We generate and optimize millions of thumbnails from original New York Times newspaper scans, which happen to be very large tiff files with dimension (~6000 × 6700). That's 40.2 million pixels for one image file. This creates a major performance bottleneck for our thumbnail generation pipeline.
For more information about what we are doing with images: http://searchaws.techtarget.com/news/2240220787/New-York-Times-goes-back-to-the-future-with-AWS-storage
building an image service like imgur and kraken, but i plan to always make it free and public because https://github.com/mgmtio/simgr
right now i'm testing it on heroku, but i'm running into memory problems and i don't want to pay for a better server haha
StoryDesk is using sharp in prod for resizing and optimizing all the user content uploaded to our cloud based presentation app. We also use it behind the scenes to resize image captures of presentation pages for thumbnail generation.
We choose sharp because it is fast and efficient compared to the alternatives.
http://unsplash.it/ Uses sharp for serving user-defined sizes of images, because of it's usability and speed
I'm using sharp to crop/resize assets on the fly for various sites (mainly portfolio/image heavy), I've used Imagemagick for years and have been amazed at the performance gains. I was prompted to investigate alternatives by the guy behind http://images.weserv.nl/ who pointed out how slow it was.
We are using Sharp as part of an in-house on-the-fly image re-sizing cluster to handle all of our real estate listing photos at HomeFinder.com. Our old system used ImageMagick to pre-process photos before publication on the web. Your library with libvips is fast enough that we can handle all the photo manipulation on-demand, giving us the flexibility to use any photo size we want on the site, while only processing photos that are actually seen by our visitors.
We're using Sharp to resize / optimize / generate thumbnails of images uploaded by our users and manipulate page screenshots on IgniteTeaching.com and our iPad app. I've been very impressed with the performance and the responsiveness of the maintainers - thanks so much for keeping up the great work!
We are using sharp to deliver dynamically resized and retina optimized movie posters for the streaming services youseeplay.dk and blockbuster.dk.
I am working on replacing my php script that get the images from the ftp server and resize it. Nodejs and Sharp seems to use less resources then php.
@FiftyThree uses sharp on our Mix backend to pyramid (resize) our user image uploads into different resolutions as well as compositing PNGs onto different background colors (hence #94). We previously used ImageMagick and improved our image processing performance by several factors once we switched to sharp. Thanks for all your hard work on this, @lovell :heart:
We are using Sharp at InVisionApp in increasing amounts as we replace legacy ColdFusion code with node microservices. We are using it for thumbnails as well as general resizes, and chose to use Sharp due to the performance gains compared to Image Magick
On the fly local and s3 image resizer.
Image validation/resize on upload and tile() for breaking big floor plans into tiles to show them with Leaflet.
We're using sharp as the guts of an on demand image resizing service. Signal is a group video chatting app that allows you to browse photos, youtube videos, and the like synchronously with your friends. Internet together! https://signal.is/
Hey Lovell, great module!
We're using sharp at GoDaddy via an open-source image server, ImageSteam. This allows us to serve only the bytes required by the device viewing the page, enabling much better user experiences.
Thank you Lovell, for the great module. We at https://app.covve.com are using sharp as part of our avatar micro-service both for creating standard sizes for our common cases as well as transforming avatars on the fly for the not so common requests. Covve is serving millions of contacts to its users and most of them have avatars, so we are putting your module to a good use.
We use sharp
to resize originals & generate thumbnails. In normal site mode gm
was ok, but for migration we have to process 1M of images, and sharp
is significantly faster.
It is used to optimize & presize facial photos. Made a handheld mobile access card reader solution for Kaba and instead of having to push images up to 500KB down a 3G or wifi network we now have in general only some 10 to 15KB per image. With up to 64k images that is rather a lot less bandwidth and loading time
at http://sunshineapp.com we are creating an application that use DLNA and peer to peer to give the users the ability to browse and stream there files across all devices in realtime and we are using sharp in our nodewebkit desktop app for collecting files info and generating thumbnails for all image files and it is Amazing :)
We sync images to an ios app, since the images are too big in width/heigth and filesize, I just landed here =)
Generate image thumbnails for pre-existing assets and also to rotate mobile images when being uploaded from the client.
Using to convert tiffs to pngs to display in browser
We use sharp to convert and resize PNGs/JPEGs into bitmaps for use in a webgl context on a headless instance.
We use sharp to stream and transform screenshots generated by chrome headless and store them in google cloud storage. We use them for customer-facing images as well as internal screenshots that are taken and analyzed for visual regressions.
Extracted compressing user uploaded images to AWS Lambda for speed, cost, scalability.
In future, we will extract resizing/cropping operations from app to Sharp @ Lambda as well.
Hi @lovell , thank you for this amazing project! I'm using it to build Caravaggio. Without sharp it wouldn't be possible, or at least so easy to realize! I'll let you know when it will be production ready.
Lazily compressing admin-supplied images in jpg/png format to web-ready jpg/png/webp.
Results are stored on disk under a predictable URL (size+aspect ratio+type), so if the file is not on disk it is auto-generated.
I take care to use webp when the client supports it, but now I found out that mozjpeg can supposedly rival webp results (at the cost of compression time) and I'm a bit sad that sharp doesn't ship with mozjpeg :)
We are using Sharp to process images in order to prepare them for printing in a Print-On-Demand system. The POD system needs to have a "print sheet" which combines several images, laid out in a grid. We are using Sharp to combine the images into this grid. We also have to rotate some of the source images before adding to the grid.
Some complications / challenges we have include...
We are working with CMYK, 300ppi, tiff files as source images. When manipulated with Sharp, we seem to lose the ppi information and the images are converted to sRGB. So far, I think this is fine as we are doing some post-processing work using GraphicsMagick to fix that in the final image.
We are running the image processing in Node on Alibaba Function Compute (basically their version of AWS Lambda). There are environmental limitations of 3GB memory and 1GB ephemeral disk space, which has been a struggle to work within. Also, we had to jump through some hoops to compile Sharp/libvips in a sandbox container in order to have it compile for the correct architecture.
Hope this helps! Thanks!
Hello @lovell,
Bunk1.com uses Sharp with AWS Lambda to create carrierwave-compatible images in s3. We use auto-rotate as well. The photos are then applied to a wide range of photo products.
Sharp is powering all image operations at authory.com.
I've used OpenCV (in depth) and PIL before, but I am amazed how incredibly fast sharp is. Also, the API is simple, but still powerful. Features are complete and thoughtfully designed (like rotate()
respecting EXIF tags). The documentation is spot-on. And all that with super few dependencies.
10/10.
We are using Sharp for transforming images on all of www.aktualne.cz. It's an online news website where journalists upload photos and we resize and extract them on the fly upon first 404 request on a parametric route to S3 and Lambda for actually generating images.
Recently sharp has also been a cool edition to a personal small-scale project saywebpage.com, which is in turn a website builder with limited functions. Here, apart from resize and extract, other image modification like blurring are possible for users creating their small websites. The setup is copied using S3 and Lambda.
Sharp has been great for both projects.
@Goues oh nice, so it normally gets the image from S3 directly but if there's a cache miss, it fires a lambda that generates the resized image? What happens with the original request, does it get 404 or does it get the resized image?
@wmertens Yes, request goes to S3 website bucket. If object is not yet generated, a 404 redirect rule is applied and user is redirected to a Lambda function. Lambda generates image and redirects the user back to S3 where the object is now present. It means there are two redirects for the first user but this cost is negligable for a large enough userbase. Putting a CDN in front of S3 is obviously needed to prevent too many GET requests to S3. For a large service, cache hit ratio on CDN can get up to 99.9 % over 10 TB od traffic, so that is nice.
@Goues brilliant :) How do you handle duplicate uploads and duplicate file names? I sanitize+uniquify filenames but I don't handle duplicates, because I'm not sure if I should then provide the original filename as the upload id, or create some alias…
Feel like maybe open sourcing your solution?
@wmertens We might open source in the future but not at the moment - there is a lot of stuff specific for our database and integration with other services that would be time consuming to separate.
In general, duplicate uploads are handled by file hashes. You do not upload to a bucket images are served from, but to an upload bucket. Put Event is then send to SQS for ingesting (another lambda) where we take an MD5 of image and create a rename a file accordingly. Duplicate upload means an image of the same md5 name already exists in public bucket. There is also a Dynamo DB table for storing status (uploaded cannot use uploaded image until it is igested and confirmed).
On the smaller project, this pipeline is downscaled by taking care of duplicates at all and users have limit of stored MB of images, so they are themselves responsible for not creating duplicates.
We went with this solution after trying out lambda@edge on cloudfront, because it would allow for no redirects at all, but edge lambdas are not good with binary data and large response bodies so it didn't turn out well.
There is a good "how to start" here: https://aws.amazon.com/blogs/compute/resize-images-on-the-fly-with-amazon-s3-aws-lambda-and-amazon-api-gateway/
Very interesting - another aws blog has a post on using sharp and lambda@edge: https://aws.amazon.com/blogs/networking-and-content-delivery/resizing-images-with-amazon-cloudfront-lambdaedge-aws-cdn-blog/
I'm using Sharp to handle image manipulation in a open source image capture application for Mac and Linux: https://github.com/MagicCap/MagicCap/blob/develop/magiccap/capture.js#L145
I would like to use Sharp to create CMYK print files.
I'm using sharp at Criteo to resize on the fly pictures I download (as a first step before doing computer vision on them). This is really important to be able to do it fast and correctly to be able to handle the scale (million of pictures) without using too much disk nor too much time. Using sharp I'm able to download and resize about 1M images per hour. (that's on a single machine, it can probably be even much faster when we'll distribute it)
It's the "engine" behind my project photosaic.
I now have >100 stars after posting to Show HN :)
PhotoStructure uses sharp to build and resize preview images: it's great to have a performant library that runs on macOS, Windows, and Linux.
:+1: Thanks! :star2:
I'm using sharp to resize/composite on the fly some pictures. The main reason: it's super fast
I use Sharp for all image resizing (both during upload and in realtime by our image server) on Arthur.io. Couldn't do without <3
I would like to use Sharp to create CMYK print files.
so do l But I have a problem now, and I have found that using sharp to convert cmyk cannot achieve the same effect as using PIL (Python) to convert. I hope all technical experts can provide some help
/
img = Image.load("pngfile")
img.convert("CMYK")
sharp("pngfile").tiff().toColorspace("CMYK")
I use Sharp to resize and optimize images.
is anyone using sharp for image sanitization? i mostly just want to make sure that no malicious code is being delivered in the image data that i am collecting.
@gibsfinancedev You should do your own audit if it matters to you.
I'm truly humbled by the recent surge in support for
sharp
.Whilst :star: and the trending status they bring are both amazing, thank you, the real measure of open source success is happy users.
I'd love to find out more about how you or your organisation are using (or plan to use) this module.
Feel free to add a comment with details of who you are, what you're doing with it and why you chose it. Alternatively let me know via @lovell on Twitter.
Thank you!