next image optimization significantly slower in next after 10.0.7

[mjenzen]

What version of Next.js are you using?

10.1.3-canary-0

What version of Node.js are you using?

12.12

What browser are you using?

any

What operating system are you using?

linux server

How are you deploying your application?

next start

Describe the Bug

The latest canary releases seem to have fixed the image optimization memory consumption, however the initial image optimization is noticeably slower than the previous version (10.0.7) and can take seconds to populate images on the page. It seems to be exacerbated by loading more images in quick succession as well.

Expected Behavior

Images should load with minimal delay as seen in 10.0.7 to prevent degrading user experience

To Reproduce

Load a page with numerous next/image components and look for images popping in significantly after page load

[karlhorky]

Below is some more context in the tweet from @lovell, the sharp maintainer. Benchmarks may not be completely optimized, but apparently the squoosh approach now in Next.js is up to 25x slower:

Benchmark: https://github.com/lovell/sharp/issues/2643

Tweet: https://twitter.com/lovell/status/1376647760309587979

[JimmyGray]

I'm having the same issue. Some of our pages have 20+ images and the difference is huge. I've rolled back to 10.0.6 for now.

[henryStelle]

I've experienced the same problem, @JimmyGray. As explained in https://github.com/vercel/next.js/pull/22253, sharp was removed to decrease bundle size and increase supported platforms. However, some of these issues have been partially addressed with the v0.28 release of sharp https://github.com/lovell/sharp/issues/2604.

[amuttsch]

We had to roll back as well. We have up to 40 images per page which takes up to a minute to process on my local machine. Due to business concerns we cannot cache images indefinitely and have to regenerate them each day. The current performance is not working on production for us if users have to wait dozens of seconds to see images. We also had to roll back to 10.0 for now.

Is it possible to revert back to sharp as they have addressed the compatibility issues or maybe using it with a feature flag? Thanks!

[kripod]

I would love to see bringing back sharp, potentially as an optional peer dependency. Although Squoosh is a fantastic project on its own, I think it should only be used as a fallback or an opt-in until the situation with ARM machines improves.

[addyosmani]

@amuttsch would it be possible to share a URL with examples of the kinds of images per page that are taking a long time to process locally? Speaking with the Squoosh team about the performance feedback on this thread and any repros would be helpful to look at :)

[NikhilVerma]

We are also noticing similar issues:

• Image sizes are larger for the same quality parameter, I believe Next.js should reduce the default image size from 75 to compensate. Otherwise the consumers will see their website sizes grow. Here is an example.
  • image1 this image comes from an older version of Next.js 56.4kb
  • image2 same image, same quality with the latest next.js version 63.4KB
• The response times for first requests have increased tremendously, you can use the same images in the previous example and tweak the quality param to something new. I see at least 3x slowdown in my testing.
• We use a lot of local images and Next.js caches them only for 60s with no way to override that value. So every 60s we have to face with a long load time for requested images along with a high CPU usage.

My recommendations

1. Re-add sharp as a additional config/override/loader. I am not sure if we can get faster than a bare-metal native implementation, even if WASM is making great strides.
2. Allow overriding the default seconds for cache

Our solution

For now to mitigate this we are adding an additional layer of caching on top of /_next/image in our NGINX instance. It's not ideal because we shouldn't be needing this layer but it helps with the 60s expiry of Next.js local assets.

Here is a sample config

http {
    proxy_cache_path /tmp/image_cache levels=1:2 keys_zone=image_cache:30m use_temp_path=off inactive=7d max_size=2G loader_files=100 loader_sleep=50 loader_threshold=200;
    proxy_cache_valid 604800s; # 7 days
    proxy_cache_lock on;
    proxy_cache_use_stale updating;
    proxy_buffer_size 16k;
    proxy_buffers 16 16k;
    proxy_buffering on;
    proxy_http_version 1.1;
    proxy_ignore_headers Vary;
    proxy_ignore_headers Expires;
    proxy_ignore_headers Set-Cookie;
    proxy_ignore_headers Cache-Control;

    server {
        # ... server params

        # NextJS Proxy for images
        # We need this because of https://github.com/vercel/next.js/issues/23637#issuecomment-822358192
        # Until it's improved we need to maintain our own copy of next/images
        location /_next/image {
            proxy_pass http://localhost:3001;

            # request_uri is important because images have GET params which define quality, url and size
            # $http_accept is important because it changes the response type of next/image
            proxy_cache_key "$request_uri$http_accept";

            proxy_cache image_cache;
        }
    }
}

[amuttsch]

@addyosmani Unfortunately, I'm not able to share images due to copyright and licensing concerns.

But to give you some metadata: our source images are around 500-1000kB in size and have a resolution around 1500x2300 pixel and they are sized down to a width of ~220px using q=75 and webp format. I hope this helps you :-)

[timcole]

Just jumping in here to add my info on this. I recently switched to next 10.1.4-canary.10 to see if that solved the image memory leak but unfortunately our web servers were still using 1GB of ram each (we only use next/image for 5 images). Switching back to 10.0.6 we average around 30Mi.

Example Images: https://constellation.spaceflight.live/falcon9.webp https://constellation.spaceflight.live/SN11.webp https://constellation.spaceflight.live/b/b1fd1f3a97183e4a8d04794c71b7fdaf.webp https://constellation.spaceflight.live/b/cd23264325291a089910a28d70139647.webp https://constellation.spaceflight.live/b/431a992c3171c39331a337f99533c3cc.webp

Code is public at https://github.com/spaceflight-live/comet

[fabb]

I'm seeing a response time increase of 12% after updating from 10.0.7 to 10.2.0 (and updating from webpack 4 to webpack 5). I'm not using the image optimization feature at all though. Could the issue still be related to this issue?

[fabb]

I created a separate issue: #25032

[avisra]

I am also seeing significant slowdowns. This has me wanting to roll back to 10.0.7... but there have been so many other fixes since then. What's the status of this issue @timneutkens? Is there enough information to work on it - or is this ticket still blocked?

[mbretter]

I am also encountering problems, after upgrading from 10.0.5 to 10.2.0 node is constantly crashing with an out of memory error. Rolling back to 10.0.5, everything is running fine.

[thomasgrivet]

Hello!

I am also having issue when using versions > 10.0.7, in order to upgrade I need to run my application in a container with more memory and even then the images load after 7 seconds or so, which is just not acceptable for our use-case. When using Next 10.0.7 images takes about 2 seconds to get fully loaded when not cached. I would love to use the new features such as router.isPreview and more, but I also know it can be hard to make the right decisions for everyone. I hope we will be able to decide the way our application handle the optimisation process in the near future.

[bravetheheat]

is there no progress on this?

[deadcoder0904]

What's the status of this issue?

There are so many issues opened regarding it.

The API is weird & leads to many styling issues. It's promoted heavily but has many issues. Ideally, it should be a drop-in replacement for <img /> tag. Probably should still be in beta. Next.js is so simple but the Image component feels really odd.

I only have 1 Unsplash image & even that takes ~30 seconds to load. The fun part? It's running on localhost.

Repro can be found here → https://github.com/deadcoder0904/better-code-blocks/

[jca41]

What's the status of this issue?

There are so many issues opened regarding it.

The API is weird & leads to many styling issues. It's promoted heavily but has many issues. Ideally, it should be a drop-in replacement for <img /> tag. Probably should still be in beta. Next.js is so simple but the Image component feels really odd.

I only have 1 Unsplash image & even that takes ~30 seconds to load. The fun part? It's running on localhost.

Repro can be found here → https://github.com/deadcoder0904/better-code-blocks/

30 second seems too high, have you tried setting priority true on the <Img />? Is it showing above the fold or not?

The performance decrease i'm seeing is around ~20% or 30%

[DPangerl]

I think the priority prop does not affect performance of the image processor. It simply deactivates lazy loading. As long as the image is not visible it should not be loaded at all.

I have about 6 images on my website. With 10.0.7 the images don't need more than 1 sec to load. With 10.2.0 every single image needs about 9 sec to load (priority or not).

In combination with the 60 sec life time of the cache this is way beyond acceptable. Why is the caching duration this short anyways?

I'm a bit surprised, this issue does not gain more attention as loading images is a common part of every website.

However… in my opinion, sticking with 10.0.7 is the best option for now.

[olikami]

In my testing, this is not a problem in the image component, but rather of the image resizing server that is used if there is no other loader setup.

My current workaround is that i have written a custom image loader as an API route, thatuses Sharp again... (This also allows me to control the caching by myself)

[karlhorky]

My current workaround is that i have written a custom image loader as an API route, thatuses Sharp again... (This also allows me to control the caching by myself)

Yeah, seems like that the Next.js team (cc @shuding) should consider adding back sharp as an option - maybe even as the default option.

@rauchg mentioned this as an option for a loader here: https://twitter.com/rauchg/status/1376654853963845638

I'm guessing the performance of the current squoosh-based option will not improve without some serious work (if there were low-hanging fruit, seems like this would have been done already).

[kristianstroka]

In my testing, this is not a problem in the image component, but rather of the image resizing server that is used if there is no other loader setup.

My current workaround is that i have written a custom image loader as an API route, thatuses Sharp again... (This also allows me to control the caching by myself)

Can you get us example how to use Sharp as custom loader? I tried to do that, but without success, not sure what I'm doing wrong ... that is sad, that next developers are not focusing on this issue, which is really big and a lot of people are complaying about that ... :(

[jca41]

I think the priority prop does not affect performance of the image processor. It simply deactivates lazy loading. As long as the image is not visible it should not be loaded at all.

I have about 6 images on my website. With 10.0.7 the images don't need more than 1 sec to load. With 10.2.0 every single image needs about 9 sec to load (priority or not).

In combination with the 60 sec life time of the cache this is way beyond acceptable. Why is the caching duration this short anyways?

I'm a bit surprised, this issue does not gain more attention as loading images is a common part of every website.

However… in my opinion, sticking with 10.0.7 is the best option for now.

Yeah true, was just considering that in his case with a 30s load time on localhost maybe the issue could be somewhere else.

This is the kind of performance im getting currently. I would have to downgrade next and mesure the perf decrease to have exact numbers, but it does feel a bit slower

• Examples of 3 images loaded with Image

• The raw image loaded on a new tab

[olikami]

If anyone else wants to build their own image resizer I dropped our code in a gist (we only use this internally or for noncritical stuff): https://gist.github.com/olikami/236e3c57ca73d145984ec6c127416340

Please note that it isn't very good, the code is potentially buggy, and you'll use it at your own risk. In production, you should probably use something like thumbor or Cloudimage if SaaS is more your style.

[deadcoder0904]

30 second seems too high, have you tried setting priority true on the <Img />?

@jca41 Yes, I tried priority={true}, loading="eager", & even quality={60} to make it work but it's too damn slow. I know ~30 seconds is high but I did see twice or thrice in the Chrome Extension, the request gets fulfilled around ~30000-40000ms.

I have also provided the repro so you can check for yourself :)

Initially, I thought something was wrong with my code but img tag works fine.

[deadcoder0904]

@jca41 Whoops, it's ~8 seconds. I was checking the thing above it but your screenshot pointed me to look below. It's a 3.9MB image locally & image optimization makes it 73.1kB.

But yeah, still noticeable to see the difference. The props mentioned above don't make any difference at all.

[jca41]

@jca41 Whoops, it's ~8 seconds. I was checking the thing above it but your screenshot pointed me to look below. It's a 3.9MB image locally & image optimization makes it 73.1kB.

But yeah, still noticeable to see the difference. The props mentioned above don't make any difference at all.

yeah that seems more or less what i'm seeing on my repo

[addyosmani]

As an update here, we have been having discussions to think through what the right way to approach this problem should be as there's some nuance to the solution space. We may ultimately need a different solution for production (sharp) vs dev (squoosh), but teasing apart some thoughts:

• When deciding between a WASM vs. native-based solution for image optimization, what you pick (sharp, squoosh, others) may depend on whether you're optimizing for overall binary size, overall image processing times, cross-platform compatibility or all the above :). We could make it clearer which of these Next.js sees as key. For most production use-cases we should recommend Sharp
• Binary size may be important if you want to make sure the first-run experience is good for users new to Next. I could see the argument that Next would need sharp to be small enough in size for it to be competitive to squoosh or alternatives for this use-case. A counter-argument might be that users won't be re-installing Next that often, but again, depends on motivation.
• Image processing times (as seen on this thread) are also important as these can have an observable developer and user-facing effect. I believe the Next team's thinking here is that performance may matter less if you're doing this as a once-off per image and then caching. My personal view is that this is working slower than I'd expect (...and we should look into whether the path forward is improving perf for the WASM path with squoosh/otherwise, making sharp an optional dependency and using that instead etc).
• Cross-platform compatibility is a legitimate concern (and I believe I recall there having been issues with sharp on M1s that are now resolved - there may be others). At the same time, sharp being a Node dep means it can't strictly be as portable as a WASM solution (node-gyp) and may not work in environments like Cloudflare workers.
• For the dev experience, I think it’s fine if we aren’t optimizing images or consider using Squoosh in the browser for this case. Perhaps via a Service Worker.

My impressions of this thread are that most folks are worried about slowdowns in production with some seeing slowdowns in dev as an issue. Is that a fair assessment?

We should align on what is most important to optimize for where Next.js is concerned as that would help us figure out next steps. Fwiw, other stacks are facing similar open questions about the right way to solve this: https://github.com/unjs/ipx/issues/31

[deadcoder0904]

My impressions of this thread are that most folks are worried about slowdowns in production with some seeing slowdowns in dev as an issue. Is that a fair assessment?

Yes, correct @addyosmani. Haven't tested on prod yet but dev takes ~8 seconds for a single image which made me wonder if I did something wrong with my code but then after ~15-30 mins realized it was a Next.js bug.

In the meantime, I guess the Next.js team can add a note on next/image docs to stick to previous versions if they want to use <Image /> component as I think it's a more common issue :)

[DSamuylov]

I also experience this issue, here there is an example where I use both next/Image and just img (after build):

If I refresh the page after the images are loaded for the first time, then I can reload the page and it works fine. But if I reload a few min later, it is again slow. Cannot go to prod with it...

Downgrading to 10.0.7 solved the issue.

[eddsaura]

@DSamuylov but is it the same in production?

[DSamuylov]

@eddsaura It is in prod.

I tried to deploy it, took the screenshot and then rolled back to the previous version.

[kristianstroka]

we have same issue also after production

[olikami]

If you use the built in image loader it will also appear in production

[mjenzen]

My impressions of this thread are that most folks are worried about slowdowns in production with some seeing slowdowns in dev as an issue. Is that a fair assessment?

We should align on what is most important to optimize for where Next.js is concerned as that would help us figure out next steps. Fwiw, other stacks are facing similar open questions about the right way to solve this: unjs/ipx#31

For my use case, the production performance issue is so overwhelming that I can't even consider the other aspects you mention. As a high product count, lower traffic traffic e-commerce site, the first time someone views a page is a fairly common event and cannot be discounted as a throwaway experience. I also have a potential for tens of thousands of bad experiences as each product page has to process and cache image(s) for the first time. Also, depending on the deployment methodology, the image cache may not persist between deployments.

It is an incredibly powerful feature to be able to add image optimization with very little developer investment and almost no impact to user experience. I felt the feature also added a ton of "stickiness" to the platform as it was a feature I was able to get basically free whereas other solutions will require significant time investment or a monthly subscription. But without a significant performance boost, I don't think I will be use the >10.0.7 without abandoning Image Optimization.

[camyyssa]

On my production website, some images are loaded normally, while others take a long time to load, and sometimes don't load at all. Right-clicking on the ones that don't load, and opening them in a new tab results in an error 500 page.

This behaviour is not reproducible on localhost, just on deployments (production and previews)

I've created a demo project before finding this issue, so I'm going to share it here in case it's useful for someone 😊. The demo project is created with npx create-next-app and I've added some images to it. It uses Nextjs version 10.2.3

This is how it looks like when I load the production deployment for the demo project:

URL to get to the demo project is: https://nextjsmissingimages.vercel.app/ Public Github project: https://github.com/camyyssa/nextjsmissingimage

[goguda]

My impressions of this thread are that most folks are worried about slowdowns in production with some seeing slowdowns in dev as an issue. Is that a fair assessment? We should align on what is most important to optimize for where Next.js is concerned as that would help us figure out next steps. Fwiw, other stacks are facing similar open questions about the right way to solve this: unjs/ipx#31

For my use case, the production performance issue is so overwhelming that I can't even consider the other aspects you mention. As a high product count, lower traffic traffic e-commerce site, the first time someone views a page is a fairly common event and cannot be discounted as a throwaway experience. I also have a potential for tens of thousands of bad experiences as each product page has to process and cache image(s) for the first time. Also, depending on the deployment methodology, the image cache may not persist between deployments.

It is an incredibly powerful feature to be able to add image optimization with very little developer investment and almost no impact to user experience. I felt the feature also added a ton of "stickiness" to the platform as it was a feature I was able to get basically free whereas other solutions will require significant time investment or a monthly subscription. But without a significant performance boost, I don't think I will be use the >10.0.7 without abandoning Image Optimization.

I agree, after upgrading in production I quickly realized it was a big mistake and reverted back to version 10.0.7. The inconsistency and amount of time image optimization is taking in all of the more recent versions is simply unacceptable for production use and if I was a user, I'd be driven away from the website by this as well.

[DPangerl]

So what about https://nextjs.org/blog/next-11… Has anyone testet the new version already?

[squelix]

The issue is still here in Next@11 :(

[kabachook]

It is so sad to see compatibility is more important than performance [#22253] I guess most production deployments are using x86 and not ARM (especially not M1). I don't see a point in dropping sharp in favor of bunch of binary wasm files with no versioning (it is not included as a depencency in package.json), especially since ARM support was added in sharp.

If we talk about install size, I would consider splitting next/image in a separate package, so that it is installed if only used.

[eddiewang]

we are experiencing this issue right now in staging with the latest version of Next. downgrading to v10.0.7 fixes the problem, but would prefer to have the latest Next.js patches added to our application.

It takes around 15-30s for a full page load, as pictures take quite a long time to render. The pod was provisioned over 1 GB of ram and 1-2 CPU cores.

[valstu]

We're experiencing this as well, I also tried v11 but didn't notice significant difference.

[jca41]

Did not experience any noticeable improvements with v11 (deployed on Vercel).

[Punkte]

Experiencing this as well, it's quite an annoying problem for the UX and the SEO ranking.
In the meantime you can improve the user experience by using the placeholder API.. 😕

[royjosefsson]

I'm experiencing the same issue. But for me, it's not the upgrading of next that is causing the issue, it's actually some peer dependencies. Look at my printscreen. As soon as i fix these 10 minor error, image becomes slow. https://drive.google.com/file/d/1b20IsW3aaZ9WYJW_sK9KkfMYFDQEn2mH/view?usp=sharing

[valstu]

@royjosefsson link doesn't work

[royjosefsson]

Sorry. https://drive.google.com/file/d/1b20IsW3aaZ9WYJW_sK9KkfMYFDQEn2mH/view?usp=sharing

@valstu

[Tralgar]

Will it be fixed soon plz ? The first generation of a size seems really too long. Downgrade to 10.0.6 seems too work for me but now I'm missing features...

[DSamuylov]

After testing if for a while, I have an impression that even with the version 10.0.7 images are loaded quite slow. It still takes a few sec. It definitely faster, then with the latest version of next.js, but it is way slower than images embedded with img. Is there a chance that with an upcoming fix the images would be loaded faster than with the version 10.0.7? Otherwise, I am really considering to avoid using Image from next.js at all and rely only on img.

[goguda]

After testing if for a while, I have an impression that even with the version 10.0.7 images are loaded quite slow. It still takes a few sec. It definitely faster, then with the latest version of next.js, but it is way slower than images embedded with img. Is there a chance that with an upcoming fix the images would be loaded faster than with the version 10.0.7? Otherwise, I am really considering to avoid using Image from next.js at all and rely only on img.

Well of course it’s going to be slower than img by a bit, it’s optimizing on the fly before it serves the image and that takes a bit of time. However, the amount of time we’ve seen with more recent versions is just too long to be useable.

[DSamuylov]

@goguda yes, I understand that, but it take so long even in the older version (10.0.7) before the amount of time was increased. Of course you need to precompute images of different size and it takes time.

However, why is the cache so short lived? From what I remember it is only 60 seconds and then the images are recomputed again on a new request.

Would it make sense to add an option to increase the cache time or even save resized images on disk during the build time (if there are only a few images on the website and the space is not a concern)?

Or should it be handled elsewhere, for example by configuring cache in nginx?