vercel / next.js

The React Framework
https://nextjs.org
MIT License
127.51k stars 27.04k forks source link

Next13 caching fetched data even with the 'revalidate = 0' route segment #51788

Closed EvandrooViegas closed 7 months ago

EvandrooViegas commented 1 year ago

Verify canary release

Provide environment information

Operating System:
      Platform: win32
      Arch: x64
      Version: Windows 10 Home
    Binaries:
      Node: 18.12.1
      npm: N/A
      Yarn: N/A
      pnpm: N/A
    Relevant Packages:
      next: 13.4.8-canary.2
      eslint-config-next: 13.4.6
      react: 18.2.0
      react-dom: 18.2.0
      typescript: 5.1.3
    Next.js Config:
      output: N/A

Which area(s) of Next.js are affected? (leave empty if unsure)

App directory (appDir: true), Data fetching (gS(S)P, getInitialProps)

Link to the code that reproduces this issue or a replay of the bug

https://github.com/EvandrooViegas/outsystem-angola

To Reproduce

  1. clone my repo
  2. run npm install in the root of the cloned repo
  3. run npm run dev

Describe the Bug

Next13 seems to be caching the data coming from supabase even with the revalidate = 0 route segment.

Expected Behavior

Get the freshest data in every page request

Which browser are you using? (if relevant)

chrome

How are you deploying your application? (if relevant)

No response

SimulShift commented 1 year ago

im having the same issue. I tried all of these

export const fetchCache = 'force-no-store'
export const revalidate = 0 // seconds
export const dynamic = 'force-dynamic'

I looked in the network tab and i see this response header:

X-Nextjs-Cache: HIT

The docs talk about the X-Nextjs-Cache for images but its applied to the route as a whole in my case

kevincobain2000 commented 1 year ago

Doesn't work, it keeps fetching the cached data

"next": "13.4.7", Edit* Also in 13.4.8-canary.14

export const fetchCache = 'force-no-store'
export const revalidate = 0 // seconds
export const dynamic = 'force-dynamic'

export default async function GetData() {
    const data = await fetch("http://127.0.0.1:3001/time", {
        next: { revalidate: 0 },

    })

    const json = await data.json()

    console.log(json)
    return (
        <div>
            <ul>
                {json.time}
            </ul>
        </div>
    )
}
nevenhsu commented 1 year ago

Same here. I currently add a random number as query to bypass this bug. Something like: http://localhost:3000/?t=1688177362

kevincobain2000 commented 1 year ago

Upon little more investigation, revalidate:0 works but only for one particular use. If the component that is fetching the data is server use server then upon hard refresh, it fetches each time, but when navigating via <Link it uses the old cache and never hits a request. Edited* That too works sometimes and sometimes it just doesn't.

naishe commented 1 year ago

Linking it to an open discussion https://github.com/vercel/next.js/discussions/51612#discussion-5323305

@nevenhsu -- unfortunately, this does not solve the page redirection issue or browser back button issue. So, I have a detail page /address/[id] and an edit page /address/[id]/edit. When the form on the edit page gets submitted, I redirect them back to /address/${id}?r=${Math.random()}. You see the new data on this URL with new query string, BUT it does not clear the page cache. If you go to /address/[id] from menu which uses <Link href=/address/${id}>..., you still see this old data. So, either you always uglify your URL with Math.random() all the time and never use browser's back button, or convert it to client component.

I ended up using page as just static shell and moved on to React-Query. The peace of mind of knowing that you are in control and not some voodoo magic keyword is returning data based on its mood, is unparalleled.

kevincobain2000 commented 1 year ago

Thanks @naishe I mean it’d be nice if nextjs fixes it, as this is a much desirable feature to unleash power of react server side components. A bit disappointed that they pushed half baked on the major version

EvandrooViegas commented 1 year ago

Doesn't work, it keeps fetching the cached data

"next": "13.4.7", Edit* Also in 13.4.8-canary.14

export const fetchCache = 'force-no-store'
export const revalidate = 0 // seconds
export const dynamic = 'force-dynamic'

export default async function GetData() {
    const data = await fetch("http://127.0.0.1:3001/time", {
        next: { revalidate: 0 },

    })

    const json = await data.json()

    console.log(json)
    return (
        <div>
            <ul>
                {json.time}
            </ul>
        </div>
    )
}

If that is a component make sure to add the export const revalidate = 0 and export const dynamic = 'force-dynamic' route segments in the page that is rendering that component

kevincobain2000 commented 1 year ago

Yes, thanks @EvandrooViegas I tried with route segments, layout and even putting the fetch code inside the route component as well. It only works when I manually refresh the page. Upon route Link click, it never hits the GET request.

EvandrooViegas commented 1 year ago

Yes, thanks @EvandrooViegas I tried with route segments, layout and even putting the fetch code inside the route component as well. It only works when I manually refresh the page. Upon route Link click, it never hits the GET request.

could you provide the repo link?

Vinlock commented 1 year ago

I'm getting this same issue. Nothing is working besides hard refreshing or revalidatePath() which feels so far removed...

matannahmani commented 1 year ago

this is literally the worst thing, why release it if it's not working... back button / link always keep cache no matter what you do

EvandrooViegas commented 1 year ago

I found this video very helpful

jack-szeto commented 1 year ago

According to the video that @EvandrooViegas shared. I got this working. I'm sharing this for my future self and hope this can help someone. Please point out if I'm wrong.

// components/post-editor.tsx
"use client"
import React, { startTransition, useCallback } from "react";
import { useRouter } from "next/navigation";
// ...

export const PostEditor = ({ post }: { post: Post }) => {
    const router = useRouter();
    const updatePost = useCallback(async (values: Post) => {
        // I'm using ky to fetch data, I guess you can use native fetch() as well
        await ky
            .put(`/api/post/${values.id}`, {
                json: values,
            });
        startTransition(() => {
            router.refresh();
        });
    }, [router]);

    return (
        // ...
    );
}
// /app/post/[postId]/page.tsx
const Page = async ({
    params,
}: {
    params: { postId: string };
    searchParams: { [key: string]: string | string[] | undefined };
}) => {
    const post = await GetPost(params.postId);

    return (
        <PostEditor post={post} />
    )
}
export default Page;

Basically, I'm using startTransition to call the router.refresh() whenever I update the posts. This can refresh the pages cached by Router without affecting the UI.

kevincobain2000 commented 1 year ago

Thanks for the video @EvandrooViegas and solution @jack-szeto My take away from the video is that Nextjs team is working on it. Ideally we shouldn't have to add startTransition, route.refresh on each time we want to make a navigation work.

masterkain commented 1 year ago

I get this too -- during an investigation on why caching works the way it does on a kubernetes cluster I get some unexpected behavior.

navigation on simple sidebar Links is being served from somewhere for an unspecified amount of time, I think this is browser behavior or react.

navigating to a revalidate = 0 page generates this response, which is clearly trying its best to avoid caching

Screenshot 2023-07-09 at 12 24 27

yet when revisiting the same link there are no requests generated. only after an unspecified amount of time the browser understands it needs to ask again the resource to the server and sometimes you can get so stuck that only a browser refresh keeps you going.

compared that kind of response to one generated from a generic rails server the only thing that stands out is the presence of a weak etag

matan00 commented 1 year ago

@masterkain apparently the client side router doesn't respect this behavior therefore links and back button doesn't work as expected to disable this functionality, you have to put on the this will solve the problem most likely, but then again this solution in my opinion is not good enough, especially when you're doing a big batch of calls on server components everytime you have to refetch it. My solution was integrating tanstack query on client side and on server component I dehydrate the query to the client, it's not perfect, it doesn't feel good, but it's what it's for now, hopefully next team will make the client router behavior more dynamic and allow us more control instead of all of this magic.

masterkain commented 1 year ago

this has also some interesting details and updates https://github.com/vercel/next.js/issues/42991 that kinda explains the behaviour I'm seeing

apparently it's a long standing issue that is being worked on

As of next@13.4.0 the client side cache has been reworked, dynamic pages are now cached in the client with a timer of 30 seconds, so every 30 seconds your server is recalled, with one catch

hozunlee commented 1 year ago
image

x-vercel-cache: HIT xxxxxx

nextjs + vercel

The fetch works properly locally, but there is a problem when I distribute it to the vercel.

'/' The route only gets new data when redistributed, and when you reload the page, 304 appears.

Funny thing is that '/post/[id]' becomes a prerender and gets the data right.

x-vercel-cache: HIT

I always want to get an x-vercel-cache: Miss.

Has anyone solved it?

jandk008 commented 1 year ago

Got similar issue with nextjs + vercel and the fetched data is always cached no matter what solutions avoiding cache that is written in the document is used. But it works well locally. And the nextjs + vercel works well after I downgraded nextjs to 13.3.4 from 13.4.7. even with the sharing of reworked cache since 13.4.0 as someone else mentioned, it looks odd to have unfunctioning 13.4.0 versions.

knightrun commented 1 year ago

im having the same issue The page should display new data every time it's accessed, but page.tsx keeps showing cached data without making API calls. I have tested it in the development environment. Is there any solution available to resolve this problem?

"next": "13.4.9"

I tried the following configurations, but unfortunately, they didn't work export const dynamic = 'force-dynamic'; export const fetchCache = 'force-no-store'; export const revalidate = 0; cache: 'no-store'

tjgeni commented 1 year ago

im having the same issue The page should display new data every time it's accessed, but page.tsx keeps showing cached data without making API calls. I have tested it in the development environment. Is there any solution available to resolve this problem?

"next": "13.4.9"

I tried the following configurations, but unfortunately, they didn't work export const dynamic = 'force-dynamic'; export const fetchCache = 'force-no-store'; export const revalidate = 0; cache: 'no-store'

@knightrun I was add the same configuration, and work in development.. export const fetchCache = "force-no-store"; export const revalidate = 0; // seconds export const dynamic = "force-dynamic";

you can look at it on my repo

kevincobain2000 commented 1 year ago

Update: Doesn't work with v13.4.11 https://github.com/vercel/next.js/releases/tag/v13.4.11

knightrun commented 1 year ago

@tegarjgap Thank you for your comment.

Have you verified if the getAllProduct is triggered when accessing the page using next/link or router push? It's still not working for me.

ml-sketch commented 1 year ago

I am having the same problem. I've tried it all, but it just won't work. I believe Vercel needs to fix this. It only works with router.refresh but that should be a big nono

tjgeni commented 1 year ago

@tegarjgap Thank you for your comment.

Have you verified if the getAllProduct is triggered when accessing the page using next/link or router push? It's still not working for me.

yeah @knightrun .. in my local development.. the CACHE - MISS show on console.. when call getAllProduct()

tolgaand commented 1 year ago

I've create a basic API, but I'm having a problem. Every time I send a request to the API, I keep getting the same URL back.

import { NextResponse } from "next/server";

const images = [
  "https://i.imgur.com/x.png",
  "https://i.imgur.com/x1.png",
  "https://i.imgur.com/x2.png",
  "https://i.imgur.com/x3.png",
  "https://i.imgur.com/x4.png",
  "https://i.imgur.com/x5.png",
  "https://i.imgur.com/x6.png",
  "https://i.imgur.com/x7.png",
  "https://i.imgur.com/x8.png",
];

export async function GET() {
  const randomImage = images[Math.floor(Math.random() * images.length)];

  return NextResponse.json({
    image: randomImage,
  });
}

page.tsx:

export const revalidate = 0;
export const dynamic = "force-dynamic";

and button onclick function:

 const { image } = (
  await fetch(`/api/generate-nft-url?d=${Date.now()}`, {
    next: { revalidate: 0 },
    cache: "no-cache",
  })
).json();

startTransition(() => {
  router.refresh();
});

I will try all the suggestions above, but the problem doesn't get fixed.

kevincobain2000 commented 1 year ago

It seems like this is the most commented issue https://github.com/vercel/next.js/issues/42991

kevincobain2000 commented 1 year ago

Haven't tested it but https://github.com/vercel/next.js/releases/tag/v13.4.13-canary.8 seems promising for related issue here https://github.com/vercel/next.js/pull/53373

emrecoban commented 1 year ago

I've the same issue. Interesting problem, revalidate solve the problem in local, but not in prod.

export const revalidate = 0;

// no need these ones, but you can try.
export const fetchCache = 'force-no-store'
export const dynamic = 'force-dynamic'

I've tried next@13.4.14-canary.2 but still the same. Is there any solution?

Update: My issue is related to Cloudflare. Everything is okay on Vercel.

kevincobain2000 commented 1 year ago

It seems like the client side caching cannot be disabled https://youtu.be/_yhSh4g0NSk

AnirudhSinghBhadauria commented 1 year ago

Router cache cannot be neglected, sad.

brain1401 commented 1 year ago

guys, is this still unsolved ?

diveddie commented 1 year ago

I can confirm that @jack-szeto's recommendation to use router.refresh() works for me. I feel like this is a workaround for now, and I'd like to see that the server-side cache can be invalidated. Might just switch to React Query for the time being until this works 😐

nodegin commented 1 year ago

same here... as an user, we have no control on how next controls our cache?

nodegin commented 1 year ago

I found a solution that works for me (Apollo with RSC).

Just add the middleware.ts on your root directory with the following content:

import {
  NextRequest,
  NextResponse,
} from 'next/server'

export function middleware(request: NextRequest) {
  if (request.nextUrl.pathname.startsWith('/_next')) {
    return NextResponse.next()
  }

  // Override Next default's overaggressive caching behavior
  // Make all internal requests must revalidate every 15 seconds
  const url = request.nextUrl.clone()
  url.searchParams.append('_nextCacheSkip', Math.floor(Date.now() / 1000 / 15).toString())

  return NextResponse.rewrite(url)
}

This ensures Next only cache for specified duration then fetch up-to-date data again.

zhouw commented 1 year ago

There’s a new client side cache implemented since v13.3.2 #48695

When you use next/link to do soft navigation, dynamic pages are cached on client side for 30seconds and there is no way to opt out. There has been discussion in other threads going on.

From the doc:

Opting out

It's not possible to opt out of the Router Cache.

You can opt out of prefetching by setting the prefetch prop of the component to false. However, this will still temporarily store the route segments for 30s to allow instant navigation between nested segments, such as tab bars, or back and forward navigation. Visited routes will still be cached.

akshit-mdr commented 1 year ago

I am fetching data on server for /user route, now, I have /user/edit to edit the data of user which is also a server side component. When the data in edit route is updated, user is redirected back to /user route but even when using no-store, getting cached data only.

not able to use router.refresh() since all the entities involved are on server side.

What can be done here -

  1. Should I move all components to client side?
  2. is there any tagging and revalidation method for achieving this?

Further, I am integrating zustand store with next 13 server side components, and not able to fetch state in server, so, have to make api call without checking if the data is already present in store. How can we take advantage of global store with next 13 server component?

Can someone help here, thanks

lmf-git commented 1 year ago

I'm going to wildly speculate that this is due to Next/Vercel not wanting to send so many requests from their side.

ghcpuman902 commented 1 year ago

I've been having issues with caching as well but managed to solve now:

if the fetched source / API also has cache mechanism like stale-while-revalidate, when the page read subsequent fetch results as not changing, it will cache and not try to call the API again.

Keep your API end point dynamic and not cached, and cache at the root level of your app to solve this issue.

Basically don't revalidate = 3600 on both your page.js AND then fetch in page.js AND your route.ts, best to make page.js dynamic with searchParams, use next: { revalidate: 3600 } in your fetch, then make sure your API route is complete dynamic (revalidate = 0 works for me but if not try using request object.

If people want to see some code please let me know, I'll try to add them when I'm done travelling on this train.

nevenhsu commented 1 year ago

I've been having issues with caching as well but managed to solve now:

if the fetched source / API also has cache mechanism like stale-while-revalidate, when the page read subsequent fetch results as not changing, it will cache and not try to call the API again.

Keep your API end point dynamic and not cached, and cache at the root level of your app to solve this issue.

Basically don't revalidate = 3600 on both your page.js AND then fetch in page.js AND your route.ts, best to make page.js dynamic with searchParams, use next: { revalidate: 3600 } in your fetch, then make sure your API route is complete dynamic (revalidate = 0 works for me but if not try using request object.

If people want to see some code please let me know, I'll try to add them when I'm done travelling on this train.

Please share some code with us, thx a lot!

louispelarrey commented 1 year ago

Still have this issue after upgrading to next.js 14.0.2

kirkstrobeck commented 1 year ago

My issue turned out to be a race condition that I was attributing to cache

kevincobain2000 commented 1 year ago

On Nextjs14, tags work!

app/actions.ts

'use server'

import { revalidateTag } from 'next/cache'

export default async function action() {
  revalidateTag('api')
}

and

    fetch(process.env.NEXT_PUBLIC_API_URL as string, {
        next: {
            tags: ["api"]
        }
    })
mirano-galijasevic commented 1 year ago

Ok, so how can we turn off the damn caching in the end? I tested with 14.0.2 and it is not working at all, nothing has changed. This practically renders the whole framework unusable. I cannot understand what the thinking behind this was, who would ever go and weld in the caching into the framework and saying: "hey, let's be faster, never mind if data will be wrong, who cares about that!". The caching should be turned off by default, and then you configure it if you want it, not the other way around. So, what are you doing to go around this? I see the hacks with adding bogus search params, revalidate, force-dynamic and other hacks do not work, so what do you do? Converting all components to client? Going down to 13.3.4 and then re-writing all the code that will not be working? What? Since nobody is even mentioning if this will even be fixed.

UPDATE 2023-12-01: Scratch that, using 13.3.4 changes nothing. I then tested the same situation with a Page router in Next 14.0.3, and it worked normally. Here's how I tested: I used 3 pages, one.tsx, two.tsx, and three.tsx. In two.tsx, I set the Vercel KV value to '2', and in three.tsx to '3'. Then, in one.tsx, I read that value. Moving between pages always shows the latest value with a page router. I read the value in GetServerSideProps, passing the props down to the component to display it (client-side, of course). With the app router, I can set the value to '2', read it properly in one.tsx, then set it to '3' and go back to one.tsx; it would always just show '2'. Hitting the refresh button in the browser would change the value, but then the data becomes stale again.

So, this issue is related to the router, not Request Memoization or Data Cache. The app router keeps the entire payload of the server component in the browser, calling this the 'router cache', which is kept for the duration of the session. There's another cache, the 'full router cache', persisted on the server. The full router cache applies only to static routes, while the router cache applies to both static and dynamic routes. I don't think trying to trick it with bogus parameters would help. The router cache in the browser invalidates if you hit refresh (not a real solution, as we can't tell users to keep refreshing after moving to each new page), or after 30 seconds (which is why I see correct values if I wait, but then it gets stale again).

There seems to be no solution, as the router cache cannot be removed. It always caches for at least 30 seconds, and using prefetch=true or router.prefetch will even increase it to 5 minutes. Possible but undesirable solutions include router.refresh, cookies.set, or revalidateTag (which works, but what if you're not fetching anything?).

Rodo1foBandeira commented 1 year ago

In the Next 14 too. I tried several ways in the component app/pessoa/page.tsx

import Link from 'next/link';
import { getPessoas } from '../../services/pessoa'

export const dynamic = 'force-dynamic';
export const revalidate = 0;
export const fetchCache = 'force-no-store';
export const runtime = 'edge';

export default async function Pessoas(){
    const tst= await getPessoas();
    return (
        <ul>
            { tst.map(x => (<li key={x.nome}><Link href={`/pessoa/${x.id}`}>{x.nome}</Link></li>))}
        </ul>
    )
}

In the fetch that my component consumes src/services/pessoa.tsx

import { Pessoa } from "@/models/pessoa";

export async function getPessoas (): Promise<Pessoa[]> {
    const reponse = await fetch('http://localhost:3004/pessoa',
        {
            cache: "no-store"
        }
    );
    return await reponse.json();
}
hozunlee commented 12 months ago
image

x-vercel-cache: HIT xxxxxx

nextjs + vercel

The fetch works properly locally, but there is a problem when I distribute it to the vercel.

'/' The route only gets new data when redistributed, and when you reload the page, 304 appears.

Funny thing is that '/post/[id]' becomes a prerender and gets the data right.

x-vercel-cache: HIT

I always want to get an x-vercel-cache: Miss.

Has anyone solved it?

https://hololog.dev/post/21 이걸로 해결했습니다.

ChoaibMouhrach commented 11 months ago

Same issue 14.O.4

Edit by maintainers: Comment was automatically minimized because it was considered unhelpful. (If you think this was by mistake, let us know). Please only comment if it adds context to the issue. If you want to express that you have the same problem, use the upvote 👍 on the issue description or subscribe to the issue for updates. Thanks!

fmsf commented 11 months ago

Was hitting the same issue yesterday. Just documenting my findings in case it helps other people. I tried multiple different combos of:

export const fetchCache = 'force-no-store'
export const revalidate = 0 // seconds
export const dynamic = 'force-dynamic'

And in the network headers I could see that vercel was applying the header to the requests comming from the browser. However my cache hits where happening between the server function and my external server.

My guess is that this was happening:

Browser ---- (no cache applies correctly) --> serverless function that does api calls then serverless function that does api calls --- (no cache is no longer applied because I only requested it on the front end) ---> external API

Browsing among the many threads with the common topic of "Disabling cache doesn't work" I eventually found this one: Fetch caching in Server actions: https://github.com/vercel/next.js/discussions/50045 which pointed to two actions:

1 - Upgrade to next 14 2 - Use the new unstable_noStore to disable caching in individual functions within functions within a .ts|.tsx file tagged with "use server". https://nextjs.org/docs/app/api-reference/functions/unstable_noStore

This has worked for me. I hope it helps others.

ChoaibMouhrach commented 11 months ago

A solution I found is using unstable_noStore with router.refresh


import { Suspense } from "react";
import { Table } from "./table";
import { unstable_noStore } from "next/cache";

const Products = () => {
  unstable_noStore();

  return (
    <Suspense fallback="loading...">
      <Table />
    </Suspense>
  );
};

export default Products;

"use client";

import { Button } from "@/components/ui/button";
import { usePathname } from "next/navigation";
import { useMemo } from "react";
import { useRouter } from "next/navigation";

interface Navigation {
  name: string;
  href: string;
}

const navigations: Navigation[] = [
  {
    name: "Dashboard",
    href: "/",
  },
  {
    name: "Products",
    href: "/products",
  },
  {
    name: "Sales",
    href: "/sales",
  },
  {
    name: "Sales Returns",
    href: "/sales-returns",
  },
  {
    name: "Suppliers",
    href: "/suppliers",
  },
  {
    name: "Purchases",
    href: "/purchases",
  },
  {
    name: "Purchases Returns",
    href: "/purchases-returns",
  },
  {
    name: "Settings",
    href: "/settings",
  },
];

interface SideBarItemProps {
  navigation: Navigation;
  pathName: string;
}

const SideBarItem: React.FC<SideBarItemProps> = ({ navigation, pathName }) => {
  const router = useRouter();

  const active = useMemo(() => {
    let active = false;

    if (`${pathName}/`.startsWith(`${navigation.href}/`)) {
      active = true;

      if (navigation.href === "/" && pathName !== "/") {
        active = false;
      }
    }

    return active;
  }, [navigation.href, pathName]);

  const navigate = () => {
    router.push(navigation.href);
    router.refresh();
  };

  return (
    <Button
      className="justify-start"
      variant={active ? "secondary" : "ghost"}
      onClick={() => navigate()}
      size="sm"
    >
      {navigation.name}
    </Button>
  );
};

export const SideBar = () => {
  const pathName = usePathname();

  return (
    <div className="w-72 shrink-0 border-r p-4 flex flex-col gap-4">
      {navigations.map((navigation, index) => (
        <SideBarItem pathName={pathName} navigation={navigation} key={index} />
      ))}
    </div>
  );
};
realrufans commented 11 months ago

I had the same issue with my code. In my case, I wasn't making use of Link component. So, users aren't expected to navigate from one page to the other. but still, my api data was being cached in page.tsx.

Here's what i did to disable the cache. or at least prevent if from returning stale data from api call on every call.

I added export const revalidate = 0; at the top level of my route.ts file in the api folder.