search

serverless-nextjs / serverless-next.js

⚡ Deploy your Next.js apps on AWS Lambda@Edge via Serverless Components
MIT License
4.43k stars 451 forks source link

Tracking ISR issues #1098

Closed dphang closed 3 years ago

dphang commented 3 years ago

Just creating this to track any bugs or issues with the newly-released Incremental Static Regeneration feature -

dphang commented 3 years ago

@kirkness the ISR tests have been fairly flaky (mostly failing but sometimes 1/3 or 2/3 pass), if you have a chance please take a look otherwise I can check it out later on the weekend: https://github.com/serverless-nextjs/serverless-next.js/actions/runs/863012949

kirkness commented 3 years ago

Hi @dphang, keen to have a look at this, however, I won't be able to get much time on it until early next week, likely Tuesday.

kirkness commented 3 years ago

Related: https://github.com/serverless-nextjs/serverless-next.js/pull/1099

dphang commented 3 years ago

No worries, yeah it would be good to add more test coverage. The last few runs seem to not be flaky, though.

Miroko commented 3 years ago

Not sure if I should open an issue about this or write straight to this thread...

We are getting the following debug log information when trying to deploy the "@sls-next/serverless-component": "1.20.0-alpha.16" (at the time of writing the latest alpha version) to AWS.

Our deploy script uses an existing serverless state made with previous deployment. That successfully deployed version uses SSR only, otherwise the configuration and component version are exactly the same. We are just trying to switch to use ISR.

Any help with this would be appreciated and thanks for the great work you have been doing for the ISR support!

  DEBUG ─ The QueueUrl has changed
  DEBUG ─ Deleting previous queue
  error:
  MissingRequiredParameter: Missing required key 'QueueUrl' in params
    at ParamValidator.fail (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/param_validator.js:50:37)
    at ParamValidator.validateStructure (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/param_validator.js:61:14)
    at ParamValidator.validateMember (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/param_validator.js:88:21)
    at ParamValidator.validate (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/param_validator.js:34:10)
    at Request.VALIDATE_PARAMETERS (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/event_listeners.js:132:42)
    at Request.callListeners (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at callNextListener (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/sequential_executor.js:96:12)
    at /builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/event_listeners.js:86:9
    at finish (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/config.js:386:7)
    at /builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/config.js:404:9 {
  code: 'MissingRequiredParameter',
  time: 2021-06-01T11:55:41.540Z
}
a-game commented 3 years ago

Not sure if I should open an issue about this or write straight to this thread...

We are getting the following debug log information when trying to deploy the "@sls-next/serverless-component": "1.20.0-alpha.16" (at the time of writing the latest alpha version) to AWS.

Our deploy script uses an existing serverless state made with previous deployment. That successfully deployed version uses SSR only, otherwise the configuration and component version are exactly the same. We are just trying to switch to use ISR.

Any help with this would be appreciated and thanks for the great work you have been doing for the ISR support!

  DEBUG ─ The QueueUrl has changed
  DEBUG ─ Deleting previous queue
  error:
  MissingRequiredParameter: Missing required key 'QueueUrl' in params
    at ParamValidator.fail (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/param_validator.js:50:37)
    at ParamValidator.validateStructure (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/param_validator.js:61:14)
    at ParamValidator.validateMember (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/param_validator.js:88:21)
    at ParamValidator.validate (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/param_validator.js:34:10)
    at Request.VALIDATE_PARAMETERS (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/event_listeners.js:132:42)
    at Request.callListeners (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at callNextListener (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/sequential_executor.js:96:12)
    at /builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/event_listeners.js:86:9
    at finish (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/config.js:386:7)
    at /builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/config.js:404:9 {
  code: 'MissingRequiredParameter',
  time: 2021-06-01T11:55:41.540Z
}

I'm having the same problem here and I have a theory about the cause (It's really just a shot in the dark).

I think the errors occurs at 

    // triggerStaticRegeneration.ts
    await sqs.send(
        new SendMessageCommand({
            QueueUrl: `https://sqs.${region}.amazonaws.com/${bucketName}.fifo`,
            ...
        })
    );

bucketName comes from s3BucketNameFromEventRequest.ts which tries to parse it from an AWSLambda.CloudFrontRequest

export const s3BucketNameFromEventRequest = (
  request: AWSLambda.CloudFrontRequest
): string | undefined => {
  const { region, domainName } = request.origin?.s3 || {};
  return domainName?.replace(`.s3.${region}.amazonaws.com`, "");
};

Looking at the the docs for AWSLambda.CloudFrontRequest the s3 origin looks like this: awsexamplebucket.s3.us-east-1.amazonaws.com

This will result in s3BucketNameFromEventRequest returning awsexamplebucket as the bucketName.

So QueueUrl would be https://sqs.us-east-1.amazonaws.com/awsexamplebucket.fifo

That QueueUrl would be incorrect in my case. The created SQS has this url: https://sqs.us-east-1.amazonaws.com/12345/awsexamplebucket.fifo

Conclusion

I suspect that the account id 12345 is missing from the parsed bucketName but needed for a valid SQS url.

valurhrafn commented 3 years ago

Some issues I have come across implementing the ISR functionality:

kirkness commented 3 years ago

I'm having the same problem here and I have a theory about the cause (It's really just a shot in the dark).

I think the errors occurs at

    // triggerStaticRegeneration.ts
    await sqs.send(
        new SendMessageCommand({
            QueueUrl: `https://sqs.${region}.amazonaws.com/${bucketName}.fifo`,
            ...
        })
    );

bucketName comes from s3BucketNameFromEventRequest.ts which tries to parse it from an AWSLambda.CloudFrontRequest

export const s3BucketNameFromEventRequest = (
  request: AWSLambda.CloudFrontRequest
): string | undefined => {
  const { region, domainName } = request.origin?.s3 || {};
  return domainName?.replace(`.s3.${region}.amazonaws.com`, "");
};

Looking at the the docs for AWSLambda.CloudFrontRequest the s3 origin looks like this: awsexamplebucket.s3.us-east-1.amazonaws.com

This will result in s3BucketNameFromEventRequest returning awsexamplebucket as the bucketName.

So QueueUrl would be https://sqs.us-east-1.amazonaws.com/awsexamplebucket.fifo

That QueueUrl would be incorrect in my case. The created SQS has this url: https://sqs.us-east-1.amazonaws.com/12345/awsexamplebucket.fifo

Conclusion

I suspect that the account id 12345 is missing from the parsed bucketName but needed for a valid SQS url.

I think its this line which is run on deployment: https://github.com/serverless-nextjs/serverless-next.js/blob/d5bbdc6d395ae732ec0757744482bf8bc25e820f/packages/serverless-components/aws-sqs/serverless.js#L70

I'll put in a fix for this.

mikejcooper commented 3 years ago

Revalidate not working as expected (I think).

Running 1.20.0-alpha.18 in eu-west-2, the request sequence/behaviour is as follows:

nth request is when Revalidate / s-maxage is reached.

Below is the small example I ran the text on: 

export const getStaticProps: GetStaticProps = async () => {
  const apolloClient = initializeApollo()

  const getAuctionsResult = await apolloClient.mutate<GetAuctionsQuery, GetAuctionsQueryVariables>({
    mutation: GetAuctionsDocument,
  })

  return {
    revalidate: 15,
    props: {
      date: new Date().toLocaleString(),
      auctions: JSON.parse(JSON.stringify(getAuctionsResult.data?.auction)) || [],
    },
  }
}

const AboutPage = (props: { date: string; auctions: any[] }) => (
  <div>
    <p>This is the about page, and the time is about {props.date}</p>
    <p></p>
    <p>This is the about page, and the time is about {JSON.stringify(props.auctions[0])}</p>
  </div>
)

Also having issues on other more complex pages where it's a cache miss every time.

Any thoughts?

dphang commented 3 years ago

@mikejcooper Unfortunately I'm not really familiar with apollo client, is it a problem without it as well?

danjiro commented 3 years ago

Great work on this so far guys. I have some questions. Is revalidate supposed to be functional with 1.20.0-alpha.23? I tried to deploy with it but it doesn't seem to be working correctly. The SQS isn't sending/receiving anything and im getting some 503 errors in console as it tries to fetch index.json multiple times. I have my revalidate on my index.js file and duplicated it to a test.js file and when accessing the test route also get 503:

503 ERROR
The request could not be satisfied.
The Lambda function associated with the CloudFront distribution is invalid or doesn't have the required permissions. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.

Am i missing any specific configurations that is needed?

dphang commented 3 years ago

Great work on this so far guys. I have some questions. Is revalidate supposed to be functional with 1.20.0-alpha.23? I tried to deploy with it but it doesn't seem to be working correctly. The SQS isn't sending/receiving anything and im getting some 503 errors in console as it tries to fetch index.json multiple times. I have my revalidate on my index.js file and duplicated it to a test.js file and when accessing the test route also get 503:


503 ERROR

The request could not be satisfied.

The Lambda function associated with the CloudFront distribution is invalid or doesn't have the required permissions. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.

If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.

Am i missing any specific configurations that is needed?

Please post the logs (503 is very generic and doesn't tell anything about the actual issue). Without knowing anything, I'm gonna guess there could be a permissions issue

teban3010 commented 3 years ago

Hi, I'm having the same issue, the error shows on the browser and there are no logs, or at least I couldn't find any.

In my case, I'm doing the deployment on a github action, here is the serverless.yml file:

name: api-hub-web

nextApp:
  component: "@sls-next/serverless-component@1.20.0-alpha.23"
  inputs:
    bucketName: web-dev-s3
    description: '*lambda-type*@Edge for dev-api-hub-web'
    name:
      defaultLambda: web-dev-lambda
      apiLambda: web-dev-lambda
    publicDirectoryCache: false
    runtime:
      defaultLambda: 'nodejs12.x'
      apiLambda: 'nodejs12.x'
    logLambdaExecutionTimes: true
    minifyHandlers: true
    memory: 2048
    timeout: 30
    policy: 'arn:aws:iam::<acount>:policy/next-lambdas-execution-policy'

The deployment permissions are the same as https://github.com/serverless-nextjs/serverless-next.js#aws-permissions-for-deployment and the lambda policy is the following:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:*",
                "logs:CreateLogStream",
                "sqs:*",
                "logs:CreateLogGroup",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        }
    ]
}

I'm using serverless 2.46.0 and next 10.2.0.

adamdottv commented 3 years ago

In testing ISR with the https://github.com/dapperlabs/nextjs-isr-testing repo, I also am seeing 503 errors on retrieving JSON data files. I found this in the default lambda logs:

ERROR   Invoke Error    
{
    "errorType": "TypeError",
    "errorMessage": "serdePlugin.lib.decodeHTML is not a function",
    "$metadata": {
        "attempts": 1,
        "totalRetryDelay": 0
    },
    "stack": [
        "TypeError: serdePlugin.lib.decodeHTML is not a function",
        "    at Object.tagValueProcessor (/var/task/index-b8afa7fd.js:3725:111)",
        "    at processTagValue (/var/task/serdePlugin-45449631.js:588:19)",
        "    at Object.getTraversalObj (/var/task/serdePlugin-45449631.js:709:31)",
        "    at Object.exports.parse (/var/task/serdePlugin-45449631.js:1742:39)",
        "    at /var/task/index-b8afa7fd.js:3720:48",
        "    at processTicksAndRejections (internal/process/task_queues.js:97:5)"
    ]
}
kirkness commented 3 years ago

Hi, I'm having the same issue, the error shows on the browser and there are no logs, or at least I couldn't find any.

In my case, I'm doing the deployment on a github action, here is the serverless.yml file:

name: api-hub-web

nextApp:
  component: "@sls-next/serverless-component@1.20.0-alpha.23"
  inputs:
    bucketName: web-dev-s3
    description: '*lambda-type*@Edge for dev-api-hub-web'
    name:
      defaultLambda: web-dev-lambda
      apiLambda: web-dev-lambda
    publicDirectoryCache: false
    runtime:
      defaultLambda: 'nodejs12.x'
      apiLambda: 'nodejs12.x'
    logLambdaExecutionTimes: true
    minifyHandlers: true
    memory: 2048
    timeout: 30
    policy: 'arn:aws:iam::<acount>:policy/next-lambdas-execution-policy'

The deployment permissions are the same as https://github.com/serverless-nextjs/serverless-next.js#aws-permissions-for-deployment and the lambda policy is the following:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:*",
                "logs:CreateLogStream",
                "sqs:*",
                "logs:CreateLogGroup",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        }
    ]
}

I'm using serverless 2.46.0 and next 10.2.0.

Hey @teban3010, make sure to switch to the region in the console that you are viewing the failing page from, a few more deets here: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-cloudwatch-metrics-logging.html#lambda-cloudwatch-logs

teban3010 commented 3 years ago

Hey @teban3010, make sure to switch to the region in the console that you are viewing the failing page from, a few more deets here: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-cloudwatch-metrics-logging.html#lambda-cloudwatch-logs

@kirkness thanks for that, I have the same error as @adamelmore on the lambdas

ERROR Invoke Error

{
    "errorType": "TypeError",
    "errorMessage": "n.lib.decodeHTML is not a function",
    "$metadata": {
        "attempts": 1,
        "totalRetryDelay": 0
    },
    "stack": [
        "TypeError: n.lib.decodeHTML is not a function",
        "    at Object.tagValueProcessor (/var/task/index-b8afa7fd.js:1:59241)",
        "    at y (/var/task/serdePlugin-45449631.js:1:7924)",
        "    at Object.h.getTraversalObj (/var/task/serdePlugin-45449631.js:1:9674)",
        "    at Object.t.parse (/var/task/serdePlugin-45449631.js:1:21540)",
        "    at /var/task/index-b8afa7fd.js:1:59096",
        "    at processTicksAndRejections (internal/process/task_queues.js:97:5)"
    ]
}

UPDATE: This happens on an update of a deployment that uses a previous version of the component and on a new one as well.

dphang commented 3 years ago

If you check affected dist files, it looks like it's trying to call a non-existent function:

index-b8afa7fd.js

var parseBody = function (streamBody, context) {
    return collectBodyString(streamBody, context).then(function (encoded) {
        if (encoded.length) {
            var parsedObj = serdePlugin.parser.parse(encoded, {
                attributeNamePrefix: "",
                ignoreAttributes: false,
                parseNodeValue: false,
                trimValues: false,
                tagValueProcessor: function (val, tagName) { return (val.trim() === "" ? "" : serdePlugin.lib.decodeHTML(val)); },
            });
            var textNodeName = "#text";
            var key = Object.keys(parsedObj)[0];
            var parsedObjToReturn = parsedObj[key];
            if (parsedObjToReturn[textNodeName]) {
                parsedObjToReturn[key] = parsedObjToReturn[textNodeName];
                delete parsedObjToReturn[textNodeName];
            }
            return serdePlugin.getValueFromTextNode(parsedObjToReturn);
        }
        return {};
    });
};

When I checked serdePlugin.lib it seems lib is initialized empty in the serdePlugin.js file. Although the decodeHTML function should actually be from the entities library. See source: https://github.com/aws/aws-sdk-js-v3/blob/fd6b7ebd7d3fc4a2ceb3b016404de73413e1fb2f/clients/client-sqs/protocols/Aws_query.ts#L104.

Not sure why it's not being bundled correctly by Rollup?

dphang commented 3 years ago

Seems there might be a problem with @rollup/plugin-commonjs: 19.0.0, after 18.1.0 it broke the bundling and those decode functions are no longer bundled: https://github.com/rollup/plugins/blob/master/packages/commonjs/CHANGELOG.md. Need to investigate why

Ref PR: https://github.com/rollup/plugins/pull/658

It can't be easily caught in unit tests (Rollup bundling issue), but possibly should add an e2e test for this? cc: @kirkness. Meanwhile I will downgrade to 18.1.0, which I tested and it bundles correctly. And also exclude it from dependabot updates.

dphang commented 3 years ago

The above fix is already published, please try it out and see if it works for you.

teban3010 commented 3 years ago

I've tried out, it works at first but then I receive the same error on the default lambda. The only difference is that now if I press F5, some times I see the page and some times I have a 503 error. On the network tab, I see a lot of 503 of the other pages json. The other thing is that the page is not being updated after the revalidate time.

ERROR Invoke Error

{
    "errorType": "TypeError",
    "errorMessage": "n.lib.decodeHTML is not a function",
    "$metadata": {
        "attempts": 1,
        "totalRetryDelay": 0
    },
    "stack": [
        "TypeError: n.lib.decodeHTML is not a function",
        "    at Object.tagValueProcessor (/var/task/index-b8afa7fd.js:1:59241)",
        "    at y (/var/task/serdePlugin-45449631.js:1:7924)",
        "    at Object.h.getTraversalObj (/var/task/serdePlugin-45449631.js:1:9674)",
        "    at Object.t.parse (/var/task/serdePlugin-45449631.js:1:21540)",
        "    at /var/task/index-b8afa7fd.js:1:59096",
        "    at processTicksAndRejections (internal/process/task_queues.js:97:5)"
    ]
}
dphang commented 3 years ago

I've tried out, it works at first but then I receive the same error on the default lambda. The only difference is that now if I press F5, some times I see the page and some times I have a 503 error. On the network tab, I see a lot of 503 of the other pages json. The other thing is that the page is not being updated after the revalidate time.

ERROR Invoke Error

{
    "errorType": "TypeError",
    "errorMessage": "n.lib.decodeHTML is not a function",
    "$metadata": {
        "attempts": 1,
        "totalRetryDelay": 0
    },
    "stack": [
        "TypeError: n.lib.decodeHTML is not a function",
        "    at Object.tagValueProcessor (/var/task/index-b8afa7fd.js:1:59241)",
        "    at y (/var/task/serdePlugin-45449631.js:1:7924)",
        "    at Object.h.getTraversalObj (/var/task/serdePlugin-45449631.js:1:9674)",
        "    at Object.t.parse (/var/task/serdePlugin-45449631.js:1:21540)",
        "    at /var/task/index-b8afa7fd.js:1:59096",
        "    at processTicksAndRejections (internal/process/task_queues.js:97:5)"
    ]
}

Based on the hash on your latest log of the affected file: index-b8afa7fd.js it looks to be you are still using the same version as before (it's same as reported above in all logs, before the fix). The new hashes of bundled files where SQS is invoked in the fixed version (which has the decodeHTML function) should be:

Please check if you had updated properly to the latest alpha version, if you specify the latest version exactly in serverless.yml file, I think it should have no problems (not sure if you use just alpha tag from npm, there could potentially be caching problems?). The latest version is 1.20.0-alpha.25, you can always find in npm: https://www.npmjs.com/package/@sls-next/serverless-component/v/1.20.0-alpha.25. From the version tab I see 0 downloads so far of the latest version (either npm is not updated so fast or there are really no downloads yet). Also it does take some time for function updates to propagate?

teban3010 commented 3 years ago

You are right, I used version 24. Now it's not returning 503, but it's not doing the ISR, I added a date property with revalidate: 10 but I alway have the same date. When I checked the logs of the lambda that has de SQS trigger I have this error

ERROR   Invoke Error    
{
    "errorType": "Error",
    "errorMessage": "Cannot find module './pages/.js'\nRequire stack:\n- /var/task/index.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js",
    "code": "MODULE_NOT_FOUND",
    "requireStack": [
        "/var/task/index.js",
        "/var/runtime/UserFunction.js",
        "/var/runtime/index.js"
    ],
    "stack": [
        "Error: Cannot find module './pages/.js'",
        "Require stack:",
        "- /var/task/index.js",
        "- /var/runtime/UserFunction.js",
        "- /var/runtime/index.js",
        "    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:815:15)",
        "    at Function.Module._load (internal/modules/cjs/loader.js:667:27)",
        "    at Module.require (internal/modules/cjs/loader.js:887:19)",
        "    at require (internal/modules/cjs/helpers.js:74:18)",
        "    at /var/task/index.js:1:4565",
        "    at Array.map (<anonymous>)",
        "    at Runtime.exports.handler (/var/task/index.js:1:4409)",
        "    at Runtime.handleOnce (/var/runtime/Runtime.js:66:25)"
    ]
}
dphang commented 3 years ago

Not sure which line is failing but it's strange to be requiring a pages/.js file, not sure why it's coming empty.

Could you please unminify the Lambda so it's easier to understand which line is failing in the SQS lambda (regeneration). And also post how your pages directory structure looks like, manifests etc if possible. Guessing it's because of the SQS-triggered lambda just tries to require the page path passed from the SQS message data after it gets triggered here: https://github.com/serverless-nextjs/serverless-next.js/blob/135e00665162c5f393cedc839ec107fead7be519/packages/libs/lambda-at-edge/src/default-handler.ts#L287-L292

Sending event: https://github.com/serverless-nextjs/serverless-next.js/blob/135e00665162c5f393cedc839ec107fead7be519/packages/libs/lambda-at-edge/src/lib/triggerStaticRegeneration.ts#L42-L58

Parsing the event and requiring page: https://github.com/serverless-nextjs/serverless-next.js/blob/135e00665162c5f393cedc839ec107fead7be519/packages/libs/lambda-at-edge/src/regeneration-handler.ts#L13-L23

I wonder if there's a special character that is not being passed properly through SQS message body?

teban3010 commented 3 years ago

I believe that I remove the minify from the serverless.yml file, but I see the exact same error. This is my serverless.yml file

name: web

nextApp:
  component: "@sls-next/serverless-component@1.20.0-alpha.25"
  inputs:
    bucketName: web-s3
    description: '*lambda-type*@Edge for web'
    name:
      defaultLambda: web-default-lambda
      apiLambda: web-api-lambda
      imageLambda: web-image-lambda
    publicDirectoryCache: false
    runtime:
      defaultLambda: 'nodejs12.x'
      apiLambda: 'nodejs12.x'
    logLambdaExecutionTimes: true
    # minifyHandlers: true
    memory: 2048
    timeout: 30
    policy: 'arn:aws:iam::599903031295:policy/next-lambdas-execution-policy'

And this is my pages directory structure:

.
├── pages
│   ├── _app.tsx
│   ├── _document.tsx
│   ├── account.tsx
│   ├── api
│   │   └── auth
│   │       └── [...nextauth].ts
│   ├── contact.tsx
│   ├── index.tsx
│   └── products.tsx

I'm not sure where can I get the manifest.

dphang commented 3 years ago

@teban3010 we need which line it is failing at. Manifests are the JSON files generated in .serverless_nextjs/*-lambda directory, they tell the Lambda how to route. Basically the point to know that is so we can figure out:

  1. Which line of code is it failing exactly
  2. What page name is being accessed and the manifests also help to see if there is some issue generating them (they are used to do the routing).

If you could provide some repro step as well (or a repo) then I can also debug myself. Otherwise it's hard to guess where is failing.

teban3010 commented 3 years ago

Sorry, I didn't get that at first. I'm doing all the deployment using a github action, so I didn't have access to those files because were created on the server.

Now, I did the deployment from my machine, and got those files. Unfortunately, I cannot provide access to the repository, but I will try to put something in place so you can test it out.

For the deployment, I'm just running npx serverless and the component version is on the serverless.yml file. I'm running it from macOS.

The .serverless_next folder structure is the following:

.
├── api-lambda
│   ├── index-9e574644.js
│   ├── index.js
│   ├── manifest.json
│   ├── pages
│   │   └── api
│   │       └── auth
│   │           └── [...nextauth].js
│   └── routes-manifest.json
├── assets
│   ├── BUILD_ID
│   ├── _next
│   │   ├── data
│   │   │   └── VYqAiP3KsZSxRrAqToc5l
│   │   │       ├── contact.json
│   │   │       ├── index.json
│   │   │       └── products.json
│   │   └── static
│   │       ├── VYqAiP3KsZSxRrAqToc5l
│   │       │   ├── _buildManifest.js
│   │       │   └── _ssgManifest.js
│   │       ├── chunks
│   │       │   ├── 1bfc9850.c8a70c1a6d80895144eb.js
│   │       │   ├── 252f366e.cc4df410de28b33613de.js
│   │       │   ├── 68d96d8dea06f46304db45ccefbfe2e77afe930f.11e556bb04dd6b857ec2.js
│   │       │   ├── 9f8cf9a5fe6f78b75cd4f3950f8751eae6026112.e24e05a6d9f0cabdbed6.js
│   │       │   ├── commons.0e4f5fcc136d6b919420.js
│   │       │   ├── framework.8f5c9b5ad9ab0f9b0592.js
│   │       │   ├── main-b98586f123ef58bc77aa.js
│   │       │   ├── pages
│   │       │   │   ├── _app-49a79c217e9d61538824.js
│   │       │   │   ├── _error-78a9006d8ba77f5d833a.js
│   │       │   │   ├── account-f0837073fb5202f16feb.js
│   │       │   │   ├── contact-7525176d3691a02fc984.js
│   │       │   │   ├── index-722e09bd7f2d74d04323.js
│   │       │   │   └── products-85e1e7200514c2407f11.js
│   │       │   ├── polyfills-4f4acd756cef4fe6da1b.js
│   │       │   └── webpack-50bee04d1dc61f8adf5b.js
│   │       ├── css
│   │       │   ├── 91c186b9d9397e7d4bb4.css
│   │       │   └── b32593cd38d5156699c5.css
│   │       └── media
│   │           ├── roboto-all-400-normal.b5a0a0df312d3423493b163e9675488c.woff
│   │           ├── roboto-cyrillic-400-normal.3605d18d8a4ce5bd39c3d32119574854.woff2
│   │           ├── roboto-cyrillic-ext-400-normal.dd55ea0a2e90d5d86e15444a623c385a.woff2
│   │           ├── roboto-greek-400-normal.352cc77aaed5208ce13c79cf9c89892f.woff2
│   │           ├── roboto-greek-ext-400-normal.bc7ace6e50c7d9d4d95923b61b1de59c.woff2
│   │           ├── roboto-latin-400-normal.176f8f5bd5f02b3abfcf894955d7e919.woff2
│   │           ├── roboto-latin-ext-400-normal.dcc07bcfcd475f34f20b2c1265bd1265.woff2
│   │           └── roboto-vietnamese-400-normal.52cebac009baf5592e7d0ee53d8e45ed.woff2
│   ├── public
│   │   ├── android-chrome-192x192.png
│   │   ├── android-chrome-512x512.png
│   │   ├── apple-touch-icon.png
│   │   ├── email
│   │   │   ├── hub_1.png
│   │   │   ├── hub_logo_blue.png
│   │   │   └── logo_white.png
│   │   ├── favicon-16x16.png
│   │   ├── favicon-32x32.png
│   │   ├── favicon.ico
│   │   ├── icons
│   │   │   ├── ask.svg
│   │   │   ├── business.svg
│   │   │   ├── clipboard.svg
│   │   │   ├── contact.svg
│   │   │   ├── developer.svg
│   │   │   ├── status.svg
│   │   │   └── word-balloon.svg
│   │   ├── illustrations
│   │   │   ├── building.svg
│   │   │   ├── home_page_1.png
│   │   │   ├── home_page_2.png
│   │   │   ├── news.svg
│   │   │   └── supplier.svg
│   │   ├── logo-blue.png
│   │   ├── logo.svg
│   │   └── site.webmanifest
│   └── static-pages
│       └── VYqAiP3KsZSxRrAqToc5l
│           ├── 404.html
│           ├── contact.html
│           ├── index.html
│           └── products.html
├── default-lambda
│   ├── GetObjectCommand-4007ef8b.js
│   ├── PutObjectCommand-ee4a4ff3.js
│   ├── S3Client-2ce44c80.js
│   ├── _commonjsHelpers-429052b6.js
│   ├── configurations-e825ea6c.js
│   ├── defaultRoleAssumers-9657d50f.js
│   ├── httpRequest-31970d1c.js
│   ├── index-0737fe4f.js
│   ├── index-084130cb.js
│   ├── index-246324a8.js
│   ├── index-4b0544e0.js
│   ├── index-855e9789.js
│   ├── index-891c56ba.js
│   ├── index-9e574644.js
│   ├── index-c10a6c3c.js
│   ├── index.js
│   ├── manifest.json
│   ├── pages
│   │   ├── _error.js
│   │   ├── account.js
│   │   ├── contact.js
│   │   ├── index.js
│   │   └── products.js
│   ├── prerender-manifest.json
│   ├── routes-manifest.json
│   └── serdePlugin-7f212178.js
├── image-lambda
└── regeneration-lambda
    ├── PutObjectCommand-847f6a70.js
    ├── S3Client-4495fe4b.js
    ├── configurations-e825ea6c.js
    ├── httpRequest-31970d1c.js
    ├── index-0737fe4f.js
    ├── index-8cd8cf24.js
    ├── index.js
    ├── manifest.json
    └── pages
        ├── _error.js
        ├── account.js
        ├── contact.js
        ├── index.js
        └── products.js

Regeneration-lambda manifest:

{
  "buildId": "VYqAiP3KsZSxRrAqToc5l",
  "pages": {
    "dynamic": [],
    "ssr": {
      "dynamic": {},
      "nonDynamic": {
        "/account": "pages/account.js",
        "/contact": "pages/contact.js",
        "/": "pages/index.js",
        "/products": "pages/products.js",
        "/_error": "pages/_error.js"
      }
    },
    "html": { "dynamic": {}, "nonDynamic": { "/404": "pages/404.html" } },
    "ssg": {
      "dynamic": {},
      "nonDynamic": {
        "/contact": { "initialRevalidateSeconds": 10, "srcRoute": null },
        "/products": { "initialRevalidateSeconds": 10, "srcRoute": null },
        "/": { "initialRevalidateSeconds": 10, "srcRoute": null }
      }
    }
  },
  "publicFiles": {
    "/android-chrome-192x192.png": "android-chrome-192x192.png",
    "/android-chrome-512x512.png": "android-chrome-512x512.png",
    "/apple-touch-icon.png": "apple-touch-icon.png",
    "/email/hub_1.png": "email/hub_1.png",
    "/email/hub_logo_blue.png": "email/hub_logo_blue.png",
    "/email/logo_white.png": "email/logo_white.png",
    "/favicon-16x16.png": "favicon-16x16.png",
    "/favicon-32x32.png": "favicon-32x32.png",
    "/favicon.ico": "favicon.ico",
    "/icons/ask.svg": "icons/ask.svg",
    "/icons/business.svg": "icons/business.svg",
    "/icons/clipboard.svg": "icons/clipboard.svg",
    "/icons/contact.svg": "icons/contact.svg",
    "/icons/developer.svg": "icons/developer.svg",
    "/icons/status.svg": "icons/status.svg",
    "/icons/word-balloon.svg": "icons/word-balloon.svg",
    "/illustrations/building.svg": "illustrations/building.svg",
    "/illustrations/home_page_1.png": "illustrations/home_page_1.png",
    "/illustrations/home_page_2.png": "illustrations/home_page_2.png",
    "/illustrations/news.svg": "illustrations/news.svg",
    "/illustrations/supplier.svg": "illustrations/supplier.svg",
    "/logo-blue.png": "logo-blue.png",
    "/logo.svg": "logo.svg",
    "/site.webmanifest": "site.webmanifest"
  },
  "trailingSlash": false,
  "domainRedirects": {},
  "enableHTTPCompression": false,
  "logLambdaExecutionTimes": true
}

Default-lambda manifest:

{
  "buildId": "VYqAiP3KsZSxRrAqToc5l",
  "pages": {
    "dynamic": [],
    "ssr": {
      "dynamic": {},
      "nonDynamic": {
        "/account": "pages/account.js",
        "/contact": "pages/contact.js",
        "/": "pages/index.js",
        "/products": "pages/products.js",
        "/_error": "pages/_error.js"
      }
    },
    "html": { "dynamic": {}, "nonDynamic": { "/404": "pages/404.html" } },
    "ssg": {
      "dynamic": {},
      "nonDynamic": {
        "/contact": { "initialRevalidateSeconds": 10, "srcRoute": null },
        "/products": { "initialRevalidateSeconds": 10, "srcRoute": null },
        "/": { "initialRevalidateSeconds": 10, "srcRoute": null }
      }
    }
  },
  "publicFiles": {
    "/android-chrome-192x192.png": "android-chrome-192x192.png",
    "/android-chrome-512x512.png": "android-chrome-512x512.png",
    "/apple-touch-icon.png": "apple-touch-icon.png",
    "/email/hub_1.png": "email/hub_1.png",
    "/email/hub_logo_blue.png": "email/hub_logo_blue.png",
    "/email/logo_white.png": "email/logo_white.png",
    "/favicon-16x16.png": "favicon-16x16.png",
    "/favicon-32x32.png": "favicon-32x32.png",
    "/favicon.ico": "favicon.ico",
    "/icons/ask.svg": "icons/ask.svg",
    "/icons/business.svg": "icons/business.svg",
    "/icons/clipboard.svg": "icons/clipboard.svg",
    "/icons/contact.svg": "icons/contact.svg",
    "/icons/developer.svg": "icons/developer.svg",
    "/icons/status.svg": "icons/status.svg",
    "/icons/word-balloon.svg": "icons/word-balloon.svg",
    "/illustrations/building.svg": "illustrations/building.svg",
    "/illustrations/home_page_1.png": "illustrations/home_page_1.png",
    "/illustrations/home_page_2.png": "illustrations/home_page_2.png",
    "/illustrations/news.svg": "illustrations/news.svg",
    "/illustrations/supplier.svg": "illustrations/supplier.svg",
    "/logo-blue.png": "logo-blue.png",
    "/logo.svg": "logo.svg",
    "/site.webmanifest": "site.webmanifest"
  },
  "trailingSlash": false,
  "domainRedirects": {},
  "enableHTTPCompression": false,
  "logLambdaExecutionTimes": true
}
dphang commented 3 years ago

@teban3010 thanks, yeah I think there github actions to upload build artifacts as well. I think I see the potential problem, is your static regeneration on the index page? I think "/" should actually be normalized to "/index" page but the code https://github.com/serverless-nextjs/serverless-next.js/blob/135e00665162c5f393cedc839ec107fead7be519/packages/libs/lambda-at-edge/src/regeneration-handler.ts#L13-L23 uses the pagePath

teban3010 commented 3 years ago

@dphang yes, I have static regeneration on the index page

teban3010 commented 3 years ago

I replaced the static regeneration with server side props and I see the same error on the regeneration lambda, this is the new manifest.

{
  "buildId": "wd-p0m1Z4vd39712xzaRx",
  "pages": {
    "dynamic": [],
    "ssr": {
      "dynamic": {},
      "nonDynamic": {
        "/account": "pages/account.js",
        "/contact": "pages/contact.js",
        "/": "pages/index.js",
        "/products": "pages/products.js",
        "/_error": "pages/_error.js"
      }
    },
    "html": { "dynamic": {}, "nonDynamic": { "/404": "pages/404.html" } },
    "ssg": {
      "dynamic": {},
      "nonDynamic": {
        "/contact": { "initialRevalidateSeconds": 10, "srcRoute": null },
        "/products": { "initialRevalidateSeconds": 10, "srcRoute": null }
      }
    }
  },
  "publicFiles": {
    "/android-chrome-192x192.png": "android-chrome-192x192.png",
    "/android-chrome-512x512.png": "android-chrome-512x512.png",
    "/apple-touch-icon.png": "apple-touch-icon.png",
    "/email/hub_1.png": "email/hub_1.png",
    "/email/hub_logo_blue.png": "email/hub_logo_blue.png",
    "/email/logo_white.png": "email/logo_white.png",
    "/favicon-16x16.png": "favicon-16x16.png",
    "/favicon-32x32.png": "favicon-32x32.png",
    "/favicon.ico": "favicon.ico",
    "/icons/ask.svg": "icons/ask.svg",
    "/icons/business.svg": "icons/business.svg",
    "/icons/clipboard.svg": "icons/clipboard.svg",
    "/icons/contact.svg": "icons/contact.svg",
    "/icons/developer.svg": "icons/developer.svg",
    "/icons/status.svg": "icons/status.svg",
    "/icons/word-balloon.svg": "icons/word-balloon.svg",
    "/illustrations/building.svg": "illustrations/building.svg",
    "/illustrations/home_page_1.png": "illustrations/home_page_1.png",
    "/illustrations/home_page_2.png": "illustrations/home_page_2.png",
    "/illustrations/news.svg": "illustrations/news.svg",
    "/illustrations/supplier.svg": "illustrations/supplier.svg",
    "/logo-blue.png": "logo-blue.png",
    "/logo.svg": "logo.svg",
    "/site.webmanifest": "site.webmanifest"
  },
  "trailingSlash": false,
  "domainRedirects": {},
  "enableHTTPCompression": false,
  "logLambdaExecutionTimes": true
}
dphang commented 3 years ago

@teban3010 yes because the page JS path was wrong for index page, no matter if it's for SSR or SSG. (We were manually constructing it and "/" was not properly accounted for and converted to "/index") I believe it should be fixed now in the latest alpha publish

teban3010 commented 3 years ago

Is latest 27? I updated it on the serverless.yml file but I have the same error

dphang commented 3 years ago

Ah I missed to update for data requests as well, let me update it and republish soon.

dphang commented 3 years ago

Fixed and published https://github.com/serverless-nextjs/serverless-next.js/pull/1241, let me know if it works for you now

danjiro commented 3 years ago

503 is gone for me now and index.json is coming back with 200 with the latest alpha. Revalidate however doesn't seem to be working still. I'm not sure what the reason is but SQS is only receiving a message once after a new build and first page load and no more after that even though revalidate is set at 10s

dphang commented 3 years ago

@danjiro are you getting a cache miss on that route after 10 seconds? It needs to miss the cache in order to hit the Lambda once again to kick off the SQS message. The revalidate cache control headers should be set to 10 second max for you (cached only on CF, not the browser) and also it uses expires header on S3 to keep track of revalidate state. But if you set a minTTL to something high in CloudFront behavior, it will override the cache control header. Or as another test, you can try to break the cache after 10 seconds by adding query param

You can see the logic here, once revalidate time has hit 0 seconds it will send a message to SQS: https://github.com/serverless-nextjs/serverless-next.js/blob/d693fe0c930bc15b13f2dad6f45b4f8f1cfdfbaf/packages/libs/lambda-at-edge/src/default-handler.ts#L285-L285

danjiro commented 3 years ago

I have two routes with revalidate, index and test. Both are identical Index route returns these response headers for the document (always hit):

age: 173
cache-control: public, max-age=0, s-maxage=2678400, must-revalidate
content-encoding: gzip
content-type: text/html
date: Thu, 17 Jun 2021 08:28:38 GMT
etag: W/"e69907006d7e6517940efc5814caa4db"
last-modified: Thu, 17 Jun 2021 02:01:12 GMT
referrer-policy: same-origin
server: AmazonS3
strict-transport-security: max-age=345600; includeSubDomains
vary: Accept-Encoding
x-amz-cf-id: wp0NTBrA7x4ry-drTRKLr_fLrlNmJAVRf6Ao1ARIWhZL7rVECve52g==
x-amz-cf-pop: SYD4-C2
x-cache: Hit from cloudfront
x-content-type-options: nosniff
x-download-options: noopen
x-frame-options: deny
x-xss-protection: 1

index.json (always miss): 

cache-control: public, max-age=0, s-maxage=0, must-revalidate
content-encoding: gzip
content-type: application/json
date: Thu, 17 Jun 2021 08:35:12 GMT
etag: W/"87afdc008c9351575bc028c9305bd9e8"
last-modified: Thu, 17 Jun 2021 02:01:11 GMT
referrer-policy: same-origin
server: AmazonS3
strict-transport-security: max-age=345600; includeSubDomains
vary: Accept-Encoding
x-amz-cf-id: X9DKDE7uXSFRQf0Jf5Hb7SodRPeqdKvg2VnaNrcsDzXOnzNRChjXdQ==
x-amz-cf-pop: SYD4-C2
x-cache: Miss from cloudfront
x-content-type-options: nosniff
x-download-options: noopen
x-frame-options: deny

test route:

cache-control: public, max-age=0, s-maxage=0, must-revalidate
content-encoding: gzip
content-type: text/html
date: Thu, 17 Jun 2021 08:36:31 GMT
etag: W/"10772111471754a612f3626c59797950"
last-modified: Thu, 17 Jun 2021 02:01:12 GMT
referrer-policy: same-origin
server: AmazonS3
strict-transport-security: max-age=345600; includeSubDomains
vary: Accept-Encoding
x-amz-cf-id: UijI9K50ttEdUUQsbhf6V9CvT6kZvJ9e7cJmnRuuqJl0R2OmeTnPOA==
x-amz-cf-pop: SYD4-C2
x-cache: Miss from cloudfront
x-content-type-options: nosniff
x-download-options: noopen
x-frame-options: deny
x-xss-protection: 1

index.json is the same. For both routes if there is a cache miss, the content doesn't get refreshed and is always the same. The messages in SQS also are not consistent. Min TTL is set to 0 in cloudfront. Am I missing some config somewhere?

teban3010 commented 3 years ago

@dphang I did the deploy with the latest version (3.1.0-alpha.0 according to npm) and it's working weird. If I go to the page index it never updates its content and I see on the lambda the same error as before, but if I move to another page, in my case products for example, it updates the content of that page and index page as well.

dphang commented 3 years ago

@teban3010 same error as in the ./pages/.js error? I think that should already be fixed, we are no longer manually building that. Not sure why another page will update it? Maybe is there some SQS processing delay?

@danjiro yeah that is strange, I don't think there's any config issue on your end, but I would recommend if you could add some log statements within the code to see what's being sent through SQS (or not), etc.

Anyway, I will need to debug more extensively over the weekend, but if either of you can provide a minimal reproduction GH repo, that will be helpful too for debugging.

I am also planning to revamp the development guide + add a debugging guide too to help you all self-diagnose (since I'm the only main contributor right now, hard to do it all myself :)

teban3010 commented 3 years ago

I've made a deployment using this repo https://github.com/dapperlabs/nextjs-isr-testing, just add static props on index and adding the same serverless.yml file that I already have, and it worked. As I'm based on Argentina I have some delay on the time I believe that is due to replication on Cloudfront.

@dphang yes, the error is the same, is it possible that there is something old on the deployment? how can i completely remove the deployment, I don't see any stack created so I'm not sure how to delete everything at one, so I can redeploy it from scratch and check if that is the actual issue. Is npx serverless remove enough?

I am also planning to revamp the development guide + add a debugging guide too to help you all self-diagnose (since I'm the only main contributor right now, hard to do it all myself :)

That's good!!

dphang commented 3 years ago

Thanks, will take a look later this weekend

hinsxd commented 3 years ago

Have already been using admin roles so SQS permissions should not be problem. But never able to get ISR working :(

Can someone please provide a simple list of setups to try ISR? I can only ctrl-f the readme.md for "incremental" and don't know if it needs extra configuration...

Also, is it expected to be flanky right now? I was trying to do ISR but it cant revalidate, then I tried gSSP but it always gives the default locale when route change. Now I am forced to use gSP without revalidate and use a CI/CD to rebuild every hour...

teban3010 commented 3 years ago

Just as an update. I've been deploying with the latest version for the last days and also removed the custom policy that I have. Now I see the following behaviour:

dphang commented 3 years ago

@teban3010 sorry I hadn't had much luck when taking a look yet. Are you able to build from source and add some logging statements, it might be helpful to understand where the code is failing.

telmperez commented 3 years ago

I'm using a custom handler defined in my serverless config file that I have in a file called custom.js in the nextjs root directory. Example,

myNextApplication: component: '@sls-next/serverless-component@3.1.0' inputs: handler: custom.handler

However when the "Next.js Regeneration Lambda" deploys it doesn't seem to copy the custom handler code over (which is fine by me). The problem is that the Regeneration Lambda seems to set the custom handler name into the lambda definition instead of "index.handler".

ISR seems to work if I update the Regeneration Lambda handler to "index.handler" instead of the "custom.handler" that I had before.

Not sure if this is a bug but it seems suspicious. Thanks!

igormartimiano commented 3 years ago

@teban3010 sorry I hadn't had much luck when taking a look yet. Are you able to build from source and add some logging statements, it might be helpful to understand where the code is failing.

@dphang I previously got some errors related to permissions but I'm currently stuck at the same point as @teban3010. And I might have some time to build from source and add some logs to it, but I'm not familiar with the project at all. Can you give me some pointers so I can try to help you guys out?

mikejcooper commented 3 years ago

@dphang Firstly, thanks for all the work on this repo, it's working great for our team!

I'm having trouble with ISR, the getStaticProps is acting like getServerSideProps. The example code below is my index.ts file adapted from your original PR with an added sleep for 5s.

Each request to index takes ~5.3s. I was expecting a cached version served back to the client. What am I doing wrong here?

const sleep = (milliseconds: number) => {
  return new Promise((resolve) => setTimeout(resolve, milliseconds))
}

export async function getStaticProps() {
  await sleep(5000)

  return {
    revalidate: 60,
    props: {
      date: new Date().toLocaleString(),
    },
  }
}

const AboutPage = (props: { date: string }) => (
  // ...
  <p>This is the about page, and the time is about {props.date}</p>
  // ...
)

export default AboutPage

Using 3.2.0-alpha.16

dphang commented 3 years ago

@igormartimiano yes, you can edit the code in default-handler (lambda-at-edge package) and look at CONTRIBUTING.md for some help as well. The trigger of the static regeneration from default lambda is here: https://github.com/serverless-nextjs/serverless-next.js/blob/3f7564730810efc17affb545022e5794a6978237/packages/libs/lambda-at-edge/src/lib/triggerStaticRegeneration.ts#L12. And this is the handler that handles regeneration of the s3 page: https://github.com/serverless-nextjs/serverless-next.js/blob/master/packages/libs/lambda-at-edge/src/regeneration-handler.ts. So you can log those places and see what the exact behavior is like.

@mikejcooper sure I can take a look this week, if you have time you can also try to debug following above steps to see the exact behavior

cc: @kirkness if you have any ideas as well.

mxmtsk commented 3 years ago

I'm encountering similar same issues with pages not being refreshed after the set revalidate time, e.g. this page: https://www.voteswiper.org/tr/almanya where State election Mecklenburg-Vorpommern 2021 is still English when it should've been updated to Turkish.

I've checked logs but couldn't find any errors. I've noticed that only the main homepage, so https://www.voteswiper.org/tr is refreshed.

The whole source code is published here: https://github.com/MOVACT/voteswiper-web/

As far as I see, I'm not doing anything out of the ordinary. Any ideas on how to debug this further?

Edit: I've noticed that every request is a Miss from cloudfront and the cache-control header is always cache-control: public, max-age=0, s-maxage=0, must-revalidate, except the main homepage, which has a correct cache-control header - so this looks similar to what @danjiro was/is experiencing. Were you able to fix it?

mxmtsk commented 3 years ago

After debugging further it seems to me that ISR is not working correctly when i specify a bucketRegion - in my case it was set to eu-central-1. When I remove it from my serverless.yml and deploy a fresh new stack ISR works correctly.

I was incorrect. I've debugged so many things that I wrongly assumed this. The actual issue for me was providing paths in getStaticPaths so that it generates some pages at build time. As soon as I returned an empty array, regeneration works.

J3tto commented 3 years ago

Also can confirm I'm also unable to get revalidate working on root / (i.e. homepage) as either a index.tsx or as a [[...path]].tsx file.

But works without issues everywhere else, so its just the root that's broken.

I'm willing to try and have a go at fixing this.

Update: Looks like it gets added to the s3 bucket with the wrong basePath name image

Update 2: After a bit of digging looks like its this line as it doesn't account for the fact the uri might be / which after regex replace results in a empty string and then a .html file which is missing the index. Just needs a explicit check of /. I will update and raise a PR. https://github.com/serverless-nextjs/serverless-next.js/blob/494121a0ebac4b8d6142036515a7758b1aae4efa/packages/libs/lambda-at-edge/src/s3/s3StorePage.ts#L32