Include a custom dir in the lambda bundle (i18n public/locales)

[gianpaolom]

I am trying to use serverless-next to deploy a NextJS 9 app that uses also next-i18next. This cause an issue with the Lambda that errors with:

{
    "errorType": "Error",
    "errorMessage": "ENOENT: no such file or directory, scandir '/var/task/public/locales/en'",
    "code": "ENOENT",
    "errno": -2,
    "syscall": "scandir",
    "path": "/var/task/public/locales/en",
    "stack": [
        "Error: ENOENT: no such file or directory, scandir '/var/task/public/locales/en'",
        "    at Object.readdirSync (fs.js:790:3)",
        "    at getAllNamespaces (/var/task/pages/_error.js:48728:19)",
        "    at createConfig (/var/task/pages/_error.js:48733:27)",
        "    at new NextI18Next (/var/task/pages/_error.js:60302:48)",
        "    at Object.k7Sn (/var/task/pages/_error.js:49033:18)",
        "    at __webpack_require__ (/var/task/pages/_error.js:23:31)",
        "    at Module.IlR1 (/var/task/pages/_error.js:12294:12)",
        "    at __webpack_require__ (/var/task/pages/_error.js:23:31)",
        "    at Module.KLg3 (/var/task/pages/_error.js:13398:17)",
        "    at __webpack_require__ (/var/task/pages/_error.js:23:31)"
    ]
}

According to this open bug report a workaround would be including the public/locales/ directory in the lambda bundle.

I have tried to manually inject the directory with a custom build script using build.cmd and build.args into the .next/serverless/ build output directory, however apparently when serverless-next scans the dir to create .serverless_nextjs/default-lambda/ doesn't pick that up.

@danielcondemarin is there a way to include it?

[danielcondemarin]

@gianpaolom I'm not very familiar with next-i18next but could you not include the locale files you need in the page bundles via webpack?

[Accumulative]

@danielcondemarin next-i18next expects the files to exist at that path /var/task/public/locales/en. Including the locale files in the page bundle wouldn't fix that?

I did see as part of https://github.com/danielcondemarin/serverless-next.js/commit/4406ebbb8937c75dfbc5644913b7c0d05ff3a52f the default-lambda folder is cleared before the build command is ran. I think we should be able to copy the locales directly into .serverless_nextjs/default-lambda as part of the build command now. Do you think this is a good workaround?

[gianpaolom]

@Accumulative I managed to make it work hacking

${HOME}/.serverless/components/registry/npm/serverless-next.js@${VERSION}/node_modules/@sls-next/lambda-at-edge/dist/build.js

and adding the instructions to copy the locales and the node modules missing for next-i18next (see: https://github.com/isaachinman/next-i18next/issues/274#issuecomment-624431075)

However clearly it's not an acceptable solution, even more in a context of CI where I have to automate it

[loukmane-issa]

I faced the same issue, and I got my setup working by creating a new Serverless Component extending serverless-nextjs (Serverless Component Documentation). This setup is a good working workaround for me, as it also would fit in the context of a CI. Here is how I achieved it:

Create a serverless.js file:

// serverless.js
const NextJsComponent = require('serverless-next.js/serverless');
const fs = require('fs-extra')

class MyNextJsComponent extends NextJsComponent {
  async default(inputs = {}) {
    if (inputs.build !== false) {
      console.log('-> Building...')
      await this.build(inputs);
      console.log('Building was successful')
    }
    console.log('-> Copying locales directory...');
    this.copyLocales();
    console.log('Locale directory was copied successfully')
    console.log('-> Updating manifest...');
    this.updateNonDynamicManifest();
    console.log('Manifest update successful');
    console.log('-> Deploying...');
    return this.deploy(inputs);
  }

  copyLocales() {
    const localeSrc = './public/locales';
    const localeDest = './.serverless_nextjs/default-lambda/public/locales';
    fs.copySync(localeSrc, localeDest, { recursive: true });
  }

  updateNonDynamicManifest() {
    const manifestFileName = './.serverless_nextjs/default-lambda/manifest.json';
    const manifestJson = require(manifestFileName);
    manifestJson.pages.ssr.nonDynamic['/index'] = "pages/index.js";
    fs.writeFileSync(manifestFileName, JSON.stringify(manifestJson));
  }
}

module.exports = MyNextJsComponent;

As you can see, the default function is reproducing the same behaviour as serverless-next.js component (Link), but it includes two steps between the build:

1. the copyLocales, copy my locales folder into the default-lambda folder, allowing them to be bundled in the package deployed to the lambda
2. updateNonDynamicManifest adds an entry for '/index' in the non-dynamic server side rendered. I found that the manifest.json didn't add an entry for '/index' which was causing 404 on my index page
   • This is not strictly related, but you might encounter the issue.

Then instead of using serverless-next.js component, I used my own component:

# serverless.yml
MyApp:
  component: ./ #path to the folder containing serverless.js file above
  inputs:
    ...

Hopefully this could be useful to you!

If you are committing your .serverless to avoid re-creation of CloudFront and lambdas for every deployment, your own component will change the name of the name of the JSON File. If like me you are committing multiple file for multiple environment, you might have to rename those files.

[asterikx]

2. updateNonDynamicManifest adds an entry for '/index' in the non-dynamic server side rendered. I found that the manifest.json didn't add an entry for '/index' which was causing 404 on my index page
   • This is not strictly related, but you might encounter the issue.

I'm experiencing this as well. I'm using serverless trace mode, and like @loukmane-issa I think this is not related to next-i18next. Getting 404 when entering the bare domain in in the URL bar. When navigating to / via on-site links an my site, everything works.

[asterikx]

@loukmane-issa I'm copying the locales to default-lambda/locales (in my next config I have localePath: path.resolve('./locales'). Translation works fine so far, however I do see error messages in the browser console:

Failed to load resource: the server responded with a status of 404 ()   https://staging.detelling.com/locales/de/common.json

b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1 GET https://staging.detelling.com/locales/de/common.json 404
(anonymous) @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
p @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
value @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
value @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
value @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
value @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
(anonymous) @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
value @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
value @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
value @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
o @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
value @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
u @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
setTimeout (async)
value @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
CYOS.t.default @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
e @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
iMMW @ b8b2303604df4950a432fae48f4cd2d5940a38db.3918e6c22f074e0edf52.js:1
l @ webpack-ccf5ab034a524403276a.js:1
hUgY @ _app-7ff38b7ff2419b721cff.js:1
l @ webpack-ccf5ab034a524403276a.js:1
(anonymous) @ _app-7ff38b7ff2419b721cff.js:1
(anonymous) @ main-9d7cf9c19ecd28d69c7f.js:1
value @ main-9d7cf9c19ecd28d69c7f.js:1
U @ main-9d7cf9c19ecd28d69c7f.js:1
74v/ @ _app-7ff38b7ff2419b721cff.js:1
l @ webpack-ccf5ab034a524403276a.js:1
0 @ _app-7ff38b7ff2419b721cff.js:1
l @ webpack-ccf5ab034a524403276a.js:1
t @ webpack-ccf5ab034a524403276a.js:1
r @ webpack-ccf5ab034a524403276a.js:1
(anonymous) @ 888d5efd0ca2d7d68bc1ca71c03f2c79186091d0.e57c51c763a13cdd550e.js:1
react_devtools_backend.js:2273 Error: STAGING.DETELLING.COM is not a valid domain. Please add it to the cookie consent manager to authorize the domain.
overrideMethod @ react_devtools_backend.js:2273
(anonymous) @ cc.js?renew=false&referer=staging.detelling.com&dnt=false&forceshow=false&cbid=887da060-2c8e-4160-b950-434960d25077&whitelabel=false&brandid=CookieConsent&framework=:1

You have the locales in public/locales, so they are uploaded to S3 as well? I'm not sure where the locales belong in this architecture, Lambda@Edge only or bot Lambda@Edge and S3 (public resources)?

[jamiechong]

@loukmane-issa

If you are committing your .serverless to avoid re-creation of CloudFront and lambdas for every deployment, your own component will change the name of the name of the JSON File. If like me you are committing multiple file for multiple environment, you might have to rename those files.

Can you give any insight on how to setup the .serverless directory of files so that they deploy the same as if using the parent component?

[loukmane-issa]

@loukmane-issa I'm copying the locales to default-lambda/locales (in my next config I have localePath: path.resolve('./locales'). Translation works fine so far, however I do see error messages in the browser console:

@asterikx, yes I have the locales in /public/locales//<...>.json . Yes the locales are in S3, and I believe they will be retrieved from S3 too, however it seems that there is a hardcoded file check requiring the files to be present on the server, hence the lambda.

I haven't checked in deep how it's working, but from what I understand, the manifest.json specify the configuration passed to CloudFront in order to decide which url will go to the SSR Lambda and which one will go directly to the static files in CloudFront. In the case of 404, I would suggest to check directly why CloudFront cannot access the file to S3 directly by going to the CloudFront configuration and checking for this specific URL Path, what would be the behavior (Edge Lambda or S3) and then understanding why the file can't be accessed.

[loukmane-issa]

Can you give any insight on how to setup the .serverless directory of files so that they deploy the same as if using the parent component?

@jamiechong I am not sure if I understand the question, what do you mean by "deploy the same"? As I mentioned above, I created a serverless.js in my root folder and updated the serverless.yml with the code in my above reply. The './' in the serverless.yml will tell serverless that the actual module to use is the serverless.js file.

If you want to understand it in detail, you can check: the Build your own section in: https://github.com/serverless/components

[kylekirkby]

Hi @loukmane-issa,

Thanks for your advice on extending the component. However, after changing my serverless.yml file from:

myNextApp:
  component: "@sls-next/serverless-component@1.18.0"
  inputs:
    timeout: 30
  memory: 1024

to

myNextApp:
  component: "./"
  inputs:
    timeout: 30
  memory: 1024

I can no longer deploy anything and get this error returned:

  Serverless Error ---------------------------------------

  "service" property is missing in serverless.yml

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     Operating System:          linux
     Node Version:              14.10.1
     Framework Version:         2.11.1
     Plugin Version:            4.1.2
     SDK Version:               2.3.2
     Components Version:        3.3.0

Alright, I know what you're thinking. Add a service name.

service: nextjsHosting
myNextApp:
  component: "./"
  inputs:
    timeout: 30
  memory: 1024

then I get hit with:

  Serverless Error ---------------------------------------

  "provider" property is missing in serverless.yml

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     Operating System:          linux
     Node Version:              14.10.1
     Framework Version:         2.11.1
     Plugin Version:            4.1.2
     SDK Version:               2.3.2
     Components Version:        3.3.0

Hmm. Okay, let me add a provider.

service: nextjsHosting
provider:
  name: aws
myNextApp:
  component: "./"
  inputs:
    timeout: 30
  memory: 1024

and now this error:

Serverless: Configuration warning at root: unrecognized property 'myNextApp'
Serverless:
Serverless: Learn more about configuration validation here: http://slss.io/configuration-validation
Serverless:

You can monitor, troubleshoot, and test your new service with a free Serverless account.

Serverless: Would you like to enable this? No
You can run the “serverless” command again if you change your mind later.

I'm guessing that when specifying the provider aws, myNextApp no longer is a valid property... I must be missing something really stupid here right?

Any thoughts would be appreciated!

[kylekirkby]

So it looks like I'm going to have to create a new Serverless component and publish this to their cloud component repository to be able to include the i18n JSON files. Surely this should be a feature supported by this component?

Also, the "./" approach is not documented in the Serverless Components README (maybe it was before but is no longer).

[dphang]

See my reply here: https://github.com/serverless/serverless/issues/8503#issuecomment-727259008.

[dphang]

I believe this issue can be closed now, if you use build.postBuildCommands you can add a command to execute your own script that copies any directories you need into default-lambda.

For extending a component, please see the comment above. I believe as long as you a valid Serverless Component exported from a serverless.js file, then it should work.

[dphang]

Closing as there is a workaround to use the custom build commands to copy the necessary files.

In addition, I also recently added a first-class integration with next-i18next to copy the default next-i18next files: https://github.com/serverless-nextjs/serverless-next.js/blob/675955c17b207b7cdae46fbab51333b79ec9fec5/packages/libs/lambda-at-edge/src/build/third-party/next-i18next.ts. Note that if you use another custom path, you will need to use your own script as above :)