performance issue with large project

[krish-dev]

For the large project, strings.ts file became very big. how we can use the namespace concept, also split the file language-wise?

[fwojciec]

For a large project with many languages I would definitely recommend something like https://github.com/isaachinman/next-i18next. next-i18next adds significant additional complexity to be able to handle large projects with multiple languages, etc.. My solution was always intended for smaller projects where something like next-i18next is too much.

I'll leave this open -- perhaps someone else has figured out another solution and will be willing to share it here.

[krish-dev]

next-i18next having a major bug with routing. router event and dynamic routing not working properly. have you ever tried this URL pattern xyz.com/us/en/about with next-i18next?

[fwojciec]

It's been a while since I tried next-i18next. How about a strategy with some sort of an api serving subsets of messages for a particular language/namespace? You'd need to load what's needed to render a particular page in getInitialProps. Just thinking aloud...

[saaymeen]

Hi! Chiming in, I'm curious to find out why this issue has been closed. I came here after days of frustration of getting next-i18next to work and I think it's needless to say that the code provided here is easy to use, understand, and works right out of the box. This issue, and #4 are the only downsides I've encountered so far (besides some super minor bugs and quirks), and I'm sure they can be solved. Im happy to provide any assistence, but first I want to check if @krish-dev has already implemented a functional approach for large(r) translation files, or namespacing? Cheers

[fwojciec]

I'll reopen the issue, since this is a problem, and it would be good to have a place to discuss possible solutions.

[Pryrios]

I am having the same issue. I have implemented this solution on my site mainly for the language detection/locale discovery which is pretty much what I need. (thank you @fwojciec !)

Although my site is (very) small, I don't like storing content on plain files. For the loading of translations I was planning on integrating i18next into this code with the mongoDB database plugin to store/load translations from there.

My plan is to refactor the useTranslation t function to map the i18next.t function. I am not sure on where I should do the initialization. Maybe the HOC or the root index page?

I don't know if this is suitable for large sites but I think i18next can fit the bill nicely and it is not that hard to use.

[BiscuiTech]

Hello! I would like to explore avenues to fix this as I find the solution offered with this example miles ahead of the next-i18n's proposition. Have you tried looking at tree-shaking the languages into different files import? I was thinking maybe each page gets a folder and inside the language file?

I'll try to code something up on my side and maybe open a PR.

[saaymeen]

I think the idea is right, but one language file per page is not the perfect solution since you want to share translations across pages. The namespace approach by next-i18next is pretty sweet. We need to somehow find a way to source the namespaces without requiring a custom server. If @fwojciec is interested, this feature could also lay the way to publish this as npm package some day. Only problem I see with that is how to export the config so that the user can redefine the locales. Just some thoughts.

I surely can help with coding if we get a solid concept. Cheers

[fwojciec]

I've been thinking about this a bit lately, but I haven't done any experiments yet... Thanks to everyone for their thoughts and ideas!

In general, I think that with larger sites which are backed by a CMS it makes sense to store the majority of the translations in the CMS, similar to what @Pryrios has done. The site I work on where this pattern originated in the first place uses an approach like this too. The text strings included directly in the code are just for the UI layer, not the content.

I have tried something similar to what @BiscuiTech is suggesting previously -- using Next's dynamic import to load translations on-demand based on the current language setting, but I couldn't get it to work. I've only spent about an hour playing with this idea, so it's possible that I missed something...

One possible solution would be to use Next's API Routes to create a simple language-serving backend for an app. I haven't played with it yet, but I think it would work, and could possibly support languages/namespaces, etc. and store the translations directly in the API Routes code or in a database deployed elsewhere.

The main idea for this project was to create an internationalization solution that would only rely on Next's native APIs, without any external dependencies and without the need for a custom server. In this sense the possible API Routes solution would fit the spirit of the original example.

I can imagine at least two possible solution using API routes:

• a single API Routes endpoint that serves translation strings based on parameters such as lang and namespace for example;
• a separate API Routes endpoint for every page of the app I think I like the first approach better.

I'm not saying this is the best idea -- but I'll try to experiment with a solution along these lines when I have some free time.

PRs most welcome, in general, as long as the solutions they provide stay true to the original idea of this project (simple/minimalistic and using only Next's native functionality). In either case, I think it would be probably best to create a new page for the example app to showcase the alternative translation fetching strategy we end up implementing, while keeping the original solution in place too.

[fwojciec]

@saaymeen Regarding an npm package idea -- I'm definitely not opposed to it. The only question is what this package should be... It probably doesn't make sense to create an npm package just to install the example app, but I think it would be possible to extract some boilerplate/glue code into a library that would make building apps using this pattern easier.

[fwojciec]

Proof of concept using API routes in the feature/apiroutetranslations branch. The [example site] (https://simple-i18n-example.fwojciec.now.sh) is currently built using this branch.

The main change is loading translation strings from an API instead of including them in the bundle. On every page load/language change translations are fetched from the server and stored in the context. This solution can be deployed to now.sh -- the API gets deployed as a lambda function.

Currently the translations are only split per language, so on every page load the entire set of translations for a given language is loaded. It would be possible, however, to add something like namespaces support as well.

Let me know what you think about this approach.

[BiscuiTech]

Proof of concept using API routes in the feature/apiroutetranslations branch. The [example site] (https://simple-i18n-example.fwojciec.now.sh) is currently built using this branch.

The main change is loading translation strings from an API instead of including them in the bundle. On every page load/language change translations are fetched from the server and stored in the context. This solution can be deployed to now.sh -- the API gets deployed as a lambda function.

Currently the translations are only split per language, so on every page load the entire set of translations for a given language is loaded. It would be possible, however, to add something like namespaces support as well.

Let me know what you think about this approach.

Beat me to it! This is so lean, I love it.

I worked on the dynamic import last night and couldn't make it work myself either. So I'm happy you found an alternative with native /api.

I find this a more elegant solution, so I'll switch my website to use this technique and try to come up with a way to use namespaces on the hoc.

[BiscuiTech]

Opened a PR #6

[rovaniemi]

Proof of concept using API routes in the feature/apiroutetranslations branch. The [example site] (https://simple-i18n-example.fwojciec.now.sh) is currently built using this branch.

The main change is loading translation strings from an API instead of including them in the bundle. On every page load/language change translations are fetched from the server and stored in the context. This solution can be deployed to now.sh -- the API gets deployed as a lambda function.

Currently the translations are only split per language, so on every page load the entire set of translations for a given language is loaded. It would be possible, however, to add something like namespaces support as well.

Let me know what you think about this approach.

@fwojciec and @BiscuiTech I think there is one major downside. When you watch page-source of the example site you will see that there are translations in the HTML, but also in the __NEXT_DATA__ application JSON. So we transfer translations two times to the user (in both the HTML and the JSON content).

Do you have any solutions or ideas that would resolve this problem?

[BiscuiTech]

@rovaniemi I think the next sensible step would be to use the new SSG api from next.js. I haven't had time to look into yet, but I do have a project which will use i18n coming up. I'll see if it's feasible.

[fwojciec]

@rovaniemi -- that's just how the traditional SSR works in Next unfortunately. This is why using namespaces was a good idea when using the approach adopted in this repo.

That being said -- SSG is the way of the future, so the code in this example is getting outdated. https://graalagency.com is currently fully SSG using only a slight modification of the approach in this repo. I'm planning to write an update to the blog post with information how to do it, but I haven't found the time yet...

Basically, on every translated page you do something like:

import React from "react";
import { LanguageProvider } from "../../context/LanguageContext";
import { NextPage, GetStaticProps, GetStaticPaths } from "next";
import Home from "../../components/Home";

interface Props {
  locale: Locale;
}

const HomePage: NextPage<Props> = ({ locale }) => {
  return (
    <LanguageProvider lang={locale}>
        <Home />
    </LanguageProvider>
  );
};

export const getStaticProps: GetStaticProps = async ctx => {
  return {
    props: {
      locale: ctx.params?.lang || "en"
    }
  };
};

export const getStaticPaths: GetStaticPaths = async () => {
  return {
    paths: ["en", "pl"].map((lang) => ({ params: { lang } })),
    fallback: false,
  };
};

export default HomePage;

From that point on everything pretty much just works.

There are plans in Next.js to provide a native solution to i18n in the future, so this is also something to keep in mind.

[BiscuiTech]

@fwojciec I'm curious to see what you do of this new LanguageProvider. Do you port withAPILocale's functionality into a render prop pattern? Basically fetch the locale prop from getStaticProps then shove it into the new Provider, fetch dynamically from the /api and populate the context?

[fwojciec]

@fwojciec I'm curious to see what you do of this new LanguageProvider. Do you port withAPILocale's functionality into a render prop pattern? Basically fetch the locale prop from getStaticProps then shove it into the new Provider, fetch dynamically from the /api and populate the context?

Here's the code of the provider:

import React from "react";
import { useRouter } from "next/router";
import { set, get } from "../services/localStorage";
import { isLocale } from "../translations/types";

/**
 * Language Context
 */

interface ContextProps {
  readonly locale: Locale;
  readonly setLocale: (locale: Locale) => void;
}

export const LanguageContext = React.createContext<ContextProps>({
  locale: "en",
  setLocale: () => null
});

/**
 * Language Context: Provider
 */

export const LanguageProvider: React.FC<{ lang: Locale }> = ({
  lang,
  children
}) => {
  const [locale, setLocale] = React.useState(lang);
  const { query } = useRouter();

  React.useEffect(() => {
    if (locale !== get("locale")) {
      set("locale", locale);
    }
  }, [locale]);

  React.useEffect(() => {
    if (
      typeof query.lang === "string" &&
      isLocale(query.lang) &&
      locale !== query.lang
    ) {
      setLocale(query.lang);
    }
  }, [query.lang, locale]);

  return (
    <LanguageContext.Provider value={{ locale, setLocale }}>
      {children}
    </LanguageContext.Provider>
  );
};

The provider doesn't care about the strings, it's only role is to sync locale value with localstorage and the path. In this particular app translation strings just live in a local module/file that's imported in the "useTranslation" hook. This approach likely doesn't scale well, but hasn't been an issue for me since the majority of translations live in the database anyways and what's hardcoded is just the interface in two languages.

In case there is a lot of translations it would be probably best to fetch the translations needed to render a particular page in getStaticProps and pass that to the component tree.

[rovaniemi]

@rovaniemi -- that's just how the traditional SSR works in Next unfortunately. This is why using namespaces was a good idea when using the approach adopted in this repo.

That being said -- SSG is the way of the future, so the code in this example is getting outdated. https://graalagency.com is currently fully SSG using only a slight modification of the approach in this repo. I'm planning to write an update to the blog post with information how to do it, but I haven't found the time yet...

Basically, on every translated page you do something like:

import React from "react";
import { LanguageProvider } from "../../context/LanguageContext";
import { NextPage, GetStaticProps, GetStaticPaths } from "next";
import Home from "../../components/Home";

interface Props {
  locale: Locale;
}

const HomePage: NextPage<Props> = ({ locale }) => {
  return (
    <LanguageProvider lang={locale}>
        <Home />
    </LanguageProvider>
  );
};

export const getStaticProps: GetStaticProps = async ctx => {
  return {
    props: {
      locale: ctx.params?.lang || "en"
    }
  };
};

export const getStaticPaths: GetStaticPaths = async () => {
  return {
    paths: ["en", "pl"].map((lang) => ({ params: { lang } })),
    fallback: false,
  };
};

export default HomePage;

From that point on everything pretty much just works.

There are plans in Next.js to provide a native solution to i18n in the future, so this is also something to keep in mind.

Hmm, I don't get it. Where do you store the translation files and how you import them in the project?

[fwojciec]

Translations are just a plain JS object stored in a file. The object is imported and used in the useTranslation hook -- it's basically the same implementation as the example in this repo.

[rovaniemi]

Okay! Thanks, I will try it out.

[simonschllng]

From my point of view it depends if you have…

1. many languages to translate to
2. many pages with translated static texts (i.e. not coming from an API / database)

For 1) you do not want to load all languages if the user only needs one; for 2) you do not want to load all pages strings if the user only visits some of them.

My approach for 2) is to add a file XXX.strings.tsx to every component, e.g. Footer.strings.tsx. This is of the same structure as /translations/strings.ts. The useTranslation hook is then extended like so:

export default function useTranslation(localStrings: Strings) {
  const { locale } = useContext(LocaleContext)

  function t(key: string) {
    if (!strings[locale][key] && !localStrings[locale][key]) {
      console.warn(`Translation '${key}' for locale '${locale}' not found.`)
    }
    return strings[locale][key] || strings[defaultLocale][key] ||
      localStrings[locale][key] || localStrings[defaultLocale][key] || ''
  }
…

Every component with lots of strings hands them in. Global/reusable strings stay in /translations/strings.ts.