<Trans> component in Typescript files is no longer parsed

[renchap]

This is most probably caused by dfd5db12262ac404ba4cb6414c6e7f6312de14bb (2.6.2)

Version

• i18next: 11.3.6
• i18next-scanner: 2.6.3

Configuration

{
  options: {
    debug: "true",
    lngs: ["en"],
    attr: false,
    func: {
      extensions: [".ts", ".tsx"],
      list: ["t", "props.t", "i18n.t"],
    },
    trans: {
      extensions: [".ts", ".tsx"],
      fallbackKey: true,
    },
    keySeparator: false,
    nsSeparator: false,
  },
}

Sample file

// test.tsx

import * as React from "react"
import { Trans } from "react-i18next"

const TestComp: React.SFC = () => (
  <div className="test-comp">
    <h3>
      <Trans>This is a test</Trans>
    </h3>
  </div>
)

Result

"This is a test" is not present in the output. The strings using the t() helper function are still correctly extracted.

[beheh]

Yeah, we ran into the same issue for out project. The problem is that the scanner now uses a full-blown ES6 parser instead of the primitive RegExp. The parser breaks if it encounters any Typescript syntax that isn't valid ES6 (like type annotations).

Fortunatenly, the solution is simple: just transpile your code to Javascript using tsc and run the scanner over that.

[Tsury]

For me it fails on arrow functions as well. As for Typescript, I tried to add a @babel/plugin-transform-typescript plugin to remove Typescript but it was not removed.

Reverting back to 2.6.1 for now...

[renchap]

I think this makes the extraction fail for any non-ES5 input, eg if you are using Typescript, or some recent non-ES5 features.

One solution could be to have some hooks so you can pre-process the files and run the parser on the ES-5 output:

• one hook called before each file, so you can run Babel on the file and feed the output to the scanner
• one hook called before any file is processed, so you can invoke tsc, and have the scanner process the output directory

Another solution (probably easier) would be to change the README and tell people to change their workflow to build their application using whatever build tool they are using, and run the scanner over the result. However, I am not sure if this will work correctly if the output is minified/optimised, so most probably not using a production build for extracting I18n data.

[beheh]

I think especially if you use Typescript, the solution is pretty simple (you don't need a babel transform): Just run Typescript itself over your codebase and let it deal with all the modern non-ES5 features by specifying ES5 as the target. We use this pattern in our codebase, and we extensively use arrow functions, and very recent features such as rest spread parameters. Typescript can handle and transpile these (as it includes ES5-valid transforms out of the box).

A suitable command line might be tsc --outDir build/locale-extract/ --target ES5 --jsx preserve (note that you want to preserve JSX so that i18next-scanner picks it up).

[renchap]

Yes this is not complex, but it needs to be documented in the README imo. Also this change has been introduced in a minor version, and it broke things for many people.

[cheton]

@beheh I think it might be useful if we can provide built-in support for transpiling TypeScript to ES5 if the file extension is .ts or .tsx.

[Tsury]

It does not work properly for me, even after doing everything mentioned here. Seems like it misses complex uses. All of my tooltip texts are <Trans> components which are supplied to the tooltip component's content attribute, and those are missing from the target files.

Example:

<Tooltip content={<TooltipTextContainer><Trans i18nKey="settings:behavior.exampleTooltip" parent={'span'} i18n={i18n} t={t}>Some example text.</Trans></TooltipTextContainer>}>
  <InfoIconStyled />
</Tooltip>

[beheh]

@Tsury Are you on v2.6.4? It was released yesterday and fixes components inside JSX props.

[Tsury]

@beheh positive. I used ncu to update just before I tried to fix it and it showed v2.6.4.

Also checked my stash, yeah it's v2.6.4 100%.

[cheton]

@Tsury @beheh I can confirm it is a missing test in v2.6.4, I will re-open https://github.com/i18next/i18next-scanner/issues/90. You can update your examples there to help increase the coverage. Thank you.

[cheton]

@Tsury You can try v2.6.5 to check if it works for you.

[Tsury]

@cheton just checked, still no go...

I'll show more of the code:

  <Container welcome={welcome}>
        {this.renderLogo()}
        {this.renderPrevMatchButton()}
        <ButtonsContainer>
          {this.renderErrorSymbol()}
          {this.renderUpdateAvailable()}
          <OpacitySliderStyled white={welcome}/>
          <Tooltip
            open={unreadUpdate}
            content={unreadUpdate ? <Trans i18nKey="menubar:tooltips.update" parent={'span'} t={t} i18n={i18n}>App has been updated! Click here to find out what&#39;s new.</Trans> : <Trans i18nKey="menubar:tooltips.changelog" parent={'span'} t={t} i18n={i18n}>Changelog</Trans>}>
            <SvgButtonStyled
              onClick={onChangelogClicked}
              white={welcome}
              flash={unreadUpdate}>
              <MdCardGiftcard size={16}/>
            </SvgButtonStyled>
          </Tooltip>
        </ButtonsContainer>
      </Container>

[fcastilloec]

I'm having the same problem with v2.6.5, I had to reverse to v2.6.1 because acorn-jsx-walk is not working for Trans components. I don't use typescrypt, I'm using ES6 for my code so that's the "problem". Here's my example on a .jsx file:

render() {
  return (
    <Dialog onClose={this.closeDialog} className="upgrade-dialog">
      <h3>{t('wantMoreSkips')}</h3>
      <p>
        <Trans i18nKey="getUnlimitedSkips">
          Get unlimited skips with <span className="pro-badge">Pro</span>!
        </Trans>
      </p>
      <Button className="button" onClick={this.closeDialog} text={t('gotIt')} />
    </Dialog>
  );
}

On version <=2.6.1 that would show up on my language files, and now it's not. I don't think this issue should be marked as an enhacement, this is a bug that broke things for many people.

[Tsury]

Any news regarding this issue? Still happening in 2.6.6.

[Coriou]

Same problem, not limited to TS as mentioned by @fcastilloec

[daliusd]

Here is workaround for this problem in case you are using some new JS stuff (e.g. arrow functions). Sorry TS guys :-)

Add following to your i18next-scanner.config.js:

const fs = require('fs');

module.exports = {
    options: {
        ...
        // this code is to disable default attempt to parse Trans
        trans: {
            component: 'Trans',
            extensions: [],
        },
        ...
    },
    transform: function customTransform(file, enc, done) {
        const parser = this.parser;
        const content = fs.readFileSync(file.path, enc);

        const options = {
            presets: ['babel-preset-stage-2'],
            plugins: ['babel-plugin-syntax-jsx'],
        };

        let code = require('babel-core').transform(content, options).code;

        parser.parseTransFromString(code);

        done();
    },
};

You need to install some packages:

npm i -D babel-preset-stage-2
npm i -D babel-plugin-syntax-jsx

I think this somehow could be integrated into i18next-scanner itself.

This has revealed number of other problems with scanner:

• If Trans content looks like this "Test {{ formatCount }} Test Z" scanner generates string: "Test <1><0>{{formatCount}} Test Z". What <0> points at? It looks unnecessary if we follow documentation (https://react.i18next.com/components/trans-component) and removing <0> does not harm.

• If Trans is missing i18nKey then scanner does not extract anything. i18nKey is optional and scanner should handle this case as well.

[daliusd]

If you are using newer babel (e.g. newer React) then you might want to use babel 7 style plugins, e.g.:

const options = { plugins: ["@babel/plugin-syntax-jsx", "@babel/plugin-proposal-class-properties"] };

It might be not enough and you will need to follow suggestions from error messages.

[daliusd]

This (almost) works with newer babel: plugins: [ '@babel/plugin-syntax-jsx', '@babel/plugin-proposal-class-properties', '@babel/plugin-proposal-object-rest-spread', '@babel/plugin-transform-spread', '@babel/plugin-transform-arrow-functions', ],

Remaining problem is that acorn-jsx stops on JSX comments, e.g. {/* comment *.}. Maybe newest acorn version would solve the problem.

[messerm]

I have the same problems described above for pure JavaScript code where we use object decomposition {...ohterObject}. I was checking the code and found that acorn-jsx-walk is 3 years old and still uses on acorn-jsx@3.0.0. I wonder whether it would be worth to switch a more recent walker which works with the latest version of acorn-jsx@5.0.1?

Edit: I see this is what @daliusd basically proposed in his latest post :).

[ctavan]

@messerm @daliusd and for those interested I've proposed a patch for the latest acorn-jsx in #112

If you can't wait, try npm install @contentpass/i18next-scanner.

Would be curious to hear if this solves your issues as well.

[fcastilloec]

@ctavan I can confirm that your version solved the issue for me. Thanks!

[ctavan]

Since my patch has been released, can you check again with i18next-scanner@2.8.0?

Should solve most issues with modern JavaScript. Typescript will likely require some more love.

[daliusd]

@ctavan your fix solves all my problems with modern js.

[peitschie]

For anyone looking for hints towards a typescript solution, it appears possible using the typescript compiler api: https://github.com/Microsoft/TypeScript/wiki/Using-the-Compiler-API

An untested example of a custom transform function might be something like:

function transform(file, enc, done) {
  const extension = path.extname(file.path)
  const parser = this.parser;
  let content = fs.readFileSync(file.path, enc);

  if (extension == '.ts') {
    content = typescript.transpileModule(content, {
      compilerOptions: tsCompilerOptions,
      fileName: path.basename(file.path)
    }).outputText;
  }

  if (options.attr.extensions.indexOf(extension) !== -1)
    parser.parseAttrFromString(content);

  if (options.func.extensions.indexOf(extension) !== -1) {
    parser.parseFuncFromString(content);
  }

  done();
}

[nucleartux]

I made package from @peitschie comment. Feel free to PR. https://github.com/nucleartux/i18next-scanner-typescript

[koalalorenzo]

This is still an issue, Any plan to make it happen? :)

I have also experienced problems when using decorators.

[daliusd]

@koalalorenzo Have you tried using @peitschie or @nucleartux solutions? Both are working. Honestly, I don't see any reason why general purpose tool should handle Typescript specifically (even if it is becoming de facto language chooise for many JS projects).

[koalalorenzo]

@daliusd yes, I had to manually modify it. It would be nice to have it in place. I guess we all are a little opinionated

[moweiwei]

@daliusd ,I tried @nucleartux solution, but it cannot works.

[daliusd]

@daliusd ,I tried @nucleartux solution, but it cannot works.

I am using that solution at least in two projects and it is working fine. You should at least show your config and errors you are getting. You must have configuration error somewhere.

[moweiwei]

@daliusd I use i18next-scanner with grunt, I works well except Trans componet. I cannot scanner the 'Log In' text,i don't know why.

Gruntfile.js

const typescriptTransform = require('i18next-scanner-typescript');

module.exports = function(grunt) {
  // Project configuration.
  grunt.initConfig({
    i18next: {
      dev: {
        // src/pages/Overview/OverviewPage.tsx
        src: ['src/**/**/*.{tsx,ts}', 'src/**/*.{tsx,ts}', 'src/*.{tsx,ts}'],
        dest: 'src',
        options: {
          lngs: ['en', 'zh'],
          // removeUnusedKeys: false,
          resource: {
            loadPath: 'src/locale/{{lng}}/{{ns}}.json',
            savePath: 'locale/{{lng}}/{{ns}}.json'
          },
          func: {
            list: ['i18n.t', 't'], // Use an empty array to bypass the default list: i18n.t, i18next.t
            extensions: ['.ts', '.tsx']
          },
          trans: {
            component: "Trans"
          },
          transform: typescriptTransform({ extensions: [".tsx"] }),
          defaultValue: (lng, ns, key) => {
            if (lng === 'en') {
              return key; // Use key as value for base language
            }
            return ''; // Return empty string for other languages
          }
        }
      }
    }
  });

  // Load the plugin that provides the "i18next" task.
  grunt.loadNpmTasks('i18next-scanner');

  // Default task(s).
  grunt.registerTask('default', ['i18next']);
};

my componet

import { Trans } from 'react-i18next';

        <Trans>Log In</Trans>

[daliusd]

transform: typescriptTransform({ extensions: [".tsx"] }),

This should be at the same level as options not in options.

[moweiwei]

@daliusd I have move the transform to the same level as options, but it also cannot scanner the Trans, thank you for your quick answer.

[daliusd]

@daliusd I have move the transform to the same level as options, but it also cannot scanner the Trans, thank you for your quick answer.

Try enabling debug, maybe that will give idea what might be wrong. Try adding console.log statements to see what is passed around.

[Bnaya]

what i've ended up doing, is to transpile typescript -> js to a temp directory, and run the scanner on it.

[martinratinaud]

I made package from @peitschie comment. Feel free to PR. https://github.com/nucleartux/i18next-scanner-typescript

Works very well thank you.

Also, make sure you have not put any ['.ts', '.tsx'] extensions from within the options, which would display the same error message

{
  input: ['src/**/*.{js,jsx,ts,tsx}'],
  options: {
    func: {
      extensions: ['.js', '.jsx'],
    },
    trans: {
      extensions: ['.js', '.jsx'],
    },
  },
  transform: typescriptTransform({ extensions: ['.ts', '.tsx'] }),
}

and not

{
  input: ['src/**/*.{js,jsx,ts,tsx}'],
  options: {
    func: {
      extensions: ['.js', '.jsx', '.ts', '.tsx'],
    },
    trans: {
      extensions: ['.js', '.jsx', '.ts', '.tsx'],
    },
  },
  transform: typescriptTransform({ extensions: ['.ts', '.tsx'] }),
}

[ekeuus]

I made package from @peitschie comment. Feel free to PR. https://github.com/nucleartux/i18next-scanner-typescript

Works very well thank you.

Also, make sure you have not put any ['.ts', '.tsx'] extensions from within the options, which would display the same error message

{
  input: ['src/**/*.{js,jsx,ts,tsx}'],
  options: {
    func: {
      extensions: ['.js', '.jsx'],
    },
    trans: {
      extensions: ['.js', '.jsx'],
    },
  },
  transform: typescriptTransform({ extensions: ['.ts', '.tsx'] }),
}

and not

{
  input: ['src/**/*.{js,jsx,ts,tsx}'],
  options: {
    func: {
      extensions: ['.js', '.jsx', '.ts', '.tsx'],
    },
    trans: {
      extensions: ['.js', '.jsx', '.ts', '.tsx'],
    },
  },
  transform: typescriptTransform({ extensions: ['.ts', '.tsx'] }),
}

Thanks! For anyone else running into this - double check that you don't have TS extensions in the non-transform configuration :)